Files
Abstract
With the widespread usage of machine learning (ML), the trustworthy application of ML has gained increasing attention. An ML model integrated into a system is playing a critical decision-making role. Simultaneously, the concerns of security and trustworthiness have been raised by using machine learning techniques in practice. In this dissertation, we identify two research challenges for building secure and trustworthy machine learning applications and propose solutions to address them. Initially, most ML techniques are designed with assumptions that data is correct and running environments are benign. However, these assumptions might not guarantee in the wild. It is challenging to build a trustworthy ML-based system due to the characteristics of ML models, non-stationary run-time environments, and the large surface of attacks along the ML lifecycle.
The first problem is contamination data that causes the counterfactual prediction of an ML model. Data collected from various sources and preprocessed by many processors can be contaminated by equipment malfunction, human errors, or even malicious attacks. We propose an Automated Contaminated Attribute Localization (ACAL) system to pinpoint the faulty attribute in a contaminated data. ACAL quantifies each attribute's suspiciousness automatically, which helps users trace the root cause of the data contamination. Our evaluation of ACAL on real-world datasets and distinct models shows that ACAL can reach over 91% accuracy of localization.
The second problem is a violation of model integrity when users outsource an ML model to a cloud. The ML model has at the risk of being modified by dishonest cloud providers and attackers, which leads to incorrect predictions. We propose an Integrity Checking for Neural Network approach to detect the model modification as a normal query. A novel way to generate querying samples is presented. We evaluate our approach on different cases of model modification and validate its effectiveness.
In this dissertation, we argued that estimating ML application risks and preparing for ML task failure is crucial for the development of reliable ML-based applications. We considered ML risks from different perspectives and proposed solutions towards trustworthy applications of ML in two scenarios: contamination data and model integrity violation.
The first problem is contamination data that causes the counterfactual prediction of an ML model. Data collected from various sources and preprocessed by many processors can be contaminated by equipment malfunction, human errors, or even malicious attacks. We propose an Automated Contaminated Attribute Localization (ACAL) system to pinpoint the faulty attribute in a contaminated data. ACAL quantifies each attribute's suspiciousness automatically, which helps users trace the root cause of the data contamination. Our evaluation of ACAL on real-world datasets and distinct models shows that ACAL can reach over 91% accuracy of localization.
The second problem is a violation of model integrity when users outsource an ML model to a cloud. The ML model has at the risk of being modified by dishonest cloud providers and attackers, which leads to incorrect predictions. We propose an Integrity Checking for Neural Network approach to detect the model modification as a normal query. A novel way to generate querying samples is presented. We evaluate our approach on different cases of model modification and validate its effectiveness.
In this dissertation, we argued that estimating ML application risks and preparing for ML task failure is crucial for the development of reliable ML-based applications. We considered ML risks from different perspectives and proposed solutions towards trustworthy applications of ML in two scenarios: contamination data and model integrity violation.