رسائل ماجيستير
English
2020
Evaluating the Robustness of Deep Learning Models for Time Series Classification
Moataz A Ahmed, ماجد سعيد علي سعيدان
كلية علوم وهندسة الحاسب الآلي-جامعة الملك فهد للبترول والمعادن · السعودية
Time series (TS) analysis has many applications in domains that are very critical for business and society. TS analysis may target different objectives including forecasting, clustering, and classification. Deep learning (DL) models, such as Residual Network (ResNet) and Long Short-Term Memory (LSTM) have been extensively considered for learning and analyzing the different patterns inherent in TS. However, it is known that DL models may not perform as expected under various circumstances that cause changes in the data. These changes could occur naturally due to noise, or could be crafted for malicious purposes, among possible other reasons. It is desirable to evaluate DL models under various circumstances to gain more confidence about their performance in deployment. In this research, we focus on various DL models considered for time series classification (TSC), which is one type of time series analysis, and evaluate the robustness of such models in response to noisy inputs. Our goal is to evaluate the robustness of various types of DL architectures and training methods in response to noisy data as well as missing data. We use different types of perturbation functions to simulate noisy inputs. A major objective of this research is to compare and contrast the effectiveness of different training methods in building robust models for TSC. Our experiments show that there is a significant statistical difference among the different training methods with regard to their impact on the models’ robustness in response to noisy or missing data. In this research, we employ various metrics to assess the robustness of the models as well as the training methods. We compare and contrast those metrics, highlighting some of their limitations. Moreover, we adapt the saliency maps method for interpreting the behavior of deep learning models from being applied to images, to being applied to multivariate time series datasets. We use these saliency maps along with the metrics to perform holistic analysis on various deep learning models to compare and understand their behavior.