Reducing Complexity of Deep Learning for Time Series Classification Using New Hand-Crafted Convolution Filters

Abstract

Deep learning for Time Series Classification (TSC) has become one relevant subject in the literature for this task. It is used for wide applications in multiple domains ranging from medical data, action recognition and robotics. In the last decade, Convolutional Neural Networks (CNNs) have shown to be the best base architecture to use when dealing with deep learning for TSC ever since the release of the UCR archive, the largest repository for TSC datasets. The UCR archive includes a variety of 128 datasets of univariate time series data, where the task is to correctly classify the samples to their corresponding annotation. Deep learning models face two main challenges. The first one is represented by overfitting and the consequent incapacity of generalizing to new unseen samples. With CNN based architectures, this is commonly due to the fact that the learned filters tend to detect specific patterns in the training set instead of generic ones. The second challenge is complexity wise, which limits its usability in real world scenarios such as embedded systems. In this work, we propose to address these two challenges with one solution: hand-crafting some generic non-learned convolutional filters to detect generic patterns. These hand-crafted filters can replace the usability of the first layer in the CNN model, resulting in a significant reduction in the number of parameters. The proposed architecture is evaluated on 128 datasets of the UCR archive and the results reveal a significant improvement inperformance compared to other approaches as well as the reduction in terms of complexity.