A Report On Human Activity Recognition Systems And Methods

Introduction

Human Activity Recognition systems has been an important area of research in field of human behavior analysis recently. Researchers have been applying many machine learning techniques to study different simple and complicated activities. Ambulant activities like, sleeping, walking, jogging, sitting, walking upstairs, walking downstairs are monitored and useful feedback is provided by HAR system. For elderly citizens it is used to help in detecting various illness, fall detection, rehabilitation. HAR systems are also used in entertainment, gaming and surveillance in smart homes. HAR framework captures activities such as ambulatory, postural, different human actions and body movements by using variety of sensors of different modalities(Cao et al. , 2017).

These sensors include wearable motion sensors, video cameras, acoustic sensors, RADAR, Echo, Wifi, magnetic sensors, infra red motion detectors (Ramasamy Ramamurthy & Roy, 2018)and so on. Recent alternative proposed methods are use of social network methods(Jia et al. , n. d. ) that utilize information from various social network sites. To understand human behavior and wireless signal based human activity recognition (Savazzi et al. , 2016) which utilizes signal produced by the wireless devices to categorize human activity. Despite the presence of different methods, use of sensor data from wearable devices and smartphones is a major research area in activity recognition, detection and monitoring because of its advantages in comparison to other sensor modalities (Nweke et al. , 2018).

Mobile phones and wearable sensors have obvious advantages over other methods in HAR due to their unobtrusiveness, ubiquity, cost effectiveness and ease of usability. Video camera based HAR system are widely used for security purposes but they have issues related to privacy, fixed installation, space constraints and capture non target information(Yang et al. , 2015). This system’s performance is also dependent on variable lighting conditions and suffers due to visual disturbances(Wang, 2016). Method of feature extraction in video sensors is Spatio-temporal interest Point (STIP), Histogram of Oriented Gradient (HOG) and Region of Interest (ROI), in comparison to statistical and frequency based features used by mobile sensors to track activities which have better computational time and less complexity (Figo et al. , 2010).

Methods applied in HAR domain

This part should contain sufficient detail so that all procedures can be repeated. It can be divided into subsections if several methods are described.

Smartphone and Wearable Sensors Data Applied with Traditional Machine Learning Methods

As in Modern smartphones collect data of daily human activities from a wide range of sensors enabling researchers to further the research in field of HAR. Smartphones are now equipped with multi sensor system having sesnsors such as accelerometer, gyroscope, magnetometer, Bluetooth, proximity, light, wifi, microphones and cellular radio sensors. Accelerometers, gyroscope magnetometer group is also referred as motion sensors and they provide us informative data to track movement and recognition of activities of daily living(ADL). The motion sensors when grouped with GPS and heart rate can be used for coarse grain and content activity recognition, social interaction of users and their location. Proximity and light sensors in cellphone can also deploy if user is in dark or light place (Sensing, 2015).

For healthy living in elderly citizens and their assisted living barometers, thermometers, air humidity and pedometers sensors are being used to check ambient conditions and used for preventive measures(Gong et al. , 2012). Recently, in September 2018 Apple has launched their new smartwatch which is equipped with an electrocardiogram(ECG) monitor which claimed to detect low heart rate and irregular heart rhythm. Despite the above mentioned advantages of wearable and mobile sensors and development of various existing and new sensors, the wearable and mobile sensors have also been facing various challenges like intraclass variability, class imbalance, interclass similarity, deciding precise start and finish time of each activity(San et al. , 2017) heterogeneities across the sensing devices, and device positioning. Intraclass variations are significant when data collected by many users for same activity is not similar. Interclass variability arises when data from two different activities such as jogging and running is similar to each other. When comparing two activities like walking and jogging and former is of more duration than later, class imbalance may occur and while continuous monitoring of user it is difficult to decide the exact start and end time of one activity (Ramasamy Ramamurthy & Roy, 2018).

Researchers are facing challenges in HAR domain because of difficulty in classifying different activities. Before classifying, various phases like preprocessing, data segmentation, identification and extraction of discriminative features are applied to data. For preprocessing, methods such as non linear, low and high pass filter, Laplacian and Gaussian filter are used. To achieve sensor data segmentation methods used are sliding windows, event or energy based activities (Bulling et al. , 2014). Segmented data is next used to extract relevant feature vectors and in order to recognize the activities, these features are further reduced through feature selection methods to select the most differentiating features. With the application of various dimensionality reduction methods, the dimension of extracted features are reduced to increase the computational efficiency. In HAR widely used such methods are principal component analysis (PCA), linear discriminate analysis (LDA) and empirical cumulative distribution functions (ECDF) (Bilal et al. , 2018). Traditionally at this stage the processed data and extracted features are now applied to various machine learning or pattern recognition methods (Bulling et al. , 2014). These machine learning techniques include the Support Vector Machine, Decision Tree, naive Bayes, Hidden Markov Model, K-Nearest Neighbour (KNN) and Gaussian Mixture Model (Nweke et al. , 2018).

Challenges and Limitations of Machine Learning Methods

In HAR domain, the conventional pattern recognition(PR) approaches with machine learning algorithms has made significant progress and in some controlled environments, have achieved satisfying results(Wang et al. , 2018). These methods are heuristic driven, require feature engineering from data and majorly dependent on human domain knowledge(Yang et al. , 2015). This limits the model developed for one domain to extend to other domain. Subsequently, shallow features learned by these approaches will lead to weak performance for incremental and unsupervised tasks(Wang et al. , 2018) and not effective in capturing complex activities consisting of a series of several micro activities (Faridee et al. , 2018). In contrast to the traditional PR methods, recently developed deep learning methods are able to learn high level meaningful feature by exercising end to end neural network. Moreover these methods such as convolutional neural network (CNN) is able to recognize complex activities and more apt to process unsupervised and incremental learning(Wang et al. , 2018).

Results and Discussion

In HAR, the most popular method consists of CNN, deep neural network (DNN) and recurrent neural networks (RNN) with long short term memory(LSTM) RNN networks. There have been many research where different deep learning methods are being applied to various HAR problems and other diseases, refer table a. Various models based on the above mentioned neural networks are created and have been outperforming state of the art results for many publicly available datasets like OPPORTUNITY(Guan & Plötz, 2017; Yang et al. , 2015; Murad & Pyun, 2017), PAMAP2 (Guan & Plötz, 2017; Twomey et al. , 2018), UCI smartphone (Inoue et al. , 2018; Ignatov, 2018; Steven Eyobu & Han, 2018) and so on as shown in table a.

Convolutional neural network(CNN)

CNN is deep neural network with multiple interconnected layers and structures. It is one of the most popular technique and has found multiple successful application areas like image classification, speech recognition, sentence modeling apart from HAR. When applied to time series data it has two advantages scare invariance and local dependency over other methods (Wang et al. , 2018). The recent research which use CNN techniques includes (Ignatov, 2018) which shows that CNN based models can be applied to get real time application to detect activities of daily learning ADL. This model was created to use shallow CNN for unsupervised feature extraction while showing time series length dependency on activity recognition accuracy. This is a platform independent model and can be used on various datasets as done in the study. The main advantage of this model over other models is that it can use short interval size of 1 sec almost zero feature engineering and preprocessing on data, because of shallow architecture can be used in smartphone in real time.

Also in this study cross dataset experiment shows that this model’s platform independent architecture and successful application to differently calibrated accelerometers. For gait recognition (Hannink et al. , 2017), CNN based models are very successful in recognizing activity on and can be used to detect other mobilty based diseases like Parkinsons disease. To make CNN based models to implement in real time mobile devices another study done by (Ravi et al. , 2017) which is a hybrid CNN model and combines the deep learning features with shallow feature, in this case preprocessing of raw data by spectral domain preprocessing method. In this model raw data is feed to both deep learning features and shallow features parallelly and later merged using soft max and fully connected layers. This model gives better accuracy than traditional method and can be used for real time processing of HAR in real time(Ravi et al. , 2017). Other major challenge in HAR for wearable sensors is the accuracy in classification of activities. (Jordao et al. , 2018) proposes a novel architecture to learn patterns in accelerometer axes in all layers of CNN. With help of data augmentation to learn signal pattern, specific novel features are devised to discriminate various activities. This method outperform traditional handcrafted feature methods.

Deep Neural Network(DNN)

Deep Neural Network(DNN) is a multilayered version of artificial neural network (ANN). ANN generally don’t have many layers(shallow) but DNN consist of many hidden layers and make it a superior approach to learn from complex and large data(Wang et al. , 2018). One of the first study with 5 hidden layers in DNN model was done by (Hammerla, 2016) which proved that auto classification and auto feature learning by such hidden layer model has better performance in comparison to shallow models. DNN consist of a series of non linear transformation of the raw accelerometer data(Hammerla, 2016). Many DNN based models have been researched and applied in posture detection, Alzheimer’s hand activity gesture recognition and HAR(Nweke et al. , 2018). A recent study by(Hassan et al. , 2018) uses DNN based model, which extract initial feature from original raw data, perform KPCA & LDA, Kernel pricipal component analysis, Linear discriminant analysis to extract robust features and use these robust features data to train DNN which outperforms the traditional approach. Its performance in accurately recognizing various activities on UCI data was impressive. (Yoo & Oh, 2018) used DNN based model for fall detection using acceleration sensor tied to wrist. Historicaly waist sensor has better accuracy then wrist based sensors. DNN used with google tensorflow. Three factors impacted the accuracy, choice of activation function (sigmoid vs ReLU), choice of input data(3 axis raw data vs SMV value), choice of initializer(for missing values either fill zero or scaled data). 100 percent accuracy reached although the study was tested only on reverse falls cases.

Recurrent Neural Network(RNN)

Some research suggests that the traditional methods for HAR prediction have many variable activities (such as feature extraction, time series data segmentation, position of sensor on body) and if used and tested in different settings may give similar or in some cases better accuracy than deep learning methods(Twomey et al. , 2018). In this research when CNN & LSTM model are compared to Empirical Cumulative Distribution Function (ECDF) and statistical features for some classification performance factors like “independently and identically distributed” iid & Conditional Random Field (CRF) using datasets HAR, PAMAP, USCHAD the result is mixed declaring no clear winner.

Conclusions

This should clearly explain the main conclusions of the work highlighting its importance and relevance. AcknowledgmentsAll acknowledgments (if any) should be included at the very end of the paper before the references and may include supporting grants, presentations, and so forth. he paper during submission, peer review process, till publication.