Timur Perst

Advisors
Johannes Link (M.Sc.), Maike Stöve (M.Sc.), Prof. Dr. Björn Eskofier

Duration
02/2021 – 08/2021

Abstract
Monitoring the actions of a player in sports helps to prevent injuries from overuse or incorrect techniques and can help to place emphasis on exercises targeting a specific movement [1, 2]. The monitoring can either be done manually by watching a video recording of the event which is
time consuming and needs an expert of the field or by using a human activity recognition (HAR) system. Human Activity Recognition (HAR) is the automated recognition and classification of activities from a continuous stream of input sensors [2]. Numerous papers have dealt with HAR in sports. Most of them tried to classify sport specific movements or give feedback on the execution, like Anand et al. did with analysing strokes in swing sports [3], Brock et al. with giving feedback on style errors of ski jumpers [4] or Kasiri et al. with classifying boxing punches [5]. Cust et al. provide an overview of this research [6]. HAR systems use different kinds of input signals. Mostly these are image sequences (videos), sensor data or a combination of both. Inertial measurement units (IMU) have proven to work well in sensor based HAR. IMUs typically contain accelerometers, magnetometers or gyroscopes which measure continuously along the three axis of space. In the stream of the sensor data HAR systems can detect when an action occurs and typically use machine learning algorithms to try to classify the action [6]. In a paper from 2017 Kautz et al. collected data from volleyball players performing sport specific movements by mounting an IMU at the wrist of the dominant hand of each player and performed a recognition and classification of these movements. The authors discussed several algorithms based on feature selection like a Naïve Bayes Classifier or support vector machines. All these algorithms are outperformed by a Deep Convolutional Neural Network (DCNN) [1].

For any machine learning or deep learning algorithm to perform well a sufficient set of training data is needed. Huge sets of labeled data are generally either very costly to obtain or not available at all. In order to overcome this limit Transfer Learning is used. The idea behind this approach is to project already acquired knowledge to a new domain which not only reduces the time it takes to train the classifier but also reduces the required size of the training dataset [7]. There has been some research which successfully used transfer learning in HAR and proved that neural networks can be trained using a reduced number of training samples [8].

The goal of this thesis is to create a deep neural network which is based on the architecture of the network by Kautz et al. and transfer the knowledge to the classification of typical types of throws which occur in an ultimate frisbee match. The seven types of throws are: forehand inwards
moving throw, forehand outwards moving throw, forehand straight throw, backhand inwards moving throw, backhand outwards moving throw, backhand straight throw and a hammer (a throw over the top of your head). Furthermore a deep neural network which will only be trained on the frisbee data is going to be created. The performance of this network will then be compared with the earlier mentioned, pretrained network.

References:
[1] Kautz, T., Groh, B.H., Hannink, J. et al.: Activity recognition in beach volleyball using a Deep Convolutional Neural Network. Data Min Knowl Disc 31, 16781705 (2017).
[2] Schuldhaus, Dominik: Human Activity Recognition in Daily Life and Sports Using Inertial Sensors. FAU University Press (2019)
[3] Anand, A., Sharma, M., Srivastava, R., Kaligounder, L., & Prakash, D.: Wearable motion sensor based analysis of swing sports. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 261267). (2017)
[4] Brock, H., Ohgi, Y., & Lee, J.: Learning to judge like a human: Convolutional networks for classification of ski jumping errors. Proceedings of the 2017 ACM International Symposium on Wearable Computers – ISWC ’17, 106113. (2017)
[5] Kasiri-Bidhendi, S., Fookes, C., Morgan, S., Martin, D. T., & Sridharan, S.: Combat sports analytics: Boxing punch classication using over- head depth imagery. In 2015 IEEE International Conference on Image Processing (ICIP) (pp. 45454549). Quebec City, Canada (2015)
[6] Emily E Cust, Alice J Sweeting, Kevin Ball & Sam Robertson: Machine and deep learning for sport-specic movement recognition: a systematic review of model development and performance. Journal of Sports Sciences, 37:5 (2019)
[7] Cook, D., Feuz, K.D. & Krishnan, N.C.: Transfer learning for activity recognition: a survey. Knowl Inf Syst 36, 537556 (2013).
[8] Ramasamy Ramamurthy, S, Roy, N.: Recent trends in machine learning for human activity recognition A survey. WIREs Data Mining Knowl Discov. 2018; 8:e1254. (2018)