Roland Stolz

Advisors
Maike Stöve (M.Sc.), Markus Gambietz (M.Sc.), Prof. Dr. Björn Eskofier, Cristine Agresta (MPT, PhD)

Duration
07 / 2022 – 01 / 2023

Abstract

Monitoring athlete movement pattern is an important skill of a coach that wants to optimize the performance of his athletes [1]. Traditionally, this is done with movement screens [2]. Due to their subjective nature, this way of monitoring is prone to personal bias and cannot be scaled to many athletes or applied reliably over time [3]. Data-driven algorithms remove those issues and hence can increase the objectivity of movement observations [4]. The improved athlete monitoring data provides useful information for coaching and training decisions. It can also be used to detect when individual athletes are outside their typical pattern profile [5]. Consequently, it can increase performance development and reduce the risk of injuries for athletes [1, 3, 6]. Machine learning algorithms that focus on athlete identification also have the potential to detect new movement characteristics that are representative of the athletes and are overlooked in a subjective analysis [4].

Human activity recognition (HAR) focuses on the algorithm-based classification of different sports activities based on movement data [7, 8, 9]. Optical motion capture systems provide high-quality input data because they are very accurate to position. However, this technology is expensive and location constrained. Non-vision based inertial measurement unit (IMU) sensors serve as a cheaper and more versatile surrogate that can still provide accurate data [1, 3, 9, 10].
A different algorithm objective is individual athlete classification, which is a less common research area, compared to HAR. Ross et al. classified elite and novice athletes in different tasks, based on optical tracking and IMU sensor data [5]. For identifying individual people, multi-class classification with neural networks can be used, where every athlete is assigned to one class. The downside of this method is that including new athletes into the identification system would require a new network architecture with one more output node and hence also retraining the network, which strongly reduces its applicability in practice. This problem was addressed by Schroff et al. [11] in the context of face identification. They trained a model with the so-called triplet loss to produce a lower-dimensional representation of the input. The classification is then performed by calculating the distance between the embedding of the input and an embedding of representative samples for each other person in a database. The person associated with the lowest distance is then assigned to the input [11]. With that method, including new athletes entails only adding a sample of them to the database. There are many other approaches to computing a representative lower dimensional embedding of the input sensor data. Classical pattern recognition algorithms, like principal component analysis (PCA) [12] or deep neural networks, like variational autoencoder (VAE) [13] can be used.

Most of the sports performance evaluations based on IMU sensor data focuses on the direct computation of specific parameters, like swing velocity or bat orientation for baseball [1]. For example, Ghasemzadeh et al. applied clustering techniques to IMU data from multiple sensors to evaluate the timing of a baseball swing [14]. Ahmadi et al. estimated hip and knee angles from IMU sensor data to compare the movement of injured and healthy athletes [15]. The lower-dimensional representation of IMU sensor data from specific movements, produced by pattern recognition algorithms, can serve as a fuzzy performance evaluation measurement of latent parameters. If it is representative, it can be seen as an athlete’s individual profile. Based on lower-dimensional embeddings from PCA, Remedios et al. [16] explored the identification of different phenotypes for deep squat and hurdle step, and Ross et al. [4] analyzed objective differences in movement patterns between athletes. However, there is currently no research published to evaluate the lower-dimensional representation produced by deep neural networks for any specific movement data. Only closely related, Converse et al. applied VAEs to baseball players’ season-long batting statistics to compute a profile of latent skills that a professional athlete needs to succeed in major league baseball (MLB) [17].

Several gaps in research emerge from this analysis:
• It is not clear whether IMU sensor data from specific movements provides sufficient information to identify individual athletes
• Person identification techniques from face recognition have not been applied to identify athletes based on their movement data
• All of the research focuses on creating a lower-dimensional representation with classical pattern recognition algorithms instead of deep neural networks
• It is not clear what information can be extracted from this lower-dimensional representation of movement data, in terms of guiding training decisions or monitoring atypical movement patterns

Based on this, the thesis aims to answer the following three research questions:
1. Can deep learning algorithms be used to identify individual athletes based on their movement data?
2. Can the lower-dimensional representation of the input, produced by those algorithms, provide additional information about the athlete? (E.g. atypical movement patterns, injury rehab progression, athlete clustering)
3. What type of input data is required for the algorithm to work? (E.g. sensor amount and position, sampling rate, pre-processing)

References
[1] V. Camomilla, E. Bergamini, S. Fantozzi, and G. Vannozzi, “Trends supporting the in-field use of wearable inertial sensors for sport performance evaluation: A systematic review,” Sensors, vol. 18, no. 3, 2018. [Online]. Available: https://www.mdpi.com/1424-8220/18/3/873
[2] R. McCunn, K. aus der F¨unten, H. H. K. Fullagar, I. McKeown, and T. Meyer, “Reliability and association with injury of movement screens: A critical review,” Sports Medicine, vol. 46, pp. 763–781, 2015.
[3] E. E. Cust, A. J. Sweeting, K. Ball, and S. Robertson, “Machine and deep learning for sport-specific movement recognition: a systematic review of model development and performance,” Journal of Sports Sciences, vol. 37, no. 5, pp. 568–600, 2019, pMID: 30307362. [Online]. Available: https://doi.org/10.1080/02640414.2018.1521769
[4] G. B. Ross, B. Dowling, N. F. Troje, S. L. Fischer, and R. B. Graham, “Objectively differentiating movement patterns between elite and novice athletes,” Medicine & Science in Sports & Exercise, vol. 50, p. 1457–1464, 2018.
[5] ——, “Classifying elite from novice athletes using simulated wearable sensor data,” Frontiers in Bioengineering and Biotechnology, 2020. [Online]. Available: https://www.frontiersin.org/article/10.3389/fbioe.2020.00814
[6] A. Ahmadi, E. Mitchell, F. Destelle, M. Gowing, N. E. O’Connor, C. Richter, and K. Moran, “Automatic activity classification and movement assessment during a sports training session using wearable inertial sensors,” in 2014 11th International Conference on Wearable and Implantable Body Sensor Networks, 2014, pp. 98–103.
[7] F. Demrozi, G. Pravadelli, A. Bihorac, and P. Rashidi, “Human activity recognition using inertial, physiological and environmental sensors: A comprehensive survey,” IEEE Access, vol. 8, pp. 210 816–210 836, 2020.
[8] C. Jobanputra, J. Bavishi, and N. Doshi, “Human activity recognition: A survey,” Procedia Computer Science, vol. 155, pp. 698–703, 2019, the 16th International Conference on Mobile Systems and Pervasive Computing (MobiSPC 2019),The 14th International Conference on Future Networks and Communications (FNC-2019),The 9th International Conference on Sustainable Energy Information Technology. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1877050919310166
[9] N. S. Khan and M. S. Ghani, “A survey of deep learning based models for human activity recognition,” Wirel. Pers. Commun., vol. 120, pp. 1593–1635, 2021.
[10] A. L. Clouthier, G. B. Ross, and R. B. Graham, “Sensor data required for automatic recognition of athletic tasks using deep neural networks,” Frontiers in Bioengineering and Biotechnology, vol. 7, 2020. [Online]. Available: https://www.frontiersin.org/article/10.3389/fbioe.2019.00473
[11] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 815–823.
[12] I. T. Jolliffe and J. Cadima, “Principal component analysis: a review and recent developments,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 374, 2016.
[13] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” CoRR, vol. abs/1312.6114, 2014.
[14] H. Ghasemzadeh and R. Jafari, “Coordination analysis of human movements with body sensor networks: A signal processing model to evaluate baseball swings,” IEEE Sensors Journal, vol. 11, no. 3, pp. 603–610, 2011.
[15] A. Ahmadi, E. Mitchell, C. Richter, F. Destelle, M. Gowing, N. E. O’Connor, and K. Moran, “Toward automatic activity classification and movement assessment during a sports training session,” IEEE Internet of Things Journal, vol. 2, no. 1, pp. 23–32, 2015.
[16] S. M. Remedios, D. P. Armstrong, R. B. Graham, and S. L. Fischer, “Exploring the application of pattern recognition and machine learning for identifying movement phenotypes during deep squat and hurdle step movements,” Frontiers in Bioengineering and Biotechnology, vol. 8, 2020.
[17] G. A. Converse, B. Arnold, M. Curi, and S. Oliveira, “Variational autoencoders for baseball player evaluation,” in FSDM, 2019.