Lea Grabmann

Advisors
Mohamad Wehbi (M.Sc.), Dr. Dario Zanca, Prof. Dr. Björn Eskofier, Tim Hamann

Duration
08 / 2022 – 01 / 2023

Abstract
The quality of handwriting is an important factor in daily lives, especially among young children who are in the first stages of advancing their handwriting techniques. With the advancements of technology, more students are moving away from traditional handwriting methods and turning into digitally inputting text using handheld devices. However, handwriting problems impact school achievement, and therefore, to ensure that the quality of pupils’ handwriting is adequate, specialized teachers are required to evaluate the students’ handwriting.

Evaluating handwriting is a time-consuming task that requires teachers to check in detail the handwriting inputs of the students, over several tries, and evaluate accordingly how good a student’s handwriting is. On that account, methods for automatic writer readability evaluation can help fasten the process of teachers, making the evaluation less time-consuming, and allowing teachers to focus on the improvement process of handwriting. COACH was developed to evaluate handwriting to identify and correct handwriting deficiencies, in which data acquired by a digitizing tablet and an instrumented pen were used [1]. However, the input methods used were still applied on specific writing surfaces which differs from the typical handwriting on regular paper used mostly during daily lives.

Inertial-based smart pens have gained popularity in the recent years due to effectiveness of such pens to remove the need of specific writing surfaces. With this increase in popularity, many applications have been developed using such pens, mostly focusing on the recognition of what is written by applying real-time digitization methods from pen movements using deep learning methods [2]. However, no previous research has focused on assessing the quality of what was written aside from transforming handwritten text into digitized text.

In this work, we aim to develop methods for automatic handwriting evaluation using a sensorequipped smart ballpoint pen, which allows the evaluation of pupil’s handwriting in a real writing scenario on regular paper. Data will be collected from multiple young users using the digital pen by writing random words on regular paper. The data will then be labeled objectively on a scale from one to six, one being very good while six means the writing is inadequate. The Bavarian curriculum for third and fourth grade is used as a guide [3], which provides the fitting scale for checking the quality of handwriting.

Following the works of handwriting recognition using inertial sensors, deep learning models, using the collected dataset, will be trained in supervised learning approach, as a time series classification task. The models will then be evaluated according the standard metrics of evaluation used in classification tasks. Moreover, an empirical evaluation of the data will be conducted to assess the impact of the different sensors on the writing evaluation.

References:
[1] Richardson, Ariella, et al. “COACH-Cumulative Online Algorithm for Classification of Handwriting Deficiencies.” AAAI. 2008.
[2] Wehbi, Mohamad, et al. “Towards an IMU-based Pen Online Handwriting Recognizer.” International Conference on Document Analysis and Recognition. Springer, Cham, 2021.
[3] Raede, Anselm: Staatsinstitut fuer Schulqualitaet und Bildungsforschung Muenchen: https://www.lehrplanplus.bayern.de/fachlehrplan/grundschule/1/deutsch (2022)
[4] Graham, Steve, et al. “Dimensions of Good and Poor Handwriting Legibility in First and Second Graders: Motor Programs, Visual–Spatial Arrangement, and Letter Formation Parameter Setting”, Routledge, Developmental Neuropsychology 29 (2006): 43-60.