Development of neural networks and machine learning algorithms for online handwriting recognition
Project leader:
Project members: ,
Start date: 2. September 2019
End date: 30. April 2022
Funding source: Bayerisches Staatsministerium für Wirtschaft und Medien, Energie und Technologie (StMWIVT) (ab 10/2013)(7.1.1.2 – Bayerisches Staatsministerium für Wirtschaft, Landesentwicklung und Energie (StMWi))
Abstract
Writing on paper and having the writing digitized is a must have application for different types of users. Online handwriting recognition applications are available and used in daily life with the use of handheld mobile devices. However, these applications require the use of specific stylus pens and require the writing to be done on specific designed surfaces that limit the writing flexibility of the user. The aim of this project is the development of a toolkit that allows recognizing handwriting in real time. With the use of a regular ballpoint pen, and while writing on regular plain paper, we aim to provide an easy handwriting digitization model with no limitations to the user.
Motivation
Writing on paper provides better cognitive processing which can be helpful for improved critical thinking, stronger theoretical understanding, and better memory recollection. While typing or writing on tablets may be faster and allows easier digitization and document sharing, it still limits the user from his best writing abilities. We combine the advantages of both types of writing to allow the user for acquiring digitized writing while writing with a normal pen on a normal paper with no limitations.
Methods & Data Collection
The digitizer used is a sensor-enhanced ballpoint pen provided by STABILO (Digipen). It includes four different sensors (two accelerometers, one gyroscope, and one magnetometer) distributed along the pen. It connects via a Bluetooth module that allows live streaming of the sensor data to any handheld device or mobile phone. Provided with soft-touch grip zone for holding, the pen allows easy writing on any paper with no additional hardware required. The STABILO Digipen is used for continous data collection from users with different age and writing styles. The collected data is used to improve the model’s robustness to allow generalized recognition with no user specific training to be required for good recognition accuracy.
Partners
https://stabilodigital.com/digipen/
Demos