Interested in what’s possible with a sensor-equipped ballpoint pen?
The Handwriting Recognition Demo is a free Android app that showcases the current recognition capabilities. It runs on phones and tablets with at least Android 7. All you need is a DigiPen and a fast and stable internet connection.
Download: Directly from Google Play
1. This is a beta version with limited computational resources. Questions or feedback? Contact us!
2. The system has been developed and tested with right-handed persons. Left-handers might experience slightly worse recognition rates.
3. Only the letters “A…Za…zÄÖÜäöüß” can be recognized. No numbers or punctuation yet.
The app (and more importantly: the recognition algorithms) are being updated and improved regularly.
Demo video: Available soon
The way the app does the recognition is user-dependent. This means any user has to provide labelled training data (by copying words and sentences) that can be recorded within the same app. Afterwards, a personal recognizer can be trained. The more training data you provide, the better the recognizer becomes.
Before recognizing handwriting, you have to specify a dictionary. The system can only recognize words that are part of the dictionary. The smaller the dictionary, the better the recognition works. The app provides a couple of sample dictionaries of different sizes. Alternatively, you can manually add custom words or even new dictionaries.
After each recognized word, you have the opportunity confirm or improve the output. If you’re doing this, you can – in turn – use these words as training data to build an even better model afterwards.
- Open the app and remove the DigiPen’s cap to turn it on (the LED should flash blue).
- Click “Connect” and select your DigiPen.
- Once the connection process is finished, the DigiPen’s LED is blue permanently.
- Every couple of days, the sensors should be recalibrated. Click “Calibrate”.
- Follow the on-screen instructions by first keeping the pen in a static position (on the table) and then rotating it in all directions.
- Proceed by logging in or creating an account.
- You need at least 4 character recordings and 1 word recording (consisting of 350 words). To further improve your recognizer, add even more words.
- In the overview screen, select the type of recording you wish to do and click “Start Recording”.
- On the recording screen, accept the recording rules.
- Click the “start recording” button and write down the displayed word/character on a normal sheet of paper. Once finished, click the green checkmark. Then, the next element to be copied will be shown.
- If you made a mistake copying that word, click “repeat”. To move the checkmark button to the other side for more convenience, click “switch side”.
- By default, the word/character to be copied is read out loud by the app. You can disable that by clicking the speaker button.
- After your recording session, click the red “stop recording” button. Your recording is saved even if your stop before the expected 350 words.
- On the overview screen, click “start training” to get an overview of the recordings you made.
- The training process takes place in the cloud. The cloud symbol shows if a recording is already available online.
- Select the recordings you wish to train your recognizer with by tapping them. Then, click “train recognizer”. This may take a while.
- Once finished, go back to the overview screen. Select the recognizer in the bottom right corner. It shows the amount of letters/words it was trained with.
- On the overview screen, select a recognizer that you already trained, and a dictionary. Then, click “start recognition”.
- Your recognizer is being prepared in the cloud. This may take a while.
- Click the green “start recognition” icon and write a word. After lifting the pen for the amount of time specified in the grey progress bar, a recognition result is calculated and displayed. You can increase or decrease this timespan yourself. A greater value requires you to pause longer between words. A smaller value might confuse the system if you pause for too long within a single word.
- The person intended to write “I demonstrate the handwriting recognition”. The system failed to recognize the “I”. If you want to improve your personal recognizer, give the system feedback about the quality of its hypotheses. Click the “edit” icon to the right of the wrong word.
- Using the device keyboard, improve the recognition result and click “submit”.
- Since the other words were recognized correctly, confirm the hypotheses by hitting the green check mark icons.
- The words are now colored green as you corrected/confirmed them all. They can be used for training an improved personal recognizer in the future.
- Once finished, click the red “stop recording” icon.
- The system only recognizes words contained in a given dictionary. You can modify the given sample dictionaries or add a new one. The dictionaries are simple .txt files located in the device folder <INTERNAL>/STABILO/WritingRecognizer/Dictionaries/.
- By the way: If you export recordings from the “train recognizer” screen to csv files, they show up in <INTERNAL>/STABILO/WritingRecognizer/Recordings/Exported/.
- Copy one of those files to your PC and open it with a text editor. Add your custom words on a new line each. Beware that only the letters A…Za…zÖÄÜöäüß can be part of that file. Make sure you keep the same formatting (UTF-8, one word per line) to avoid errors.
- Copy the file back to the tablet folder with a new name. Sample dictionaries may be replaced with future updates; custom dictionaries will stay the same.
- You can now select this new dictionary from the overview screen.
Don’t hesitate to contact us!