OCR for [my] handwriting

My Aunt & Uncle Christmas gifted me the pragmatically German LAMY Safari fountain pen, and it sparked an urge to handwrite the rough draft of my journal entries. I mean, this pen feels wicked smooth, I’ve spent an insane amount of time learning to and writing throughout my life, and it connects me to my past, to my mother who has beautiful handwriting.

But I still got to digitize. As text, the entries can be edited, searched, and published. And, this brings us to the point: how can I most easily OCR my handwritten notes.

This post isn’t a comprehensive comparison of OCR tools, largely because I short-circuited quickly on to an option that did the trick (spoiler: it’s GPT-4). I did start with Google Lens, which had done the OCR trick for me in the past.

However, I hit some major snags with Lens, specifically: 1) it takes an mobile share step vs copy, 2) then you still have to clean the output, and 3) last but not least, the output wasn’t all that good. I’m talking to the extent that the OCR’d text would repeat chunks of the handwritten text.

Now we reach the GPT-4 punchline: I’d tried out the model’s image understanding in early Dec, but primarily to generate an image and subsequently modify that image. Turns out GPT-4 can also OCR quite well out of the box, anecdotally better than Lens given my handwriting. What made GPT-4 great was passing in extra instructions. The model could be prompted to take into account the lossy source and clean appropriately.

For example, I found the OCR often made mistakes on things like e.g. my last name, picking up Moulina. By passing the expected vocabulary into the prompt, however, it’ll be able to handle strange incoming words, like “Moulia”. And, it does encourage me to write legibly 😬

Laying out the workflow, it’s delightfully dead-simple: handwritten entry -> digital text -> Final draft. Just two actions: GPT-4 OCR + prompting and personal editing. The GPT-4 step was well wrapped by a GPTs. And yes, this post was handwritten with the LAMY. It felt great.