I want to understand transformers, so I’ve been banging my head against them since the New Year from all angles: YouTube, podcasts, O’Reilly books, attempting an explanation towards anyone who’d listen, etc. I’m getting close, I think! (at least for the basic feed forward inference – gradient descent remains mysterious) Possibly premature, but I’m feeling that two chumps like my Dad and I are capable of understanding how LLM’s work. As non-chumps have pointed out, the math isn’t too complicated, the real unlocks seems to be scale + data. (somewhat on that note: wtf is Chinchilla optimal?)

…

This journal and my professional life are getting a reboot (LLMs!). It’s a space for reflection, for marking time, and for organizing my daily journey. January has been designated as my immersion month, and I am already amidst the chaos of various half-understood topics and the creative zeal of individuals pushing the boundaries. The frontier is in constant motion. This journaling endeavor is tasked with outlining the threads I am following, providing some continuity in a shifting landscape.

…

My Aunt & Uncle Christmas gifted me the pragmatically German LAMY Safari fountain pen, and it sparked an urge to handwrite the rough draft of my journal entries. I mean, this pen feels wicked smooth, I’ve spent an insane amount of time learning to and writing throughout my life, and it connects me to my past, to my mother who has beautiful handwriting.

But I still got to digitize. As text, the entries can be edited, searched, and published. And, this brings us to the point: how can I most easily OCR my handwritten notes.

…

If you’re reading this online, you might not realize that this text primarily exists within a single long org document, a dump of bits and bobs. The org file is somewhat incidentally posted to the internet via ox-hugo, and there’s even a lot of unpublished content I just keep as notes. This god-file initially didn’t distinguish between my journal as well as a daily log. Over time though, they separated. And then I started using Obsidian.

…

Today is Dad’s birthday and the fam is all up in Tahoe! The wrinkle is CEO has covid and I’m exposed (tho tested negative last night), so will have to keep outside / distanced / masked. So it goes; I have a gas fire pit and can layer up to handle the 35. Will also apologize to the Marty’s for the mess caused.

First, I have to covid test… annd tested positive. Fuck. Now I’m headed home with CEO to quarantine.

…