June Twentieth

GT7 racing with SJO, sixteen laps at Spa. We’re not quite equal on pace but damn it’s a fun track.

LLMs continue to rock the world:

• Prompting Is Programming: A Query Language for Large Language Models – “we present the novel idea of Language Model Programming (LMP). LMP generalizes language model prompting from pure text prompts to an intuitive combination of text prompting and scripting. Additionally, LMP allows constraints to be specified over the language model output. This enables easy adaption to many tasks while abstracting language model internals and providing high-level semantics.”
• Sourcegraph – Code Intelligence Platform. This is the layer that brings in coding context. I was unimpressed by Sourcegraph’s new client, Cody, which was buggy at the moment (libraries not indexing, black screens, etc). Still, I’ve heard good things, so I have a feeling that when it clicks it’s good. Uses Anthropic LLMs.
• CollabGPT – Business oriented AI companion that integrates with the various systems of record. This is the layer that brings in business context. Signed up for the waitlist to learn more 🤨
• vLLM – a new high performance inference library on the block. Competes with HuggingFace Transformers (and llama.cpp?). Uses PagedAttention to achieve it’s throughput.