This week I attended AI Conference in San Francisco. I don’t really do a lot of work with machine learning or generative AI, but the entire software world is shifting in that direction so… gotta keep up. I’m not particularly bullish on generative AI, but I’d say my mindset shifted to one of cautious, skeptical optimism.
This post is mostly some notes to myself for things to follow up on.
Learn Less, Parse Parse More
Given by Marie Chatfield Rivas, this was my favorite talk. They have a series of articles on linked in that cover the same content, so check those out.
Large language models are inherently probabilistic and inconsistent; instead of trying to prompt and train them to perfection, it’s much easier to parse and post-process their responses to get reliable results.
This was about having an LLM output some sort of structured data — a grammar that can be parsed and then post processed. That can be an existing format (like JSON) or a custom, domain specific grammar.
Embeddings
Embeddings turn data into a vecor of floats. The format is what LLM and other AI uses an intermediary to store their own data, etc.
This is one area I need to understand a bit better and plan to go through this paper on the topic.
Retrieve Augmented Generation (RAG)
Say you have a corpose of of documentats for your business/domain/whatever. You can take though, run them through an emedding algorithm and store the embeddings in something like a vector database.
When a prompt comes it, you run the promp through the same embedding algorithm and find related documents in your vector database, then add those documents to the context window of the prompt as additional context to achieve a better result.
This is a cool idea (and one I need to play with more) because it could possibly let you skip a fine tuning a model on proprietary data and instead rely sole on RAG to generate relevant results.
Also means that stuff like access control could come into play on the vector database side (restrict users from seeing stuff they shouldn’t see). Every single vector database person at the conference brought up that filtering like this for security or other attributes then running the nearest neighbors queries on embeddings is a hard problem.
Ethics & Bias
A lot of talks brought up ethical AI and things like bias and how it’s import to both validate this before lauching an AI platform (Fiddler Auditor seems like a cool tool for this) and while the platform is running and overtime.
Couple of related concepts here:
Other Random Cool Stuff
- Skyplane for transferring data across PaaS or regions (also thank god I don’t have to deal with this on the regular)
- Hugging Face in general, I was aware of this but never really dive in. Need to do that.
- Anthropic because I was very impressed by their mission statement around AI saftey and harmlessness and Bejamin Mann’s talk
- MusicML and AudioCraft — generated music is neat.
Christopher Davis 