From the various tools that enable building solutions with large language models (LLMs), DSPy stands out to me as one of the most promising tools for building LLM pipelines. It introduces a powerful abstraction for pipelines that interact with language models for generation as well as retrieval.
I got to speak with Stanford Researcher Omar Khattab, the author of DSPy and have him introduce the intuitions behind DSPy, and where he sees the popular library going in the future. I also asked him about another of his popular libraries, ColBERT for Multivector semantic search.
Here's my full conversation with Omar:
This conversation took place at the 2023 NeurIPS conference (Neural Information Processing Systems Foundation), where Cohere and Cohere For AI were participating with an array of papers. Learn more about these here: https://cohere.com/events/neurips-2023