Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
DSPy Prompt Optimization
Experiments with DSPY, a prompt optimization package, applied to a fact-checking system are shared, demonstrating a structured approach to prompt engineering.
DSPY is a prompt optimization package for self improving AI programs. I implemented some experiments on a fact checker. I’m happy to share those results. Github is here: https://github.com/julianghadially/FactChecker
DSPy-based FactChecker uses iterative web search, achieving GEPA-optimized verification performance.
- DSPYDSPy (Declarative Self-improving Python) is the Stanford NLP framework that compiles declarative Python code into self-improving, optimized language model pipelines: program LMs, don't prompt them.DSPy radically shifts LLM development from brittle prompt engineering to compositional Python code (Declarative Self-improving Python). The framework, developed by Stanford NLP, uses a compiler to automatically optimize your AI system: it synthesizes effective prompts, generates few-shot examples, or fine-tunes model weights based on a specified metric (e.g., quality or cost). This systematic approach uses three core abstractions: Signatures (defining input/output), Modules (reusable components like Chain of Thought), and Optimizers (Teleprompters). Since its late 2023 release, DSPy has seen significant adoption, boasting over 28,000 GitHub stars; it is the definitive solution for building robust, modular AI systems, from simple classifiers to complex RAG pipelines.
Related projects
Infra Vibes
Austin
See the process of architecting, building, and launching an AI infrastructure application for Shopify stores, including challenges and…
Too Big to Think
Austin
Examining how transformer capacity influences memorization versus arithmetic extrapolation, using synthetic character tasks to reveal trade‑offs and limits…
From Vibes to Metrics: Live Prompt Optimization with DSPy
Berlin
Live demo transforms a fragile LLM pipeline into an optimized DSPy workflow, comparing baseline and optimized accuracy, latency,…
Prompt Optimization and Experimentation Platform for AI workflows.
Tokyo
See a live demo of a platform for A/B testing prompts, scientific evaluation without LLM-as-judge, ROI measurement, and…
DSPy 101
Boston
Learn to build a DSPy classification pipeline: load data, evaluate a base model, apply optimization, and compare results…
Dittto
Los Angeles
Learn how Dittto lets teams instantly test multiple brand voices to generate clearer, more effective hero copy, with…