Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
January 15, 2026
·
Tokyo
Variably Prompt Experimentation Platform
See a live demo of a platform for A/B testing prompts, scientific evaluation without LLM-as-judge, ROI measurement, and the underlying architecture.
Overview
I’ll be live demoing the Prompt Optimization and Experimentation Platform that Variably Technologies has built.
I’ll show:
- How to optimize the prompts and run experimentation in the AI workflow using an A/B test methodology.
- How to run the scientific evaluation and score the responses without LLM-As-Judge and automatically decide the winner and deploy it.
- Measure the impact on the Business ROI.
- Lastly, I’ll explain the product’s architecture.
Links
Tech stack