Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
November 15, 2025
·
Amman
LiteLLM: LLM Gateway Engineering
Explore LLM Gateways like LiteLLM, examining engineering concepts like load balancing, moderation, and abstraction for production inference deployments.
Overview
Taking a deep look into LLM Gateways (LiteLLM or similar), and the software engineering concepts behind the scene.
We will be looking into the features of an LLM Gateway, from loadbalancing, moderation, abstractive approach and more.
Tech stack