Speech-to-speech models

End-to-end neural networks that bypass text intermediates to deliver sub-300ms latency and emotional nuance.

Speech-to-speech (S2S) models represent a paradigm shift from cascaded systems (ASR + LLM + TTS) to unified multimodal architectures. By processing audio tokens directly, models like GPT-4o and Meta's SeamlessM4T eliminate the 'robotic' lag and data loss inherent in text transcription. These systems preserve prosody, detect emotional inflection, and handle cross-lingual translation with human-like cadence. Current benchmarks show latency dropping below 232 milliseconds (matching human conversational speed) while maintaining high BLEU scores for real-time interpretation and accessibility interfaces.

https://openai.com/index/introducing-gpt-4o