Mistral 7B IT

Mistral 7B Instruct is the 7.3B parameter LLM that beats Llama 2 13B on all benchmarks, leveraging Grouped-Query Attention (GQA) for rapid, state-of-the-art performance.

This is the instruction-tuned version of Mistral 7B: a compact, high-performance model released under the permissive Apache 2.0 license. It delivers superior results, outperforming Llama 2 13B across all metrics and rivaling CodeLlama 7B on code tasks. Key architectural features drive this efficiency: Grouped-Query Attention (GQA) ensures faster inference speed, while Sliding Window Attention (SWA) handles longer sequences efficiently, supporting a context window up to 32k tokens (v0.2).

https://mistral.ai/news/mistral-7b/