Time to First Token (TTFT) is a critical performance metric that measures the duration between a user submitting a prompt and the Large Language Model (LLM) generating its very first token.

Unlike total generation time, TTFT specifically tracks the initial latency of the inference process, including network overhead, prompt processing, and the start of the decoding phase.

In an AI Gateway, monitoring TTFT is vital for optimizing user experience, as a low TTFT ensures that applications feel responsive and interactive. It serves as a primary benchmark for evaluating the efficiency of different model providers, hardware configurations, and load-balancing strategies within a production environment.