What Is Eleven Realtime? What ElevenLabs' Real-Time Voice AI Can Do and the Difference Between Mini and Max

Hi, I’m Hoda!

In October 2026, ElevenLabs announced a new real-time voice AI model, “Eleven Realtime.”

Eleven Realtime is a speech-to-speech model for building “voice agents” that listen to a user’s voice, understand what was said, think on the spot, and reply by voice.

From the name alone, you might wonder, “Is this just a faster version of ElevenLabs’ speech generation?” In fact, it is a much bigger update.

The key feature of Eleven Realtime is that it optimizes speech recognition, conversation turn detection, the LLM, and speech generation as a single system.

In other words, it is now easier than before to build real-time voice AI that works like this:

The user speaks → the AI listens → it understands the content → it uses tools if needed → the AI answers in a natural voice

What Is Eleven Realtime?

Eleven Realtime is a series of models for real-time voice agents provided by ElevenLabs.

Until now, building a natural AI voice agent meant combining multiple technologies.

For example:

• Speech recognition with speech-to-text
• Speech detection with VAD (Voice Activity Detection)
• Turn-taking, which judges whether the user has finished speaking
• An LLM that answers questions
• Speech generation with text-to-speech

You can choose and configure each model separately, but this can cause problems such as “the AI is slow to start replying,” “it talks over the user,” and “it behaves unnaturally when interrupted.”

Eleven Realtime optimizes all of these from the start as a single system for voice conversation.

What Can You Do with Eleven Realtime?

Specifically, with Eleven Realtime you can build the following kinds of AI.

Talk with AI by Voice in Real Time

The most basic use is a real-time AI assistant, like the voice conversation in ChatGPT.

When the user speaks into the microphone, the AI understands the speech and answers by voice.

It doesn’t just repeat a simple “question → answer” pattern. It also takes into account:

• Short backchannel responses
• Pauses in the conversation
• Interruptions from the user
• Quick back-and-forth exchanges

That makes it easier to create an experience closer to a conversation between people, compared with the traditional approach of “record audio and send it → transcribe → AI answer → speech generation.”

Build AI Tutors and Language Learning Services

ElevenLabs gives the AI Tutor as a typical use for v1 Mini.

For example, in an English conversation service, when the user tells the AI,

“Today I want to practice English for use on a trip,”

they can continue the conversation in English with the AI teacher.

It can also be applied to:

• Pronunciation practice
• Answering questions
• Role-play
• Learning for children
• Interview practice
• Presentation practice

Eleven Realtime supports more than 90 languages, so it can also be used for multilingual services.

Build AI Characters for Games and Apps

It can also be used to let users talk on the spot with characters in games and apps.

For example, when a user asks an NPC,

“Where can I buy weapons in this town?”

the character answers by voice in line with its personality and the world of the game.

ElevenLabs has a voice library of more than 17,000 voices, and you can use these voices with the Realtime models too.

So instead of the same “AI-sounding” voice every time, you can choose a voice that suits your service or character.

Build AI Customer Support

Eleven Realtime v1 Max is also well suited to customer support, which involves more complex tasks.

For example, when the AI receives an inquiry such as,

“The item I ordered hasn’t arrived yet,”

it can work through these steps while talking:

1. Ask the user for the order number
2. Search the order management system
3. Check the delivery status
4. Explain the status to the user

The key point is that it doesn’t merely answer questions: by using Tool Calling, it can retrieve information from external services and carry out actions.

Automate Lead Qualification

In sales, it can be used for lead qualification.

For example, over the phone or on the web, the AI asks,

“What kind of service are you looking for?”

“About how large is your company?”

“When are you planning to adopt it?”

and changes the next question depending on the answers.

You could also build a system that hands off only the leads who meet your criteria to a salesperson.

Use It by Phone, on Websites, and on WhatsApp

When you use it through ElevenAgents, you can deploy the AI agent you build to multiple channels.

For example:

• Websites
• Phone
• WhatsApp
• Contact centers
• Web widgets

ElevenAgents also provides a knowledge base, guardrails, testing, monitoring, and more.

So if you want to build real customer support or phone handling, starting with ElevenAgents is easier than using the Realtime API directly.

Differences Between Eleven Realtime v1 Mini and v1 Max

Eleven Realtime currently comes in two models.

Basically, it’s best to start with v1 Mini.

ElevenLabs itself says Mini is enough for most common uses.

On the other hand, v1 Max is better suited to uses such as:

• When you need the AI to understand complex instructions
• When you need it to run multi-step processes
• When it uses external tools frequently
• Customer support where accuracy is important

Comparing Mini and Max Performance

In the benchmarks published by ElevenLabs, each model’s strengths show up quite clearly.

What is interesting is that Mini is also very strong in the naturalness of the conversation itself.

Full-Duplex-Bench is a benchmark that evaluates pauses, interruptions, turn-taking, backchanneling, and similar aspects of conversation, and Mini scored 95.8%.

On the other hand, in Big Bench Audio, which measures reasoning ability through audio, Max scored 98.8% and Mini scored 75.9%.

The success rate on tasks that use Tool Calling also shows a big gap:

• Mini: 60.3%
• Max: 87.1%

So it is easiest to think of it this way:

Mini if you want natural conversation, Max if you want it to do work

With ElevenAgents, You Just Choose the Model

The easiest way to use Eleven Realtime is ElevenAgents.

In the traditional Custom mode, you can configure these separately:

• LLM
• Speech to Text
• Text to Speech
• Reasoning settings

In Realtime mode, when you select “Eleven Realtime v1 Mini” or “Eleven Realtime v1 Max,” ElevenLabs manages the main voice AI components.

At the same time, you can still configure:

• System Prompt
• First message
• Voice
• Language
• Tools
• Knowledge Base
• Conversation flow
• Interruption settings

In other words, you no longer have to think about combinations of detailed AI models, and it becomes easier to focus on what you want the agent to do.

Compatible with the OpenAI Realtime API

For developers, a big feature is its compatibility with the OpenAI Realtime API.

Eleven Realtime uses the same event structure and WebSocket approach as the OpenAI Realtime API.

So if you have already built an app with the OpenAI Realtime API, you don’t need to switch to a different SDK.

Basically, you can use it by changing items such as:

• The endpoint you connect to
• The model you use

to ones for ElevenLabs.

For example, specify this model:

eleven_realtime_v1_mini

or:

eleven_realtime_v1_max

This means you can keep using the OpenAI Realtime SDK while switching the audio processing part to ElevenLabs.

I think this is a design that makes it quite easy to try for developers who are already building voice apps with the OpenAI Realtime API.

The Realtime API Uses a WebSocket Connection

One thing to watch for is the connection method.

At release, the Eleven Realtime API supports WebSocket only.

So if your app uses the OpenAI Realtime API over WebRTC, you can’t just change the endpoint. You need to switch to WebSocket.

Also, when connecting from a browser, instead of embedding your ElevenLabs API key directly in the frontend, a mechanism is provided for issuing a short-lived Client Secret on the backend and using that.

The Realtime API and ElevenAgents Are Different Things

This is another point that is a little hard to follow.

There are two ways to use Eleven Realtime:

Using it through ElevenAgents

and

Using the Realtime API directly.

However, connecting through the Realtime API does not give you all of the features of ElevenAgents.

For example, if you want to use:

• Phone
• WhatsApp
• Knowledge base
• Guardrails
• Testing
• Monitoring
• Contact center integration

it is better to use ElevenAgents from the start.

On the other hand, the Realtime API is better for developers who want to build voice AI directly into their own web app or service.

How Is It Different from Scribe v2 Realtime?

ElevenLabs also has a service called “Scribe v2 Realtime,” so the names may be a little confusing.

Scribe v2 Realtime is a speech-to-text model that converts speech to text in real time.

For example, it is for cases like this:

“Hello” → get the text “Hello” in real time

Eleven Realtime, on the other hand, goes further:

“Hello” → understands the content → the AI thinks of an answer → replies by voice with “Hello! What would you like to do today?”

It’s easy to remember this way: Scribe v2 Realtime if you want real-time captions or meeting minutes, and Eleven Realtime if you want the AI to hold a conversation.

It Is Actually Not a Single, Fully Unified AI Model

When you hear “speech-to-speech model,” it can sound as if one huge AI model processes everything directly from input audio to output audio.

However, Eleven Realtime is not a fully unified fused model.

Internally, it uses a cascaded architecture, with components such as:

• Speech Recognition
• Turn Taking
• LLM
• Voice

But these are not simply connected as separate services. ElevenLabs tunes them so that they are optimal for voice conversation as a whole.

So from the user’s side, it takes a simple form: “just select one Realtime model.”

Pros and Cons of Eleven Realtime

Who Eleven Realtime Is For

Eleven Realtime is especially suited to people who want to build services like these:

• AI voice assistants
• AI English conversation and education services
• AI characters
• AI customer support
• Automated phone answering
• Lead generation and sales qualification
• AI reservation handling
• Voice-enabled web apps
• Voice-enabled SaaS

Until now, building voice AI meant choosing and connecting speech-to-text, an LLM, and text-to-speech separately, and then tuning the timing of the conversation as well.

With Eleven Realtime, you can leave that complicated part to ElevenLabs.

Summary

Eleven Realtime is a new series of models for real-time voice agents that ElevenLabs announced in October 2026.

It is not a simple speech generation model. Its biggest feature is that it optimizes the processing needed for voice conversation as a single system:

Speech recognition → turn detection → LLM → speech generation

There are currently two models:

• Eleven Realtime v1 Mini: For natural, fast conversation
• Eleven Realtime v1 Max: For complex reasoning and Tool Calling

What is especially interesting is that with ElevenAgents, you can build a voice agent just by choosing a Realtime model, without selecting detailed AI models.

It is also compatible with the OpenAI Realtime API, so it is an easy option to try for people who are already building real-time voice AI.

If you want to build natural conversation with AI into your service, and not only have AI generate text, this is an ElevenLabs feature worth watching.