Vapi is the Voice AI platform for developers.

Vapi AI

Hi, I am currently running this stack:

TTS: cartesia sonic-3
SST: deepgram nova-3
LLM: gpt-4.1-nano-2025-04-14

livekit-snappy + my endpointing is set to be very fast, mostly the latency according to the logs is coming from the voice model, its taking like 700 ms to get a response from cartesia. is this normal?

and my roundtrip/voice-to-voice is not super awesome. the logs put it around 1100 ms but ive found them to not be super accurate. 

anyone have any suggestions on what stack i can use? my only requirement is sub-500ms and custom voices.

**Suggestion:**  
Try switching your TTS provider to **LMNT**, which is designed for ultrafast, low-latency streaming and supports custom voice cloning. You can quickly integrate LMNT using their Python SDK:

```python
import lmnt
client = lmnt.Client(api_key="YOUR_API_KEY")
audio = client.synthesize(text="Hello world!", voice_id="your_custom_voice_id")
# Play or process 'audio' as needed
```

*LMNT claims real-time performance and custom voice support, making it a strong fit for your sub-500ms latency goal.*  
For setup details, see the [LMNT documentation](https://docs.vapi.ai/providers/voice/imnt).

Source:
- [LMNT documentation](https://docs.vapi.ai/providers/voice/imnt)

are you already streaming audio as it's generated, or waiting for full chunks before playback?

double confirming now but i believe the former

is sub-500ms with custom voices reaosnable for me to expect?

it is technically possible, but I wouldn’t say it’s reliably achievable yet if you also want high-quality custom voices.

If you strip things down (super fast endpointing, tiny/streaming LLM, and very lightweight TTS), you can get close to sub-500ms. 
But once you add custom voice cloning + good prosody, TTS alone often eats 400–800ms.

<@1368614104994156698> I am streaming before the full response completes. Active config is Vapi chunked streaming, not full-utterance playback:

chunkPlan.enabled=true, 
minCharacters=8, 
formatPlan.enabled=true. 

so playback begins from Vapi-generated chunks, not after the whole response is finished.

In my experience, reduce the chunk size and use a fast audio voice.

ok. my only limitation is i need a voice cloned voice.

fast you mean, a provider, or increase speed 1.25x? if the former, what provider is the fastest in your experience?

I meant provider/model latency, not playback speed (1.25x won’t reduce actual latency).

is playht still active? their site doesnt even load <@1368614104994156698>

@wenbo it seems neither of these offer voice cloning either

elevenlabs doesnt easily allow voice cloning.

it requires alot of red tape, i need celebrity voices

Are you a developer?
And are you working on the project mentioned above?

As a full-stack and voice AI developer, I can help you with your project.

could you please provide us with few of your call IDs so we can take a better look and suggest?