https://vapi.ai/ logo
Join Discord
Powered by
Docs for WebSocket transport with python?
# support
m
I'm trying to make calls using the WebSockets transport to send some generated audio to a Vapi agent. I'm using the python WebSockets library as well and am able to successfully send audio over and get back a response but any new audio sent over the same connection doesn't seem to be acknowledged. I eventually get a 
silence-timed-out
error. Any guidance would be appreciated.
v
Vapi's WebSocket transport expects a continuous stream of 16-bit PCM audio data (little-endian) to avoid triggering a 
silence-timed-out
error. In Python, ensure you are sending audio chunks at regular intervals without long pauses. Here's a minimal outline:
Copy code
python
import asyncio
import websockets

async def send_audio(uri, audio_chunks):
    async with websockets.connect(uri) as ws:
        for chunk in audio_chunks:
            await ws.send(chunk)  # chunk should be bytes in PCM s16le format
            await asyncio.sleep(0.1)  # adjust interval as needed for real-time streaming

# Make sure audio_chunks is a generator yielding small, regular PCM chunks
Keep the stream active by sending audio data continuously, and close the connection with a control message when done. For more, see the [WebSocket transport docs](https://docs.vapi.ai/calls/websocket-transport). Source: - [WebSocket Transport – Vapi Docs](https://docs.vapi.ai/calls/websocket-transport)
k
Make sure continuous audio flow without long gaps, adjust the silence timeout setting if needed, and manage the call using the control URL to avoid silence-timed-out errors..
m
ah gotcha, was able to get it to work. Happy to contribute to any docs with a python sample so it helps others out.
One thing I think that is very much needed is docs on the types of messages that are sent over the WebSocket. Which of these can we rely on? For example I need to know when the Vapi agent has finished speaking, and for that I think I can look for the 
speech-update
message with a status of 
done
. But I also need to get the full transcript of what the agent just said. For that should I look for the 
voice-input
message, or 
transcript
message with status 
final
, or the 
conversation-update
message. All of this I feel like I had to figure out on my own by just connecting to the WebSocket and see what messages were being sent and observing how they behaved.
cc @Kings_big💫
k
Got it
To detect when the Vapi agent finishes speaking, use the speech-update message with "status": "done", and to get the full transcript, listen for the transcript message with "status": "final"
m
what control message should be sent to avoid silence-timed-out errors? Any dummy assistant control message would work? https://docs.vapi.ai/calls/call-features
k
Send periodic control messages like conversation-update or speech-update to keep the call active.
k
Hey Mathusan, checking if this is resolved/solved for you?
m
yep solved for me, thanks yall!
v
To avoid silence-timed-out errors, send a periodic control message like this:
Copy code
js
sendControlMessage({ type: "conversation-update" });
Any valid control message (such as 
conversation-update
or 
speech-update
) will keep the call active. Adjust the interval as needed to prevent timeouts, since the docs do not specify a required frequency. Reference: [Vapi WebSocket Transport Docs](https://docs.vapi.ai/calls/websocket-transport) Sources: - [Vapi WebSocket Transport Docs](https://docs.vapi.ai/calls/websocket-transport) - [Call Features](https://docs.vapi.ai/calls/call-features)
g
Hi @mathusan, @Kings_big💫 , can you share the python snippet that worked for you for using Vapi websockets api with python? I am initiating sending of audio stream once I receive speech-update, status:closed event. This is my current snippet, but it seems in the vapi dashboard that the audio I am sending and the assistant that is speaking are overlapping significantly, leading to error: call.in-progress.error-assistant-did-not-receive-customer-audio
k
This ticket has been marked as solved, and as a result, you will no longer receive any further responses. Kindly create a new support ticket to escalate your inquiry. 
PreviousNext
Powered by