https://vapi.ai/ logo
Join Discord
Powered by
OpenAI Realtime is frequently cut off while model ...
# support
i
I've been trying to use the OpenAI realtime preview and it's had very poor performance, basically unusable: * The VAD is really bad, the model is very frequently cut off while it's speaking. I'm getting 100% repo. * (Unrelated, but would be great) I can't select the new voices: https://platform.openai.com/docs/guides/realtime#voices My config: el": { "model": "gpt-4o-realtime-preview-2024-10-01", "provider": "openai", "maxTokens": 150, "temperature": 1.0, "emotionRecognitionEnabled": true, "messages": [ { "role": "system", "content": "REPLACE ME" } ] } ic| call.ID: '895482af-dab3-4898-a742-db78bcea1e98' ic| call.ID: 'd6e810e8-b421-4317-913c-bee710f01457' @bryant
v
message has been deleted
@IgorD I'm sorry to hear that you're experiencing issues with the OpenAI realtime preview. It sounds like there might be a problem with the Voice Activity Detection (VAD) and accessing the new voices. Let's try to address these issues one by one. For the VAD issue, it's important to ensure that the audio input is clear and without background noise. Sometimes, adjusting the sensitivity of the VAD can help. However, since you're experiencing a 100% reproduction of the issue, it might be more technical than just a sensitivity setting. Regarding the new voices, if they are not appearing as options, it could be due to the specific configuration or limitations within the preview model you are using. Since these issues seem to be quite specific and technical, I would recommend reaching out to our support team for more detailed assistance. They can help you troubleshoot the VAD problem and provide guidance on how to access the new voices. Would you like me to guide you on how to contact our support team or is there anything else I can assist you with right now?
You can continue this conversation by mentioning me in the message.
b
Hey @IgorD , the new voices issue is solved now, i will look into the VAD issue
i
Thanks @bryant - I don't see the new voices on the UI, does it need to propogate to prod maybe?
Ha...
OK, setting it to sage directly has it not working at all now - oompf. 9bee5e25-0cb6-43e8-81e9-b0ff47a2842e
* not working at all => the model never talks, even though it says it did in the log.
9bee5e25-0cb6-43e8-81e9-b0ff47a2842e
Thanks for the support! I really enjoy VAPI, can't wait to get it on this super emotive/low latency model.
b
hey @IgorD , certain features such as first message won't work at the moment with the new realtime voices. really sorry about this, but we will add them as soon as possible
yes, soon!
i
Hi, have you noticed that Realtime model is hallucinating a lot? I did several tests and I understand that this model doesn't read KB yet. But also during tests I found that the agent surprisingly changed name of the location I provided and I couldn't correct it in the conversation
b
you're right that KB is not supported as of now. the realtime model is still in beta so I assume the hallucinations would improve over time (and get cheaper)
@IgorD RE: the speech cutting off randomly, i believe this thread might shed some light into why it is happening: https://community.openai.com/t/realtime-api-audio-is-randomly-cutting-off-at-the-end/980587/22
i
I don't get it - does this mean I need to talk first? And then it will respond? I don't think that worked, but will try again.
Sigh, yup that does sound like the issue. Probably worth add a link in the UX warning users it's still mostly unusable and link to this thread.
b
Hey @IgorD , yes this is the case for the newer voices. We have some fixes that will be released soon that will hopefully improve your experience further. Once again thank you for your feedback, we really appreciate it!
5 Views
PreviousNext
Powered by