Call ID:

e6d8316e-dd0b-427b-8506-988eaf6ad61d

Assistant ID:

8bf8d5f0-9629-415f-9857-ea74e630e5af

https://cdn.discordapp.com/attachments/1339201697188286525/1339201697444008008/Screenshot_2025-02-12_at_12.46.38.png?ex=67addc45&is=67ac8ac5&hm=0d1ef7a353a7015f75dda68ed62120f69d418da3c8ab4f1442b71b5fe7f953ad&

Hey barbelos, can you try another call with startSpeakingPlan waitSeconds set to 0.8. { "start_speaking_plan": { "waitSeconds": 0.8, "smartEndpointingEnabled": false, "transcriptionEndpointingPlan": { "onPunctuationSeconds": 0.5 } }

Sure, let me try it and will let you know

Hey barbelos, I just listened to your call recording and the response came on 4th second from start so which is approx 2-3 seconds to first response after user saying hello on call 1st second. Also I looked into your assistant you have done the following config here waitSeconds is supposed to be 0.8 and for onPunctuationSeconds I suggest using the default value unless your aware of how it works. "start_speaking_plan": { "waitSeconds": 0.1, "smartEndpointingEnabled": false, "transcriptionEndpointingPlan": { "onPunctuationSeconds": 0.5 } },

From your previous recommendation, the only change you are suggesting is to update

waitSeconds

from

0.8

to

0.1

correct? This Call

eb2d22c9-4877-4ebb-9fdd-b0e96bc83437

should already have the values shared here https://discord.com/channels/1211482211119796234/1339201697188286525/1339205359465795624 Just to confirm, it should be

startSpeakingPlan

(camelCase) and not

start_speaking_plan

(snake_case) right?

Sorry my typo mistake its supposed to be 0.8 not 0.1 which i suggested previously But you didn't made the change so i suggested to change it 0.8

I did make the changes. It's even verified in the response of the

/call

endpoint See below for Call ID

eb2d22c9-4877-4ebb-9fdd-b0e96bc83437

{"id":"eb2d22c9-4877-4ebb-9fdd-b0e96bc83437","assistantId":"8bf8d5f0-9629-415f-9857-ea74e630e5af","phoneNumberId":"xxx","type":"outboundPhoneCall","createdAt":"2025-02-12T13:42:38.090Z","updatedAt":"2025-02-12T13:42:38.090Z","orgId":"xxx","cost":0,"customer":{"number":"xxx"},"status":"queued","phoneCallProvider":"vapi","phoneCallProviderId":"xxx","phoneCallTransport":"pstn","phoneCallProviderDetails":{"sbcCallId":"1093e61b-63ea-123e-f5a7-3e2dcc68de1e"},"assistantOverrides":{"startSpeakingPlan":{"waitSeconds":0.8,"transcriptionEndpointingPlan":{"onPunctuationSeconds":0.5},"smartEndpointingEnabled":false},"variableValues":{"...

logs

🔵 13:42:49:509 VAD Detected: Speech Started 🔵 13:42:50:109 VAD Detected: Speech Stopped 🔵 13:42:51:139 \[user LOG\] Transcriber output: hello 🔵 13:42:51:140 Endpointing Heuristic Detected No Punctuation, Setting Timeout For 1500ms... 🔵 13:42:51:140 \[user LOG\] Endpointing timeout 1500ms (rule:

heuristic

) 🔵 13:42:51:313 Azure Speech

user

Final Transcript : Hello؟ 🔵 13:42:52:889 Completion Request Starting (#1, provider: azure-openai, model: gpt-4o-mini-2024-07-18-global, region: westus, credential: false)... 🔵 13:42:53:205 \[user CHECKPOINT\] Model sent start token [barbelos](https://app.plain.com/workspace/w_01HYEQ5FKPH8CPX2MS608B6S51/customers/c_01JKT71A7XV5ZW0Q6EC56T9S76/)**1** As you can see in the logs first it took almost 1.5 seconds to identify if user has stopped speaking because of arabic punctuation, and then 1 next seconds for sending user input to LLM to voicing out almost 2.5s. I suggest tweaking

onNoPunctuationSeconds

field of start speaking plan. https://cdn.discordapp.com/attachments/1339201697188286525/1339334518301134888/Screenshot_2025-02-13_at_2.06.41_AM.png?ex=67ae57f8&is=67ad0678&hm=be05b620942d1c3b59f5cb87302da76175970fb60ad5ac3962dcdc05bb084d6e&

Okay, will play around with this a bit more and update you by EOD tomorrow qq - out of curiosity, you mentioned You couldn't see the suggested changes previously. I'm adding them in

assistantOverrides

. Are these respected or do you recommend building the assistant through the API?

oh my bad I looked directly into your assistant ID previously.

Okay, thank you! I'll try out tweaking

onNoPunctuationSeconds

and let you know if I can't get a better performance by EOD tomorrow. Any delay above 1-2 seconds from the user speaking to AI responding is a bad experience

Hey Barbelos To help track down this issue, could you share: - The call ID - When exactly this happened (the timestamp) This would really help us figure out what went wrong!

Shubham you can market this as resolved! It seems LLM will always wait for a user's input before responding after the first message. Maybe it is something you can indicate clearly in your documentation. Thank you

Hey Barbelos, this concept is called turn taking after LLM has finished speaking and to generates next response it needs user input. You can use **firstMessageMode *example: assistant-speaks-first*** This is the mode for the first message. Default is 'assistant-speaks-first'. Use: - 'assistant-speaks-first' to have the assistant speak first. - 'assistant-waits-for-user' to have the assistant wait for the user to speak first. - **'assistant-speaks-first-with-model-generated-message' to have the assistant speak first with a message generated by the model based on the conversation state. (**`assistant.model.messages` at call start,

call.messages

at squad transfer points).