Good day. I would like to request assistance regarding the following calls: e7a6c197-cad3-42aa-a32d-c12621229cf2 and e4b9a896-7120-457d-be63-8a4c692caeac. These calls are from a translator. This has happened multiple times already; for some reason, the assistant sometimes adds additional words or phrases at the end of sentences when translating. For example, on Call e4b9a896-7120-457d-be63-8a4c692caeac at 3:12, User Sure. Send it to the email address send it from the email address you have on file. AI Claro, envíelo desde la dirección de correo electrónico que tiene registrada. Está bien. Gracias. The AI added 'Está bien. Gracias', which is 'Ok. Thank you' in English. But the User did not say that. And on Call e7a6c197-cad3-42aa-a32d-c12621229cf2 at 2:20, User Soy Martínez S Noelia dos tres siete arroba Gmail punto com AI I am Martinez S. Noelia 2 37 at gmail dot com. How about that? The AI added 'How about that?'.

@BCX-JKD Good day! Thank you for bringing this to our attention. It seems like there might be an issue with the translation feature adding unintended phrases. I will look into the calls you've mentioned and ensure that the problem is addressed. In the meantime, is there anything else I can assist you with regarding this matter or any other concerns you might have?

Hi, are there any updates? Also, we are encountering additional issues with spelling. When a Spanish speaker spells their name or email address using Spanish pronunciation, it is not being correctly translated into English, even though it appears correctly in the User's transcript. Here are some Call IDs: bd402a81-82ae-4133-b9c3-f23b1b466a1e at 0:48, User R U I Z F L O R E S AI Are you Eiz F L O R E S 15d6a542-cc9d-443d-97d0-6c85ac994772 at 02:08, User Sí. Es a r a g t z Diaz arroba G mile punto com AI Yes. It's ragdiazgmail dot com. 412bf0e5-e162-4d33-9cf4-9417cbe53617 at 0:56, User ESMA punto RU I Z F L O R E S arroba Hotmail punto com. AI Esma. Ruizf l o r e s ask audrahotmail dot com. ecc73f92-5cb6-420b-8340-220448984020 at 01:18, User H E R G A T O AI h e r this a to o. User h e r g a t o. AI H e r g a t o. User U n d e r s t a n d. AI Entiendo, de acuerdo, perfecto, gracias. 46ac0464-ddd3-4c84-9149-e5443d78c3b8 at 01:30, User A a r a g t z días gmail.com AI A r a j t z d I a z mail dot com.

@BCX-JKD looking into this, give me sometime to get back to you.

Any update regarding this issue? We are also having an issue with DOBs. Here's the call id: d69376c3-3e8b-4500-9806-0888f1337e10

@BCX-JKD Upon reviewing your call ID, I have identified two potential solutions(for email issue only): one involving prompting and the other concerning a change in the transcriber. I will outline both approaches below, with the first two pertaining to prompting and the last one related to the transcriber. 1. Request that the user spell their email address one character at a time. For instance, instruct them to articulate: "g - o - l - d - m - e - i - e - r, dot, d - e." After they have spelled it, kindly repeat it back for confirmation: "I heard 'g-o-l-d.' Is that correct? Please confirm or correct me." 2. Encourage users to adhere to a structured input process. For email addresses, prompt them to articulate each component clearly and separately, using "at" for "@" and "dot" for "." For example: "Please state your email address letter by letter. For '@,' say 'at,' and for '.', say 'dot.'" 3. As an alternative, you may consider utilizing the Azure transcriber.

Yeah, I don't think instructing customers to spell out their email address fits our use case. We don’t want to add extra instructions for our live agents, nor do we want them telling Spanish-speaking customers to use 'at' and 'dot' instead. Is there a way to fix these issues without changing the customer service process? For now, we will try Azure and see if that will make a difference.

@BCX-JKD You may consider utilizing Azure and other prompting alternatives. I understand the challenges you face, as emails are not easily captured using transcribers, particularly for non-English languages. By the way, could you please clarify why you are converting Spanish user input to English?

What we are working on is a translator that will allow our English-speaking live agents to communicate in real time with our Spanish-speaking customers. The translation part is working; our biggest issue right now is with the email addresses and dates of birth. Also sometimes the AI will randomly add extra words or phrases that aren't part of the User input.

> Also sometimes the AI will randomly add extra words or phrases that aren't part of the User input. Share the call id, and expected vs actual responses(try to be more specific) so i can easily pin-point the issue. > dates of birth. I can help with these, can you share the call ID so I can take a look again with expected vs actual response. > email addresses Try sharing the recent call ID, so I can share with transcriber service providers again with expected vs actual response.

> Also sometimes the AI will randomly add extra words or phrases that aren't part of the User input. Call ID: e4b9a896-7120-457d-be63-8a4c692caeac at 3:12, User Sure. Send it to the email address send it from the email address you have on file. AI Claro, envíelo desde la dirección de correo electrónico que tiene registrada. Está bien. Gracias. The AI added 'Está bien. Gracias', which is 'Ok. Thank you' in English. But the User did not say that. And on Call e7a6c197-cad3-42aa-a32d-c12621229cf2 at 2:20, User Soy Martínez S Noelia dos tres siete arroba Gmail punto com AI I am Martinez S. Noelia 2 37 at gmail dot com. How about that? The AI added 'How about that?'. > dates of birth. Call ID: d69376c3-3e8b-4500-9806-0888f1337e10 at 02:28 User Doce veintisiete de mil novecientos noventa y seis. AI Do I think in 20 17, 19 99 96. It should be 12 27 1996 > email addresses Call ID: 412bf0e5-e162-4d33-9cf4-9417cbe53617 User ESMA punto RU I Z F L O R E S arroba Hotmail punto com. AI Esma. Ruizf l o r e s ask audrahotmail dot com. It should be ESMA . RUIZFLORES @ HOTMAIL . COM Also Call ID: 0329fab0-c306-40b5-92df-668d3e362bc1 at 0:55, User Claro, es Martínez S Noelia dos tres siete arroba Gmail punto com AI Sure. It is Martinez s Noelia 2 and 37 air gmail dot com. It should be MARTINEZ S NOELIA 2 3 7 @ GMAIL . COM

@BCX-JKD > The AI added 'How about that?'. It's added because of your prompt spanish to english conversion requirement, try one call with spanish prompt and then it won't appear again without conversion. > email_address Let me think of some alternative for email, for non-english languages.

@Shubham Bajaj Got it. Can you clarify what you mean by the conversion requirement and to try one call with spanish prompt? Should I write the prompt in spanish?

@BCX-JKD Yes write your prompt in Spanish. As your current assistant necessitates that user input be translated from Spanish to English for the LLM, and subsequently translated back to Spanish by the LLM, additional words may be incorporated by the LLM. Please ensure that every component of the assistant remains in native Spanish, and then proceed to test it. I would appreciate it if you could inform me of the results.

I understand. Will do. Just a quick question, it won't affect English to Spanish translations, correct? Because the agents will still speak in English and we want those to be translated into Spanish for the customers.

@BCX-JKD I would like to confirm whether you require a single agent capable of communicating in both English and Spanish. If this is indeed the case, you may create prompts in either languages however, I would recommend maintaining the prompts in English exclusively and utilizing an additional system message to establish response generation boundaries. Please ensure that you employ Deepgram as the transcriber along with 11Labs' multilingual voice capabilities. Furthermore, you may utilize Deepgram's multi-model functionality for this purpose. It would be beneficial to include a line instructing the LLM to generate responses in the user's language consistently, such as: "Generate your response either in the user's input language only, or if unknown, default to English for response generation." Do correct me if I am wrong here.

What we're building is essentially an interpreter/translator. When a Spanish-speaking customer calls us, our English-speaking agents will conference in the AI assistant, making it a three-way call. When the customer speaks, the AI will translate that into English, and when the agent responds, the AI assistant will translate it into Spanish.

@BCX-JKD It will be better to do it (spanish to spanish) customer-assistant conversation because then you won't have the word errors when comes to english to spanish conversions. Later you can generate the analysis in english langusge but if you want to continue with existing process you have to be secured of voculabory mistakes by LLM.