https://vapi.ai/ logo
Join Discord
Powered by
Assistant does not say "title of event" when givin...
# support
f
Hi @Vapi team, We've done tedious testing on this for awhile now and havent been able to find a solution. Our Assistant is not saying the Title of Events or Promotions consistently. Here's a great example of when the Assistant shares some Weekly Promotions to a customer but does not say the Title which is confusing. CALL ID: b4623aa4-8f3c-4db1-9d68-74fcd847042c What went wrong: - 02:03 Assistant shares information on the promotion but does not say that Promotion Name. - 02:13 Assistant shares the next promotion and does not say the Promotion Name. It continues to do this for all promotions that are listed in the knowledge base that it is referring to. What we expect to happen: when referring to any Event or Promotion, it should say the Name before sharing details. This information is stored on the KB - I have attached the Word Document for reference along with a screenshot of how it shows in Trieve. We have put Title Name in Quotion Marks and BOLD text in the document https://cdn.discordapp.com/attachments/1347376960354914355/1347376960829128725/promotions_-_knowledge_base.JPG?ex=67cb9a14&is=67ca4894&hm=8e78117ae9e594b7999cbccefdee98b89a48d44e96f1afccb6118c59cd894df8& https://cdn.discordapp.com/attachments/1347376960354914355/1347376961084985436/Trieve_KB.JPG?ex=67cb9a14&is=67ca4894&hm=1964e24e12c11b8fc9a222da34e287fcd5139f811ac5c93beced398a70579e54&
k
Let me investigate this call ID and return with outcomes and conclusions.
f
Hi @Shubham Bajaj , any update here?
Hi @Shubham Bajaj checking if there is any update here?
k
Hey Beck, looking into this now.
Beck, could you do me a favor and try it after making your RULES FOR HANDLING ENQUIRIES section around it a little more specific? Along the lines of "retrieve the correct title of the promotion along with the details"?
f
Hi @Mason, i appreciate your input. I've gone ahead and tested this with a number of variables and no luck. I have added into Prompts: > Always retrieve the correct title of the Club Promotions and Draws along with the details This did not work, so I tried to define: > When talking about Club Promotions or Draws always retrieve the correct Title and mention the Title of the promotion or draw along with the details. I also tried adding: > The Format is: "Title" - Description (and matched the KB content to this format) Here's the Call ID if need: 7fed12ce-6320-4349-bd27-726e2eee9148 Unfortunately no luck with these inputs. Any insight?
Hi @Shubham Bajaj & @Mason - any update on this? We find this happening across multiple topics
k
Hey Beck, This is happening due to your formatting plan. Please disable markdown from your formatting plan. Here are some docs on the topic:
f
Hi @Mason Thanks for your input. I have revisited the formatting of the KB information so we can keep markdown enabled. Although, still having issues and hoping to find a workaround. *With this test, we have the following prompt: * > When talking about Club Promotions or Draws always retrieve the correct Title and mention the Title of the promotion or draw along with the details. *In the KB file, we have the following structure: * # # Weekly Draws Title: Draw Name 1 Description/Answer: “Runs every Monday and Wednesday. They start from 11:00 A.M. and drawn every 30 minutes. Prizes include cash and vouchers. Ends 26th February.” Title: Draw Name 2 Description/Answer: “Runs every Tuesday from 6:00 P.M. happening every 30 minutes. Cash prizes to be won. At 8:00 P.M. win up to $X on the jackpot. For more details on Draw Name 2, visit our Regular Events tab on the “What’s On” page on our website.” Title: Draw Name 3 Description/Answer: “Is every Thursday night and Sunday afternoon. A chance to share in our Draw Name 3 Jackpot. Check in club for conditions.” When asked "what weekly draws do you have running?" the Assistant still does not say the Title and goes straight to saying the description. Assistant is also saying "1.. description, 2.. description.. etc. Call ID: 6abc8b99-5634-4aa8-a135-bfb59b7a8759
k
Hey Beck, I just looked into your call ID, and looking at the voice input formatted call logs, I can still see why some of your output is being trimmed off because of special markdown syntax. So I suggest you to tweak your assistant voice formatting plan to not to format these special markdown symbols and other symbols which are part of your KB. Also, I'm DMing you the call logs for this call ID shared so that you can also take a look.
Let me know how it goes for you. If you have more questions or require further help, feel free to reach out.
f
Hi @Shubham Bajaj , we have testing this and we are still having the same issue. We have removed the " and have tried many alterations of prompting with no luck. Are you able to supply a prompt example we can test with the above content?
s
Just share those call Ids i will do it
f
Hi @Shubham Bajaj , here is the callID: 53d081d6-8127-485b-88ef-869681a10ec2 I will DM you our document so you can see what we have for the KB. Thanks!
Hi @Mason just wanted to check in to see if you have any insight on this. We are having this issue across multiple Assistants and have tried many variations. We have even put information into sentence like structure and Assistant still does not say the names consistently. Here is a recent example. CallID: 30fad314-69fb-4da5-8a03-2f15e529daca Assistant said Title of Packages correctly. CallID: 1e333fa2-afae-4f4d-9819-a2d10d341456 Assistant does NOT say title of packages. CallID: 80de2a86-26d4-4a74-a818-709f03509294 Assistant does NOT say title of packages I've attached the content for reference. https://cdn.discordapp.com/attachments/1347376960354914355/1357516320484364288/image.png?ex=67f07d18&is=67ef2b98&hm=bd3fc3680f54c4ef7088dadb37b44724d82bc37ad221f720ff8cfdb3650781bc&
k
Hi, checking if this is resolved/solved for you?
s
🔵 00:40:41:071 Voice Input Formatted: "Hi, thanks for calling Club Southport! This is Millie, how can I help?", Original: "Hi, thanks for calling Club Southport! This is Millie, how can I help?" 🔵 00:40:56:035 Voice Input Formatted: "Hi there! For functions at CSi Club Southport, we offer a variety of beverage packages to suit your needs:. - : You can choose your limit and beverages. - : Guests can pay as they go. - : Available at three dollars and fifty cents per person. If you need more detailed information or have specific requirements, our Functions Manager can provide all the details.", Original: "Hi there! For functions at CSi Club Southport, we offer a variety of beverage packages to suit your needs: - **Bar Tab**: You can choose your limit and beverages. - **Cash Bar**: Guests can pay as they go. - **Tea, Coffee, and Water Station**: Available at $3.50 per person. If you need more detailed information or have specific requirements, our Functions Manager can provide all the details."
@firefly_0206 Hey , we send out the formatted LLM output before sending them to the voice providers. During this phase/step, some of your LLM-generated content was trimmed off (as you can see in the logs). This is called LLM voice formatting plan. This is done so that a better output can be generated by the voice provider. The good part is you can toggle it off according to requirements. You just need to change your assistant voice formatting plan which comes under your assistant voice chunk plan. Then formatting plan, and then you can turn off unrequired LLM utilities which are in your case which are related to Markdown.
Copy code
bash
curl -X PATCH "https://api.vapi.ai/assistant/{ASSISTANT_ID}" \
     -H "Authorization: Bearer {YOUR_API_KEY}" \
     -H "Content-Type: application/json" \
     -d '{
       "voice": {
         "chunkPlan": {
           "enabled": true,
           "formatPlan": {
             "enabled": true,
             "formattersEnabled": [
               "acronym",
               "dollarAmount",
               "email",
               "date",
               "time",
               "distance",
               "unit",
               "percentage",
               "phoneNumber",
               "number"
             ]
           }
         }
       }
     }'
This configuration: 1. Explicitly keeps the chunk plan enabled (
"enabled": true
) 2. Explicitly keeps the format plan enabled (
"enabled": true
) 3. Only includes non-markdown related formatters by removing: - 
"markdown"
- directly handles markdown symbols like underscores, backticks, tildes - 
"asterisk"
- handles text between asterisks (markdown for bold/italic) - 
"quote"
- handles quotes (can be part of markdown formatting) - 
"dash"
- handles dashes (used in markdown for lists) - 
"newline"
- handles newlines (markdown uses these for formatting) - 
"colon"
- handles colons (can be used in markdown formatting) The remaining formatters focus only on content-specific formatting like numbers, dates, acronyms, etc., rather than markdown-specific syntax elements. Remember to replace: - 
{ASSISTANT_ID}
with your actual assistant ID - 
{YOUR_API_KEY}
with your Vapi API key
f
Hi @Shubham Bajaj , we understand the Titles are removed from speech due to markdown. We are wanting to keep the markdown setting as is and not turn it off, so in doing so we adjusted the content structure **from**: - Bar Tab: You can choose your limit and beverages. - Cash Bar: Guests can pay as they go. - Tea, Coffee, and Water Station: Available at $3.50 per person. **To**: - Bar Tabs available choose your limit and beverages. - Cash bar available where guests can pay as you go. - Tea, Coffee, Water Station with price $3.50 per person. With this, the Assistant should assumedly read the content as it is and not "remove" the Titles? But this is not the case, even with this structure the Assistant does not mention Bar Tab, Cash Bar or Tea, Coffee, Water Station even when markdown is not applied. Is there a reason for this?
@Shubham Bajaj We also have removed bullet points and added as sentence structure: ~ Beverage Packages. Bar Tab available choose your limit and beverages. Cash Bar available where guests can pay as you go. Tea, Coffee and Water Station available with price $3.50 per person. With this, Assistant still did not say the titles of the packages. I have just tested this again today - CallID: b8a4b7c2-e83f-4676-86af-ff96fc66a9ed
@Shubham Bajaj After more testing, I have also noticed the Assistant does not say the Titles of Packages (or Titles of Events) when the Assistant is communicating information as a "list". For example.. Test 1: We asked "What beverage packages do you have available?" Assistant answers in a "list form" and DOES NOT say Titles of Beverage packages Call ID: 9b4c3e9a-4151-4226-845a-74071a4277fc Test 2: We asked "What other packages do you have available other than Cash Bar?" Assistant answers in a more conversational manner and says Titles of Beverage Packages. Call ID: 85132a4f-5716-4ace-b5e8-7db571323fb7 This seems to be consistent across multiple Live Assistants on different topics including Event Title, Promotions, Games etc. We want to ensure that all Assistants says the Titles no matter how the information is delivered (e.g as a list form or conversational).
k
Copy code
"voice": {
        "provider": "11labs",
        "voiceId": "LtPsVjX1k0Kl4StEMZPK",
        "stability": 0.6,
        "similarityBoost": 1,
        "style": 0.1,
        "useSpeakerBoost": false,
        "speed": 1,
        "optimizeStreamingLatency": 3,
        "model": "eleven_turbo_v2_5",
        "chunkPlan": {
          "enabled": true,
          "minCharacters": 30,
          "punctuationBoundaries": [
            ".",
            "，",
            "!",
            "?",
            "،"
          ],
          "formatPlan": {
            "enabled": true,
            "formattersEnabled": [
              "markdown",
              "asterisk",
              "newline",
              "colon",
              "acronym",
              "dollarAmount",
              "email",
              "date",
              "time",
              "distance",
              "unit",
              "percentage",
              "phoneNumber",
              "number"
            ],
            "numberToDigitsCutoff": 2100,
            "replacements": []
          }
        },
        "fallbackPlan": {
          "voices": []
        },
        "inputMinCharacters": 30,
        "inputPunctuationBoundaries": [
          ".",
          "，",
          "!",
          "?",
          "،"
        ],
        "inputPreprocessingEnabled": true,
        "inputReformattingEnabled": true
      },
Hey back, the solution is already shared with you. In case you don't apply it, then I cannot help you further. You can look into the voice config which you are currently using against recommended.
3 Views
PreviousNext
Powered by