Deepgram nova-3 keyterm not working
# support
j
I've tried multiple ways of adding keyterm to my assistant's transcriber via the API but each time I've added some, the call would not work. TO DO TO REPRODUCE : add key term to the assistant via the API For example : "transcriber": { "model": "nova-3", "language": "multi", "numerals": false, "provider": "deepgram", "endpointing": 300, "keyterm": ["Eva", "Jean", "Benoit", "Henry", "assistante", "virtuelle"], "fallbackPlan": { "transcribers": [ { "model": "gemini-2.0-flash", "language": "Multilingual", "provider": "google" } ] } }, **OBSERVED **: call happening without transcription, hence closing after silence timeout. **EXPECTED **: 1. The call to happen normally, deepgram connection working smoothly 2. If not, the transcriber should use the fall back plan and use my other transcriber (Google gemini flash) I have set up. Call id example : e24e6fb3-28e4-497a-8c05-87ba8c513219 From assistant id : 8abc3963-a71e-43e3-8465-ae8568d27179
v
Hi there! 👋 I see you're having trouble with Deepgram nova-3 keyterm functionality in your Vapi configuration. I'm unable to determine the exact cause of why your fallback mechanism isn't working as expected. However, I can definitely help with the Deepgram nova-3 keyterm issue based on information I've found. Looking at your configuration, I've identified a few potential issues: 1. **Language Setting Issue**: According to the information I found, nova-3 has restricted language support and currently only supports 'en' and 'en-US' - not 'multi'. Your configuration shows:
Copy code
json
   "language": "multi"
This could be causing the problem as [nova-3 only supports English languages](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1341463939246198784). 2. **Keyterm vs Keywords**: There's an important distinction between these parameters. For Nova-3, you should specifically use
keyterm
(which you are), while Nova-2 uses
keywords
. You've got this part correct, but it's worth confirming. 3. **Potential 502 Error**: Some users have reported that Deepgram on Nova-3 returns 502 errors when certain configurations are used. This might be happening in your case as well. Here are some troubleshooting steps you can try: 1. Change the language setting to 'en' or 'en-US' instead of 'multi' since nova-3 only supports English.
Copy code
json
   "language": "en"
2. **Check Deepgram Status**: Verify if [Deepgram is experiencing any degradation](https://status.deepgram.com) that might be affecting your integration. 3. **Verify your Vapi configuration**: Double-check your API keys and permissions to ensure they're correct and haven't expired. 4. **Try without keyterms first**: Test your configuration without any keyterms to establish a baseline, then add them back one by one to identify if a specific term is causing issues.
5. **Check your network activity**: Use developer tools to inspect for any failed requests or errors when attempting to use the keyterm functionality. Let me know if any of these suggestions help resolve your issue or if you need further assistance! Sources: - [Deepgram nova-3 keyterm known issues and limitations](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1341463939246198784) - [Deepgram nova-3 keyterm functionality and usage](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1340021048342610104) - [Troubleshooting Deepgram integration with Vapi](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1296852804631265392)
j
So it works without the keyterm. Error is the following : 17:15:50:059[ERROR] Transcriber failed (reason: Received network error or non-101 status code.) I suspect it's due to the new multilingual language of Deepgram and its keyterm compatibility ?
@Vapi @Shubham Bajaj any help here ?
v
checking if this is resolved/solved for you?
j
@Shubham Bajaj no, not solved. Call id in my initial post is still valid. This one can be used as well : 28931920-e1af-49a7-a4c8-b6793a565384. Every time I specify keyterms with nova-3 multi, it fails to launch the transcriber
@Shubham Bajaj any news on nova-3 keyterm ?
v
🔵 17:15:50:059 Deepgram Websocket Error: { "message": "Received network error or non-101 status code." } 🔵 17:15:50:060 Deepgram Transcriber Error: pipeline-error-deepgram-returning-502-network-error Hey Jeebs, This error has been resolved. It was originated from Deepgram side due to a spike in network requests. I'll suggest you try making another call now and let me know how it goes for you.
j
@Shubham Bajaj this is getting very weird : I can't even set my transcriber to nova-3 anymore (see attached video). Even without keyterm or whatsoever ! Same issue with the UI where I can change to nova-3 but when "getting" the assistant via API it still says nova-2. Assistant id : 7727ea52-ef05-4ad2-ab4b-658b70d3ff36 https://cdn.discordapp.com/attachments/1357042489554632794/1359450670339457044/Nova3-Issue.mp4?ex=67f78699&is=67f63519&hm=3192c06d69129e1bbc84b787f81922e2f258af54033be3a7e1407558104f5e0c&
@Shubham Bajaj ? I know you have a lot on your plate 😬 Sahil mentioned you'd come back to me about this issue
Today I could again set nova-3 to my assistants. But I'm stuck with the initial problem : as soon as I set keyterm on my assistant transcriber, the assistant fails. In VAPI logs, it says "*Transcriber failed (reason: Received network error or non-101 status code.)** Call id example: 2e058cbc-91bf-44c7-99b1-2e1ad86703ca
"keyterm": "Florence Wéra docteur Genappe"
Is there soemthing wrong in there ?
Total transcriber setup :
Copy code
"transcriber": {
        "model": "nova-3",
        "language": "multi",
        "numerals": false,
        "provider": "deepgram",
        "endpointing": 300,
        "confidenceThreshold": 0.2,
        "keyterm": "Florence Wéra docteur Genappe"
    }
k
Got it will get back to you soon
s
@Jeebs from eva.be looking into it.
Copy code
/**
   * Keyterm Prompting allows you improve Keyword Recall Rate (KRR) for important keyterms or phrases up to 90%.
   */
  @IsOptional()
  @IsArray()
  @IsString({ each: true })
  @ValidateIf((o) => !o.model?.includes('nova-3') && o.keyterm !== undefined)
  @Equals(undefined, {
    message: 'keyterm must be null if model is not nova-3',
  })
  keyterm?: string[];
Copy code
curl -X PATCH "https://api.vapi.ai/assistant/YOUR_ASSISTANT_ID" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "transcriber": {
      "provider": "deepgram",
      "model": "nova-3",
      "keyterm": ["important", "terms", "to", "recognize"],
      "language": "en"
    }
  }'
@Jeebs from eva.be You need to pass the key term as an array of strings instead of a string with terms separated by spaces.
@Jeebs from eva.be Let me know how it goes for you
j
@Shubham Bajaj tested both format, but still the error. This call for example I used the right one : ed710336-5123-494b-a62d-5b1765e218b8 Still the same error Or this one (removed the "é" in keyword) : 52ab8c8a-b4d0-4c0c-8d3d-3aaac221c5c7
Copy code
"transcriber": {
        "model": "nova-3",
        "language": "multi",
        "numerals": false,
        "provider": "deepgram",
        "endpointing": 300,
        "confidenceThreshold": 0.2,
        "keyterm": ["Florence", "Wera", "docteur", "Genappe"]
    }
v
The first call ended because the customer entered the call.
Copy code
🔵 14:15:12:447 Initializing Deepgram. Options: {
  "punctuate": true,
  "numbers": true,
  "no_delay": true,
  "encoding": "linear16",
  "channels": 2,
  "multichannel": true,
  "filler_words": true,
  "interim_results": true,
  "language": "multi",
  "model": "nova-3",
  "sample_rate": 44100,
  "mip_opt_out": false,
  "keyterm": ["Florence", "Wéra", "docteur", "Genappe"],
  "endpointing": 300,
  "numerals": false
}...
 "endedReason": "customer-ended-call",
Hey Jeebs. Okay, second call is a completely different story. Let me check with the Deepgram's team what could be the possible issue for this.
k
@Jeebs from eva.be
j
@Shubham Bajaj indeed, I have ended the call because there was no answer 😬 Basically, when there is no transcriber, no sound get captured hence also no answer get voiced by the LLM & voice models. This happened in both calls ; I've ended the call because I knew it wasn't working. In both cases, the logs show an error for the transcriber.
v
Copy code
"transcriber": {
        "provider": "deepgram",
        "model": "nova-3",
        "keywords": [],
        "language": "multi",
        "smartFormat": false,
        "endpointing": 300,
        "keyterm": [
          "Florence",
          "Wéra",
          "docteur",
          "Genappe"
        ],
        "codeSwitchingEnabled": false,
        "mipOptOut": false,
        "numerals": false,
        "fallbackPlan": {
          "transcribers": []
        },
        "confidenceThreshold": 0.2
      }
As you can see, Deepgram's key terms are only supported in English. Either you have to remove them or change the language of your transcriber. https://cdn.discordapp.com/attachments/1357042489554632794/1361120594598432849/image.png?ex=67fd99d6&is=67fc4856&hm=aa78f21db47514f0798c78b343496bf310745e36233ce2e3a79db5f5939f760d&
j
😨 you are right @Shubham Bajaj, their nova-3 is really under-par in the end 😭 Let me know if you plan to integrate the latest gladia model, that seems the best shot for multilingual ! I'll mark this ticket as closed
v
Sure.
2 Views