https://vapi.ai/ logo
Join Discord
Powered by
Custom LLM Transfer Call Not Working
# support
s
https://gist.github.com/bajajcodes/1dc66d7efd2250155885e03d180ac152 The payload.destination is not getting sent to me. Based on the code above its required to perform transfer calls for cutom LLMs.
@User
@User
v
Hey here's the right resource
can you share the call ID, so I can take a look?
s
e5faec79-d8fc-4cb9-ac81-d46bbe18d564 @shubham.bajaj
I’m getting a pipeline error for this one
@Shubham Bajaj
The llm is calling the tool, but in the example there is a payload.destination which comes from the body. That I’m not getting.
What is the flow here?
Hey @Shubham Bajaj, this a bit urgent, the call forwarding isn’t working which means we can’t deploy the agent for our customers.
v
✅ Ticket status updated to solved by Vapi Ticket Bot#0124
s
@Shubham Bajaj This is what I am streaming back to vapi {"function_call":{"name":"transferCall","arguments":{"destination":"+14045633811"}}}
@User
@User
This is super urgent 🚨
Our agent is not able to transfer the call
s
@slickVik Looking into it, your tickets was missed from list. Sorry for it.
logs
🔵 21:34:50:468 Couldn't Complete Completion Request (#1, provider: custom-llm, model: gpt-4o, region: undefined, credential: true) Error: { "message": "chunk.choices is not iterable", The error "**chunk.choices is not iterable**" occurs because the custom LLM response doesn't match the expected OpenAI-compatible format. We expect the response to follow the OpenAI streaming format, which should include a choices array property that is iterable. When this property is missing or not an array, the error occurs. [expected]
Copy code
{
  choices: [
    {
      delta: {
        content?: string,
        tool_calls?: Array<...>,
      },
      finish_reason?: string
    }
  ]
}
Do let me if you still require further help.
s
Copy code
{
  "id": "chatcmpl-B2PihK9m7Yag0aBdJdhWyG36Rpa93",
  "object": "chat.completion",
  "created": 1739915223,
  "model": "gpt-4o-2024-08-06",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_dTmZivlvze6Z7SGqcAxUEftq",
            "type": "function",
            "function": {
              "name": "transferCall",
              "arguments": "{\"destination\":\"+14045633811\"}"
            }
          }
        ],
        "refusal": null
      },
      "logprobs": null,
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 1805,
    "completion_tokens": 18,
    "total_tokens": 1823,
    "prompt_tokens_details": {
      "cached_tokens": 1792,
      "audio_tokens": 0
    },
    "completion_tokens_details": {
      "reasoning_tokens": 0,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  },
  "service_tier": "default",
  "system_fingerprint": "fp_523b9b6e5f"
}
I tried sending this back as well, but got a tool type not set error
@Shubham Bajaj
Does it have to be stream of chunks that’s set back?
v
The issue occurs because Vapi expects the response to be streamed in chunks following the OpenAI streaming format, rather than a complete response. The handler expects each chunk to be in a specific delta format. The complete response you sent, while valid OpenAI format, doesn't match the streaming format expected by the handler. Here's how you should structure your response stream for the transfer call:
Copy code
// First chunk - Start the stream and set tool call ID

{
  choices: [{
    delta: {
      tool_calls: [{
        index: 0,
        id: "call_dTmZivlvze6Z7SGqcAxUEftq",
        type: "function",
        function: {
          name: "transferCall"
        }
      }]
    }
  }]
}

// Second chunk - Send the arguments
{
  choices: [{
    delta: {
      tool_calls: [{
        index: 0,
        function: {
          arguments: "{\"destination\":\"+14045633811\"}"
        }
      }]
    }
  }]
}

// Final chunk - End the stream
{
  choices: [{
    delta: {},
    finish_reason: "tool_calls"
  }]
}
This is the correct format because:The streaming handler processes chunks incrementally. **TLDR: I**nstead of sending the complete response, you need to stream it in chunks. The error "tool type not set" occurs because the handler expects the type to be set in the first chunk before processing arguments.
s
Great! Thank you for the help!
v
Marking this ticket as Solved ✅
PreviousNext
Powered by