Initiate tool call as first action upon call start
# support
d
Hello, fwiw I am very pleased with my vapi experience thus far. I was an early adopter and I’ve returned to see that you all have been putting in an impressive amount of work on the platform. I have a use case where I would like to have my Vapi assistant invoke one of my custom tools immediately upon call initialization in order to bring in extra context about the caller from my API into the conversation. I’ve not been able to figure out how to do this via prompting, does anyone have any prompt hacks or hidden Vapi features that I might be able to utilize to achieve my goal? Thank you
v
You can continue this conversation by mentioning me in the message.
@DollarD Hello! I'm delighted to hear that you're enjoying your experience with Vapi. It's great to see users like you making the most of the platform and coming back to explore new features. For your use case, you can achieve the immediate invocation of a custom tool to bring in extra context about the caller by using the
assistantOptions
during the initialization of your Vapi assistant. You can set up a custom tool to be called as part of the assistant's startup sequence. Here's a general idea of how you can configure this: 1. Define a custom tool that fetches the caller's context from your API. 2. Include this custom tool in the
assistantOptions
when you start your Vapi assistant. If you need assistance with the specific code or setup, please provide more details about your custom tool and how you've structured your assistant, and I'll be happy to help you integrate it. If this is something that isn't currently supported directly through the Vapi Dashboard or API, I would recommend reaching out to the support team for more information on potential workarounds or upcoming features that might meet your needs.
d
that doesnt seem configurable via the vapi dashboard, which is what i need.
@Vapi do you know where in the vapi dashboard if anywhere that i can configure that?
v
@DollarD I apologize for any confusion. If the functionality to invoke a custom tool upon call initialization is not readily apparent or configurable through the Vapi dashboard, it may not be a feature that is currently available through the standard user interface. However, I would recommend reaching out to the Vapi support team for more detailed assistance. They may be able to provide you with information on whether this feature is in development or if there are any alternative methods to achieve your goal. You can contact support by visiting the Vapi dashboard and looking for a support or contact option, which is typically available in the form of a help center, chat support, or a contact form. If you would like me to assist you further with any other questions or if you need guidance on how to contact support, please let me know.
c
just set it to async......
d
i am open to trying that, but i am not able to find it in the vapi dashboard i also fail to see how async is going to meet my requirement of invoking a tool on call start, but as i said i am open to trying anything at this point do you know where in the assistant dashboard i can make the suggested changes?
c
It sounds like you don't know how tools work You can have a tool trigger when the call first starts.
d
you seem to have a bit of an unjustified ego
if you can trigger a tool when the call first starts, that would be great if you would tell me where to configure that in the vapi dashboard, instead of giving such attitude πŸ™‚ and yes, if you dont want to help me now, i don't really care that much, as you did not seem like quite the helpful individual in the first place
c
I am not sure how I have an ego. From what you type it does not sound like you understand tools. You can trigger a tool async or sync. meaning a tool can be triggered in the background when the call is started. I have an ego because I am trying to explain that? ok. Your talking about the dashboard when what you are talking about is in the API - but I have an ego.... I help everyone in the same way. I give an answer. I am not responsible because you are in your feelings about something.
d
do not worry, i am not in my feelings πŸ™‚ "just set it to async" and "Sleeping emoji" on your own post is a sign that you probably don't know how to interact with people you can give assistance without being on a high horse, especially when you haven't proven yourself lol nontheless, For my implementation, I am pretty limited to what I can configure in the dashboard. I've got some custom tools that I want to call on call start. Trust me, I am intimately familiar with the concept of tools etc, just perhaps not entirely with the VAPI dashboard Either you know how to configure the VAPI assistant to do a tool call immediately on call start, in which case, great, please let me know how. If not, I will leave the floor open to someone who might be able to assist me. "Make it async", as i said, is something i am willing to try, but i cannot find it in the dashboard. Ideally, In the same way that VAPI can call my microservice when a call is completed via
end-of-call-report
, i would love that it could call my service when a call is started to so that I can inject context.
c
I am on a high horse because you are admitting that you don't fully understand how VAPI works. πŸ‘πŸΏ I gave you an answer but you are focusing on the wrong things entirely. Instead of testing the answer I gave you you started to get into your feelings.
d
u must be trolling
c
Your upset that I set an emoji on my post because you didn't read the documentation that tells you about async and sync settings.πŸ‘πŸΏ
d
setting the tool to async will not make it execute on call start
are you daft mate ?
c
I wish you the best of luck DollarD If you actually went around the discord you would see I actually help out all the time - but once again you want to be stuck in your feelings go ahaead.
If you create a propmt to trigger a tool on the call start it would - if you have not tried it then you should.
Why would I give you an idea that purposely does not work? That makes no sense whatsoever.
d
well, i finally found where to configure async in the dashboard, which i asked about to you around 3 times already, i will try it and see how it goes, if it works, then thank you. but let it be known i am not in my feelings, i just don't like your attitude
c
You have never met me to know if I have any attitude - I should be the one upset but I really dont care. You can focus on building the solution. or projecting about someone who has never met you.
d
btw the async does not work - it will call my endpoint and retrieve context on call start but it will not wait for the context before responding (becasue of the async nature, obviously). it is important that there is context injected into the chat at the start of the call from my database, but also that the assistant waits for the response from the server befroe moving forward with the chat a "call started" webhook that allows me to return context in the response body would be ideal. going to see if anything like this is possible with status updates, but doubtful
c
You can set the context to say nothing - you have to edit through the API. Call starts - tool is triggered - no response - all information is pulled,etc
And if you have a database with the information why are injecting it? Just use it in your prompt. I see you guys doing that and just making it way harder. Dashboard is dynamic whatsoever. Use the API and add that information. Does not matter how many people you are calling.
d
there are many reasons why one would use a database instead of just injecting it directly into the prompt. like in my case, where the information changes based on who is calling πŸ™‚
c
Most databases are using custom feilds etc - so yea its different for every person but its the ame field. Literally put that in the prompt instead of injecting it every time. That is the case for everyone lol - Why do people think they are the only ones with a database with different information.
d
let me flip it on you -
It sounds like you don't know how vector databases work
πŸ˜‚
c
Let me flip it on - you did not say vectordatabse - just database 🀣
d
it doesn't matter - my clarification that the data changes on a per-caller basis should have been enough for you to understand that "just inject it into the prompt" is not a viable solution
the more you talk the more its clear you dont know much
c
Most are not injecting a vector database in their prompt. When database is said its usually referring t a CRM of some kind
Becuase you didnt clarify and still dont know how VAPi works? i love your thinking and logic.
I can see why you are struggling.
d
there's no struggle m8
c
Apparently.
d
ur a bigger struggle than the question i asked, would prefer if u would leave the thread so that someone who actually understands the problem can help resolve it (it might need to be a feature request )
c
Def not _ I am person who helps people stop struggling. Your just in your feelings for some reason because you didn't fully clarify you are using a vector database and you don't understand a platform that you have not used in a long time (your words). And you project a lot.
d
anyone else want to tackle this problem? i think it might be a feature request @User a "call started" webhook that allows me to return context in the response body would be ideal. going to see if anything like this is possible with status updates, but doubtful thanks
@User I managed to resolve this through some advanced prompting techniques involving some workarounds, but having the ability to invoke a web a webhook for initial context on call start would make this much more consistent, as my prompt based solution is not flawless and will sometimes not work as expected
s
you can use server event request assistant to get caller info.
d
So, I can achieve this behaviour by un-assigning my assistant to a phone number in the vapi dashboard, and then it will ask my server who the caller should be connected to?
awesome, i think i can get it working based off of that. thank you for your help @Shubham Bajaj Might i suggest making this behaviour more apparent for the future? the documentation on this is not so clear but at the same time it seems like something that a lot of applications would want to take advantage of
currently trying to forward an assistant from a call of
GET https://api.vapi.ai/assistant/:id
as the response body, but it doesn't seem to work appropriately at the moment (i have indeed wrapped it in the
assistant
object as shown in the documentation). will continue to debug in a bit, have a meeting to attend. but this is definitely the right track.
s
can you elobrate more on this.
d
Copy code
python
def handle_assistant_request(request):
    url = "https://api.vapi.ai/assistant/<my assistant id>"

    headers = {"Authorization": "Bearer <token>"}

    assistant_response = requests.request("GET", url, headers=headers)

    response = {
        'assistant': json.loads(assistant_response.text),
    }
    return JsonResponse(response, safe=False)
I am retrieving an assistant by ID from the VAPI api, wrapping it in the
assistant
json object key (as per the specification on the document you linked me) and returning that as the response to the
assistant-request
server message. VAPI tells me it failed to get the assistant when i do this
perhaps i need to do some transformation of the response from the assistant api. I want to get things working with the result of the response from the VAPI api, and then i will go ahead and add my custom context messages and so on.
I was hoping that feeding the result from GET /assistant/:id as an answer to
assistant-request
would work out of the box, but it seems not.
willing to provide you any ids or credentials if necessary.
Copy code
python
def handle_assistant_request(request):
    url = "https://api.vapi.ai/assistant/<my assistant id>"

    headers = {"Authorization": "Bearer <token>"}

    assistant_response = requests.request("GET", url, headers=headers)

    response = {
        'assistant': json.loads(assistant_response.text),
    }
    del response['assistant']['orgId']
    del response['assistant']['id']
    del response['assistant']['createdAt']
    del response['assistant']['updatedAt']
    del response['assistant']['name']
    del response['assistant']['voice']
    del response['assistant']['endCallFunctionEnabled']
    del response['assistant']['transcriber']
    del response['assistant']['clientMessages']
    del response['assistant']['serverMessages']
    del response['assistant']['dialKeypadFunctionEnabled']
    del response['assistant']['serverUrl']
    del response['assistant']['maxDurationSeconds']
    del response['assistant']['backgroundSound']
    del response['assistant']['isServerUrlSecretSet']
    return JsonResponse(response, safe=False)
I went through and manually deleted every entry. other than what was in that sample response in the docs. seems to kind of work now, it at least accepts the assistant response. I am going to comment one by one and see which of these are causing the failure
definitely feels like something weird is going on here. i want to initiate my assistant with the exact settings that i have configured in the vapi UI, except i want to inject extra messages as context into the messages array of the model.
@Shubham Bajaj this is minimum code I was able to utilize to get things working. Things, are working, but obviously not an ideal developer experience πŸ™‚
Copy code
python
def handle_assistant_request(request):
    url = "https://api.vapi.ai/assistant/:id"

    headers = {"Authorization": "Bearer <token>"}

    assistant_response = requests.request("GET", url, headers=headers)

    response = {
        'assistant': json.loads(assistant_response.text),
    }

    # must delete
    del response['assistant']['orgId']
    del response['assistant']['id']

    del response['assistant']['isServerUrlSecretSet']

    del response['assistant']['createdAt']
    del response['assistant']['updatedAt']
    del response['assistant']['name']
    return JsonResponse(response, safe=False)
by deleting those keys from the response of the VAPI endpoint, things seem to work as expected.
if you include 1 or more of those fields, the api says "Could not get assistant".
s
your doing wrong you just need to remove the mutation fields.
d
@Shubham Bajaj what do you consider a mutation field? I found that if i remove any of those
del
commands in my previous code, that vapi will not understand my response. it seems like the bare minimum to get things to work.
as of right now i have things working, but if there is a way to simplify, i am all ears πŸ™‚
s
it's alrready simple you just have to return assistant,/
d
that is the problem though - returning the assistant directly without deleting those keys prevents vapi from operating properly. if any of the keys that i am deleting are present, vapi will say "Could not get assistant". I think it is a bug in the platform?
s
it's what your doing wrong try returning assistant id, for transient assistant how can pass mutation keys or reserved keys or properties.
a
The team already gave you an answer with server side events but wanted to give some input just in case. I found that the vapi dashboard forces you to have deafult first message which it will say before running it does anything else like runnning a custom tool. I found that the vapi api actually alllows you to disable this which allows me to call the function as soon as the call starts
d
@Alozie | AI Voice Developer Thanks for reaching out, I actually solved this issue by injecting the system prompt with extra information during the assistant-request webhook, circumventing the need for the tool call at the start of the call altogether. There is an added benefit that changing the first message mode to β€œassistant speaks first with model generated message” allows it to utilize the extra context in a personalized greeting for the caller
s
closing this ticket for now, continue conversation over general channels.
24 Views