OpenAI's structured outputs
# support
c
We've found some of the function or tool calling to be inconsistently invoked (the AI won't call the tool) or for the outputs to be inconsistently formatted. We were wondering when or how OpenAI's structured outputs could be used with the Vapi tools.
v
@CAKO OpenAI's structured outputs can be effectively used with Vapi tools by ensuring that the responses from OpenAI are formatted according to Vapi's expected structure. This typically involves converting the text into a JSON object that adheres to the schema required by Vapi. To ensure consistent invocation and formatting: 1. **Invocation Consistency**: Make sure that the conditions for calling a function or tool are clearly defined and that the AI is programmed to recognize these conditions accurately. This may involve checking for specific keywords, phrases, or context that indicate a function or tool should be used. 2. **Output Formatting**: Once you receive a response from OpenAI, you'll need to format it into the structure that Vapi expects. This usually means creating a JSON object with specific fields that Vapi uses to process and display the response. 3. **Testing and Debugging**: Regularly test the system to identify any inconsistencies or errors in the invocation and formatting processes. Use debugging tools to trace and fix issues. 4. **Documentation and Guidelines**: Maintain clear documentation and guidelines for developers on how to use OpenAI's outputs with Vapi tools, including examples of properly formatted JSON objects. 5. **Error Handling**: Implement robust error handling to catch any issues with invocation or formatting and provide meaningful feedback to developers or users to correct the input. If you have specific examples of the inconsistencies you're facing, I can provide more targeted advice. Otherwise, if you need further assistance with integrating OpenAI's structured outputs with Vapi, please ask support for more info on this.
You can continue this conversation by mentioning me in the message.
Got it. Tagging @User for additional help.
c
Click on OpenAI structured output docs.
c
1) Our Vapi Gpt-4o cluster has us using a 05 (May) version according to our logs. We need to be using a more recent version to get the support. > Supported models > Structured Outputs are available in our latest large language models, starting with GPT-4o: > gpt-4o-mini-2024-07-18 and later > gpt-4o-2024-08-06 and later 2) GPT needs an enabling script with "response_format" do we put this into the System Prompt or do we put this into the tool description??? Neither one seems like it would work as intended. > response_format: { type: "json_schema", json_schema: {"strict": true, "schema": ...} } https://platform.openai.com/docs/guides/structured-outputs/introduction
c
Would have to confirm that from the team.
s
can you share the call id
c
Thank you so much Here's ids where it Didn't call the tool. 5f4b6684-1601-4c84-a35f-2b67960ce945 5f7bfcf0-91df-4105-9317-a12cbb76a85e c7bf4066-2c5a-4342-b68c-b1620e1d63f5 183c6896-687b-4063-883a-dca663953bc2 dfd08170-3921-4973-b1dd-a8b4f42ed7bf Here's one where it worked, for reference. 2fa898a5-6ce7-4f86-a8c5-191b69b3f412
s
check vapi prompting guide and it will be fixed.
your not calling
create_request
tool correctly.
c
Can you please link?
I honestly doubt this will help.
We follow something similar enough already and have added double and triple checks because it wasn't reliable enough.
s
c
I don't think we're doing anything different from this.
We tried it and it didn't work. b05b568b-f5d2-403d-9761-b912d94aaa7a
s
can you exactly let me know which tools or functions are called inconsistently?
c
It's the create_request tool we've made. 99c7d95b-c884-499d-adab-7edab52245fa
s
your refering
create_request
tool 45 times. so, it's supposed to be happening isn't?
can you try refactoring your prompt.
c
When we referenced it once, it was only calling 1/10 times. Now it's calling the tool ~9/10 times.
We've been refactoring the prompt solely trying to optimize for the
Copy code
create_request
tool over the past weeks.
s
i still suggest try refactoring if still occurs then we will do it over the call.
c
Its still not resolved btw.
s
@CAKO create a new support ticket add me there i will help you fix your prompt.
3 Views