We've found some of the function or tool calling to be inconsistently invoked (the AI won't call the tool) or for the outputs to be inconsistently formatted. We were wondering when or how OpenAI's structured outputs could be used with the Vapi tools.

@CAKO OpenAI's structured outputs can be effectively used with Vapi tools by ensuring that the responses from OpenAI are formatted according to Vapi's expected structure. This typically involves converting the text into a JSON object that adheres to the schema required by Vapi. To ensure consistent invocation and formatting: 1. **Invocation Consistency**: Make sure that the conditions for calling a function or tool are clearly defined and that the AI is programmed to recognize these conditions accurately. This may involve checking for specific keywords, phrases, or context that indicate a function or tool should be used. 2. **Output Formatting**: Once you receive a response from OpenAI, you'll need to format it into the structure that Vapi expects. This usually means creating a JSON object with specific fields that Vapi uses to process and display the response. 3. **Testing and Debugging**: Regularly test the system to identify any inconsistencies or errors in the invocation and formatting processes. Use debugging tools to trace and fix issues. 4. **Documentation and Guidelines**: Maintain clear documentation and guidelines for developers on how to use OpenAI's outputs with Vapi tools, including examples of properly formatted JSON objects. 5. **Error Handling**: Implement robust error handling to catch any issues with invocation or formatting and provide meaningful feedback to developers or users to correct the input. If you have specific examples of the inconsistencies you're facing, I can provide more targeted advice. Otherwise, if you need further assistance with integrating OpenAI's structured outputs with Vapi, please ask support for more info on this.

Click on OpenAI structured output docs.

1) Our Vapi Gpt-4o cluster has us using a 05 (May) version according to our logs. We need to be using a more recent version to get the support. > Supported models > Structured Outputs are available in our latest large language models, starting with GPT-4o: > gpt-4o-mini-2024-07-18 and later > gpt-4o-2024-08-06 and later 2) GPT needs an enabling script with "response_format" do we put this into the System Prompt or do we put this into the tool description??? Neither one seems like it would work as intended. > response_format: { type: "json_schema", json_schema: {"strict": true, "schema": ...} } https://platform.openai.com/docs/guides/structured-outputs/introduction

Would have to confirm that from the team.

can you share the call id

Thank you so much Here's ids where it Didn't call the tool. 5f4b6684-1601-4c84-a35f-2b67960ce945 5f7bfcf0-91df-4105-9317-a12cbb76a85e c7bf4066-2c5a-4342-b68c-b1620e1d63f5 183c6896-687b-4063-883a-dca663953bc2 dfd08170-3921-4973-b1dd-a8b4f42ed7bf Here's one where it worked, for reference. 2fa898a5-6ce7-4f86-a8c5-191b69b3f412

check vapi prompting guide and it will be fixed.

Can you please link?

check this for reference and let me know https://gist.github.com/bajajcodes/ff0bc6e126d4956fb422d47a397abeff

I don't think we're doing anything different from this.

can you exactly let me know which tools or functions are called inconsistently?

It's the create_request tool we've made. 99c7d95b-c884-499d-adab-7edab52245fa

your refering

create_request

tool 45 times. so, it's supposed to be happening isn't?

When we referenced it once, it was only calling 1/10 times. Now it's calling the tool ~9/10 times.

i still suggest try refactoring if still occurs then we will do it over the call.

Its still not resolved btw.

@CAKO create a new support ticket add me there i will help you fix your prompt.