have been trying to reference the repo for streaming responses https://github.com/VapiAI/server-side-example-serverless-vercel/blob/master/api/custom-llm/openai-sse.ts

'generator' object does not support the context manager protocol

Really just need a python guide or pointers as to what is the expected response format.

@darwin911 It sounds like you're looking for guidance on how to implement server-sent events (SSE) in Python to stream responses from your server. The error message you're encountering (

'generator' object does not support the context manager protocol

) suggests that there might be an issue with how you're using a generator in your Python code. Server-sent events require a specific format for the response. Each event is sent as a block of text terminated by a pair of newline characters. Here's a basic example of what the response should look like:

data: This is the first message.\n\n
data: This is the second message.\n\n

In Python, you can implement SSE by using a generator function that yields data in the correct format. Here's a simple example using Flask, a popular Python web framework:

python
from flask import Flask, Response, stream_with_context

app = Flask(__name__)

def generate_messages():
    # This is a simple generator function that yields two messages.
    yield "data: This is the first message.\n\n"
    yield "data: This is the second message.\n\n"

@app.route('/stream')
def stream():
    return Response(stream_with_context(generate_messages()), content_type='text/event-stream')

if __name__ == '__main__':
    app.run(debug=True)

In this example,

generate_messages

is a generator function that yields strings formatted as SSE messages. The

stream

route returns a

Response

object with the

stream_with_context

wrapper, which takes the generator as an argument. The

content_type

is set to

text/event-stream

, which is required for SSE. If you're not using Flask, the implementation details may vary, but the core idea remains the same: you need to stream data in the SSE format using a generator. If you need further assistance or have specific questions about your implementation, please provide more details or code snippets, and I'll be happy to help!

Yeah, ended up changing from Flask to FastAPI since it supports async out of the box. That seemed to resolve issues. Maybe I just did not know how to setup the streaming response w/ Flask. And thank you I did use this as a reference 👍