Vapi does give you flexibility. I thought they allowed custom transcribers but if not I'm sure they will eventually allow you bring your own transcriber.
Also have the custom LLM feature which give you lot of flexibility and what you can do with vapi. Seen people add cool features to thier phone agent with that