For me it helped to enable VAPI voice detection and then provide it with a tool

flag_voicemail

which can be used as a fallback whenever VAPI's detection methods fail but the model itself realise what's going on.