I found that the

smartEndpointing

setting was enabled on one of my assistants as well, when it appeared to be turned off in the UI. In case anyone is experiencing extra latency/delays, make sure to remove that setting using the api:

'startSpeakingPlan': {
        'transcriptionEndpointingPlan': {
            'onNoPunctuationSeconds': 1,
        },
        'waitSeconds': 0.3,
        'smartEndpointingEnabled': 'livekit',
    },