I have built a call center application with africa...
# ask-air
refined-football-49080
12/31/2025, 4:53 PMI have built a call center application with africastalking (platform for the call number ) spitch ai and gpt model (brain). During the call testing, the tts only works once and the caller’s voice is never picked for the next response. Please would livekit help with a two way communication between the caller and the AI?
t
tall-airline-16756
12/31/2025, 4:53 PMThanks for reaching out, ntongha. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
12/31/2025, 4:53 PMThanks — I’m glad you described the setup and the problem. I’m responding to your question about enabling reliable two-way audio between the caller (Africa’s Talking) and your AI (Spitch + GPT), and the issue where TTS plays once and then the caller’s voice isn’t captured.
Good news: LiveKit can provide a full bidirectional voice loop and solves the common causes of the behaviour you described. What LiveKit’s Agents framework offers that will help you:
• Persistent audio streaming so the caller audio and agent audio remain live in the same room.
• Turn detection / VAD so the system knows when the caller finished speaking and when the agent should respond (prevents TTS from blocking inbound audio).
• A media pipeline that connects STT → LLM → TTS in a loop so responses and listening alternate cleanly.
• Telephony/SIP bridging to connect Africa’s Talking (or any SIP/PSTN provider) into a LiveKit room.
To help you quickly, I need two clarifying details about your current integration:
1. How are you bridging Africa’s Talking into your app right now — are you using SIP trunking into LiveKit, an outbound-caller example, or a custom connector?
2. Where is turn control happening today — does Spitch handle VAD/turn detection, or are you relying on the TTS service or your application to mute/unmute streams?
Based on your answers I can give targeted next steps (example fixes and configuration checks), for instance:
• If TTS is played into the room without ducking/unmuting the caller track, you’ll need VAD or a turn-detector so the agent stops injecting audio and re-enables the caller mic.
• If your SIP bridge creates a new stream per turn, we’ll check session/track management so the inbound audio remains subscribed after TTS playback.
• If Spitch doesn’t emit continuous streaming or VAD events, we can show how LiveKit Agents can run STT + VAD and call your LLM/TTS only when the turn ends.
Tell me which bridging method you use and where turn detection lives, and I’ll provide concrete configuration steps or code snippets to fix the one-shot TTS and restore continuous two-way conversation.
Sources: Agents telephony integration | LiveKit docs | sip | livekit/livekit_composite | DeepWiki | https://blog.livekit.io/introducing-livekit-phone-numbers-zero-to-ringing-in-60-seconds/#/portal| | LiveKit turn detector plugin | LiveKit docs
r
refined-football-49080
12/31/2025, 6:02 PMBridging Africa's Talking to LiveKit:
The integration uses SIP trunking to connect Africa's Talking to LiveKit. Calls from Africa's Talking are routed to a LiveKit SIP URI, which creates a room for each call. The AI agent then joins this room to handle the conversation.
Turn Control Implementation:
VAD (Voice Activity Detection): Handled by the
AudioProcessor
class using webrtcvad for detecting speech.
Turn Management: Managed in the application layer with a state machine (
TurnState
).
TTS Integration: Spitch is used for text-to-speech, but the application controls when to play TTS and when to listen to the user.
Key Components:
AudioProcessor: Manages VAD, audio levels, and turn states.
AIAgent: Handles the conversation flow and integrates with Spitch TTS and GPT.
Turn Management: Uses a state machine (USER_SPEAKING, AI_SPEAKING, SILENCE) to manage conversation turns.
Current Flow:
User Speaks: VAD detects speech and transitions to USER_SPEAKING.
Processing: The application processes the audio (currently simulated; ASR would be integrated here).
t
tall-airline-16756
12/31/2025, 6:02 PMThanks — this is very helpful detail. I’m responding to your description of SIP bridging, VAD via AudioProcessor, and your TurnState state machine.
Quick assessment: the symptoms (TTS plays once, then caller audio isn’t picked up) usually come from one or more of these issues:
• The AI/TTS playback isn’t releasing the room audio or is muting/unsubscribing the inbound track and not restoring it.
• Turn-state transitions never return to USER_SPEAKING because VAD stops firing while TTS is active or the state machine is blocked during TTS playback.
• SIP/track lifecycle: the SIP bridge or LiveKit participant for the PSTN caller is being replaced or its track is being detached after the first turn.
I can help with concrete checks and fixes. To narrow to the right action, please confirm two quick things:
1. During TTS playback, does your app mute the PSTN participant’s publish track, or does it keep the caller publishing while you play TTS into the room?
2. After TTS finishes, do you see any VAD events (AudioProcessor) at all (even silence detection), or do VAD callbacks stop until you restart the call?
If you want immediate steps to try now, try these in order:
1. Ensure TTS is injected as a separate track/participant (agent) rather than muting the caller’s publish track. Playing TTS from the agent participant keeps the caller track live and VAD able to resume.
2. Add explicit short delay + VAD warmup after TTS finishes before declaring USER_SPEAKING — e.g., 200–500 ms to allow the PSTN audio path to stabilize and VAD to re-detect speech.
3. Make TTS playback non-blocking for the state machine: let the agent enter AI_SPEAKING but keep the audio pipeline accepting inbound frames; only transition to waiting-for-user once VAD signals user activity or silence timeout.
4. Watch participant/track events from LiveKit: confirm the PSTN participant remains connected and its audio track ID does not change after the first turn. If the track is replaced, re-subscribe logic may be needed.
5. Log RTP/SSRC or LiveKit track IDs during the session to confirm continuity across turns—if SSRC/track changes, map the new track to the existing participant and ensure your AudioProcessor is attached to the current stream.
6. If Spitch injects audio with different codec/clocking, ensure resampling or proper media format is used so the PSTN inbound path isn’t interfered with.
If you confirm answers to (1) and (2), I’ll provide specific code-level suggestions for where to change your state transitions and a small snippet showing a non-blocking TTS playback pattern and VAD warmup you can drop into AudioProcessor/TurnState handling.