A voice AI campaign call lasts somewhere between 45 and 120 seconds. In that window, four distinct AI systems run in tight coordination, on top of two layers of telecom infrastructure, in two or three Indian languages, while staying inside a budget of roughly ₹1 per call.
This guide is the architecture under the hood — what each component does, where the latency hides, what the failure modes are, and what choices a campaign actually has to make.
If you are pitching, evaluating or building one of these systems for the 2027 state elections or the 2029 Lok Sabha, this is the layer below the marketing slides.
The four-component stack
Every modern voice AI agent — campaign, customer-service, healthcare or otherwise — is the same four components plugged together. The art is in latency, language tuning and orchestration.
1. Speech-to-text (STT)
The voter speaks. The audio stream — typically 16kHz PCM over a WebRTC or SIP channel — flows into a speech-recognition model that produces a stream of partial transcripts in real time. The output looks like this:
t=0.4s "मेरे"
t=0.8s "मेरे बच्चे"
t=1.4s "मेरे बच्चे को बुखार है"
t=1.6s [end of utterance]
For Indian elections, the STT must:
- Handle Indian-accented Hindi (very different from Bollywood Hindi).
- Tolerate code-switching mid-sentence.
- Detect end-of-speech (VAD) without cutting off slow speakers.
- Run fast enough that downstream LLM call is triggered within ~300ms of the voter pausing.
The main commercial options in 2026 are OpenAI's Whisper family, Deepgram Nova, Google Cloud Speech-to-Text, and the Indian-built Bhashini ASR stack from MeitY. Each has different price-accuracy-latency tradeoffs.
2. LLM (the brain)
The transcript arrives at a large language model along with a system prompt (the agent's instructions, persona and HARD STOP rules), the conversation history, and optionally a knowledge base retrieved via RAG.
The system prompt is where 80% of the campaign logic lives:
- Who is the agent? ("मैं सिया हूँ, AI सहायक, UP Vidhan Sabha campaign की ओर से...")
- What is the goal of this call? (Get manifesto feedback / GOTV reminder / grievance capture)
- What are the HARD STOP rules? (Goodbye → end. Angry voter → apologise once, end. Silent for 2 turns → end. Cost cap at 120s → wrap up.)
- What is allowed and what is not? (Never claim to be human. Never make promises. Never criticise opponents by name.)
The output is a short reply (one or two sentences for natural conversation). The latency budget for the LLM is 150–500ms for fast models like Gemini 2.5 Flash, Claude Haiku 4.5, GPT-4.1 mini, or Qwen 3 30B A3B (MoE — only 3B active parameters, so latency similar to a 4B dense model).
3. Text-to-speech (TTS)
The LLM's reply text goes to a TTS model that streams audio chunks back to the voter — typically as 24kHz PCM that gets transcoded down to 8kHz telephony format. Modern TTS produces the first audio chunk within ~150–250ms of receiving the text, and streams the rest as it generates.
For Indian elections, the TTS must:
- Sound native in the target language and dialect (not generic Hindi spoken with an English accent).
- Support voice cloning if the campaign wants the candidate's voice (within ECI rules — disclosed clearly as AI).
- Hit first-audio-chunk under 250ms so the conversation feels live.
- Cost under ~₹0.30 per minute of generated speech.
ElevenLabs Turbo v2.5, Cartesia Sonic 3, Google's TTS, and the Bhashini TTS models are the live commercial options. Most Indian-language voice agents in production today use ElevenLabs multilingual or Cartesia Sonic for the speed, with the Bhashini stack as a localisation/sovereignty fallback.
4. Telephony
The voice agent is useless without a phone line. The telephony layer connects the AI stack to India's mobile network via SIP trunks or WebRTC. For a campaign call:
- Outbound: dial the voter's number from a TRAI-DLT registered sender. Most campaigns rent SIP trunks from Indian providers (Knowlarity, Exotel, Ozonetel, MyOperator) or use international platforms (Twilio India, Vonage) for cross-border routing.
- Inbound: a published number (often 1800-series) routes to the agent. Voter calls in, the agent answers within 1–2 rings.
Telephony latency adds 100–250ms one-way for in-country routing and up to 500ms one-way for international round-trips. Campaigns that run their agent stack outside India and route Indian calls through US datacenters routinely deliver 1.5–2.5s end-to-end latency — voters perceive this as "the line is laggy" and hang up.
For Indian elections, the entire stack must run in Indian or APAC datacenters. Period.
Putting it together: end-to-end latency budget
For a conversational call to feel natural, the time from voter pausing → AI starting to speak must stay under 1 second. Here is the budget:
| Stage | Latency |
|---|---|
| End-of-speech detection (VAD) | 100–200ms |
| STT final transcript | 100–250ms |
| LLM time-to-first-token | 150–500ms |
| TTS time-to-first-audio | 150–250ms |
| Network (in-country) | 100–200ms |
| Total | 600–1400ms |
Anything that pushes the total above 1500ms triggers the "is this a real conversation?" perception in the voter. Pushing it under 800ms feels remarkably natural. A 350ms TTFW agent feels indistinguishable from a slightly distant human caller.
The cheap-but-painful trap: serving the LLM, TTS and STT from three different providers in three different regions. Each cross-region hop adds 50–150ms, and they stack.
Multilingual handling: what actually works
Indian elections need true multilingual, not bolted-on multilingual.
Same model, all languages. The right architecture is one LLM that handles Hindi, English, Tamil, Bengali, Marathi, etc. in a single inference pass. A voter who says "मेरा driving licence renewal pending है" is sending one sentence — not a Hindi clause and an English clause. The LLM must process it as a whole.
Dialect detection at first turn. A well-designed agent listens to the first ~5 seconds of the voter's first response, classifies the dialect (Standard Hindi vs Marwari vs Bhojpuri vs Awadhi), and switches its own response register accordingly. This is the single biggest "wow" moment in user testing — the voter literally pauses and says "अरे, ये तो मेरी ही भाषा में बोल रहा है".
TTS voice match. The same dialect must come out of the TTS. A Marwari conversation answered by a Standard Hindi voice feels insulting. This requires either dialect-tuned TTS voices (rare) or multiple voice IDs the agent switches between.
Code-switch graceful degradation. If the voter uses an English technical term ("hospitalisation", "licence", "scheme"), the agent should keep those terms in English, not awkwardly translate them. The system prompt explicitly tells the model: "तकनीकी शब्द English में रखो।"
What the agent should and should not do
Two categories of behaviour need explicit specification in the system prompt — silence here is what produces the embarrassing campaign-cycle headlines.
Must do
- Self-identify as AI on call open. ECI requirement. No exceptions.
- Acknowledge the voter's question before answering. Even one syllable ("हाँजी") makes the interaction feel like a conversation, not a script.
- Cite source on factual claims. ("ECI के अनुसार आपका voter ID 7301 booth पर है।")
- Capture next-step intent. ("क्या मैं आपके लिए आपके booth का नंबर SMS कर दूँ?")
- Respect HARD STOP rules. Goodbye → close. Anger → apologise once, end. Silence × 2 turns → end. Time cap → wrap up.
Must not do
- Never claim to be the candidate. Always "AI assistant on behalf of [candidate]".
- Never criticise opponents by name. Triggers ECI Model Code of Conduct issues.
- Never collect Aadhaar, bank, UPI or OTP details. Even if the voter offers.
- Never make promises in the candidate's name ("मैं आपका काम करवा दूँगा"). Capture the request, route it, end.
- Never use English-Hindi machine translation as a fallback. If the agent can't speak the voter's language, route to a human.
Observability and audit
A campaign running 50 lakh AI conversations needs a way to know what happened. The standard observability surface includes:
- Per-call record: voter phone (hashed), start/end time, language detected, full transcript, sentiment score, top three issues mentioned, intent class (supportive / undecided / negative / neutral), hand-off flag.
- Daily dashboard: aggregate sentiment by booth, top issues trending up/down, completion rate, average duration, cost.
- Audit log: every system prompt change, every model swap, every release deployed — with timestamps.
- Voter-erasure pipeline: a DPDP-compliant pipeline that removes all records for a voter who requests it within 7 days.
The audit log is what saves the campaign when a journalist asks "what was the agent saying on March 14?". You can replay the exact prompt, the exact model and the exact behaviour at that point in time.
Build vs buy
Most state-scale campaigns will not build their own voice AI stack. The list of things you need to get right is long:
- Multi-region GPU inference with sub-second failover
- 22-language STT/TTS tuning
- TRAI-DLT integration and template registration
- ECI-compliant audit pipeline
- DPDP-compliant data residency and erasure
- Voice cloning workflow with consent capture
- Real-time observability across crore-scale traffic
Specialist platforms (AiSewak, Voxdonna, others) ship this as a service. The campaign provides the candidate's voice samples (one hour), the knowledge base (manifesto, FAQs, local issues) and the voter list, and the agent is in production in 14–21 days.
For Lok Sabha-scale rollouts, an in-house team makes sense — but only if the team has shipped a production voice agent before. The learning curve in the first 90 days is brutal.
Where to go next
- AI vs Traditional Election Outreach — head-to-head ROI numbers
- Voice AI vs SMS vs WhatsApp — channel comparison
- Hindi-First AI Voice Agents — what Indian-language fluency really means
- The Two-Way AI Voter Engagement Engine — the full technical playbook
The technical stack matters because the political message rides on top of it. Mishandle the latency, the dialect or the audit trail and the most carefully written manifesto reads as spam.