Multilingual AI voice agents for India
Supports 22 languages is the least informative claim in this market. What decides whether a call lands is whether the agent speaks the register the caller actually uses, understands them when they switch mid-sentence, and pronounces the scheme name the way they have heard it. Those are three separate engineering problems and none of them is a language count.
- 22 scheduled
- Plus the regional registers that decide comprehension
- Mid-call switch
- The caller changes language; the agent follows
- Per-language QA
- Acceptance tested in each language, not in English
- Bhashini-capable
- MeitY stack or commercial models, chosen per language
A multilingual AI voice agent is one that can hold a real conversation in more than one Indian language over an ordinary phone line — recognising what the caller said, answering from the same knowledge in their language, and switching when they switch. For any Indian public service or national programme that is not a feature, it is the precondition: a line that works only in Hindi and English is a line that does not reach a large part of the country it was built for.
The hard part is not adding languages to a list. It is that Indian speech on a phone call is messier than any language label suggests. A caller in eastern Uttar Pradesh is speaking Bhojpuri-inflected Hindi, not textbook Hindi. A caller in Vidarbha is speaking Varhadi, not the Marathi of a Mumbai news bulletin. Most callers move between their language and English inside a single sentence, and almost all of them say numbers, dates and scheme names in a way that general speech models handle badly.
AiSewak builds and operates these agents across Hindi, English, Marathi, Gujarati, Bengali, Tamil, Telugu, Kannada, Malayalam, Odia, Punjabi and the regional variants beneath them, on a mix of Indic-first commercial models and the MeitY Bhashini stack. Which of those is used for a given language is an engineering decision made per language, not a platform decision made once.
Four things that break, in the order they break
Almost every disappointing multilingual pilot in India fails at one of these, and it is rarely the one the team expected.
Register, not language
A speaker of Awadhi, Magahi, Marwari, Mewari, Varhadi, Malwai or Kosli hears the standard form of their own state language as the voice of an outsider. They do not report this — they shorten the call or hang up. Completion rate falls and nobody can say why, because the language label on the dashboard says Hindi and the agent was speaking Hindi.
Code-switching mid-sentence
Application, status, subsidy, KYC, last date — English nouns arrive inside an otherwise Marathi or Bengali sentence, constantly. A recogniser locked to one language transcribes them as nonsense words and the agent answers a question nobody asked. The recogniser has to accept the mixture rather than pick a side.
Numbers, dates and identifiers
Account numbers, consumer numbers, application IDs, Aadhaar digits and dates spoken in Indian number habits are where phone-call recognition actually fails. Anything the citizen will act on has to be read back digit by digit and confirmed. This costs call time, so decide which identifiers you genuinely need before the prompt is written.
Proper nouns and scheme names
A synthesised voice that mispronounces the scheme, the district or the officer's designation tells the caller immediately that it does not know the domain. Scheme names, place names and department titles need an explicit pronunciation list per language, maintained as the programme adds new ones.
One knowledge base, eleven voices
The answer a Tamil caller gets and the answer a Bengali caller gets must be the same answer. Translating the knowledge base per language is how they drift apart, and the drift is invisible until an audit. Keep one source of truth and render it, rather than maintaining eleven copies.
Testing in English
The most common process failure. A team builds in English, tests in English, adds languages last and validates them by asking a colleague whether it sounds fine. Each language needs its own acceptance set, scored by someone who speaks that register, before that language goes live.
Bhashini or a commercial model — decided per language, not once
The MeitY Bhashini stack, built under the National Language Translation Mission, gives Indian public programmes a sovereign, government-backed language layer across the scheduled languages. For departments with data-residency requirements or a policy preference for public digital infrastructure, that matters, and for several languages it is the strongest option available.
Commercial Indic-first models are ahead in other places — usually latency, conversational naturalness, and handling of code-switched speech. Latency is not a cosmetic property in a voice call: a pause long enough to read as a dropped line ends the conversation regardless of how correct the answer would have been.
The useful posture is therefore neither loyalty nor avoidance. Benchmark per language against your own recordings — not a vendor's demo set — measuring recognition on your actual call audio, latency at your concurrency, and naturalness judged by speakers of that register. Then route each language to whichever stack wins it, behind one agent that the citizen never sees the seams of. Departments that mandate the sovereign stack throughout can have that; it should be a stated procurement decision with its consequences understood, not a default arrived at by accident.
How a multilingual deployment is actually built
- 1
Map languages to population, not to the state list
Which languages and registers do the people who will call this line actually speak, by district? A state-level language list systematically misses the border districts and the migrant populations, which are often precisely the citizens a programme is trying to reach.
- 2
Collect real call audio before choosing anything
A few hundred recordings from the actual line, in the actual conditions — weak network, background noise, a phone held wrong. Model selection made on clean studio audio does not survive contact with a rural mobile call.
- 3
Build one knowledge base, render many
Facts live once, in one place, traceable to the department's own circular. Each language renders from that source. No per-language document sets, because they diverge and nobody notices for months.
- 4
Pronunciation and terminology lists
Scheme names, districts, departments, designations, common abbreviations — written out per language and maintained as the programme grows. This is unglamorous and it is the difference between an agent that sounds informed and one that does not.
- 5
Per-language acceptance sets
A fixed set of real caller utterances per language with expected outcomes, scored by a speaker of that register. A language ships when it passes its own set — never on the strength of the English one passing.
- 6
Watch completion rate by language after launch
The number that exposes a register problem nobody reported. If one language's completion rate sits well below the others, the model is not wrong — the register is, and that is fixable.
What multilingual actually buys a public programme
Reach into the households a portal never reached. A phone call needs no smartphone, no data pack, no app store and no ability to read a form — and in the caller's own language it needs no intermediary either. For welfare delivery, that removes the middle layer between an eligible household and knowing that it is eligible.
Information that survives the last mile intact. A scheme explained in the citizen's register is a scheme they can repeat correctly to a neighbour. A scheme explained in a language they half-follow becomes rumour, and correcting rumour costs a department far more than explaining the scheme did.
And a straightforward equity argument that is easy to state and hard to argue with: a service available in eleven languages is available to people a two-language service excludes. In programmes with statutory coverage obligations that is not a nice-to-have, and in any programme it is the difference between announced coverage and actual coverage.
Privacy, residency and the honest limits
Call audio is personal data, and speech is biometric-adjacent in a way transcripts are not. Deployments run under a named purpose with consent captured in-call, a defined retention window and an executed deletion path under the DPDP Act 2023. Indian data residency is available, and departments that cannot send audio outside their own network can run on-premise or private-cloud — with the trade-off stated plainly, since isolated deployment constrains which models are available and therefore which languages and registers are well covered.
The limits worth stating before a procurement: not every register is equally well served, and the gap is largest exactly where the need is often greatest — smaller languages, strong regional accents, elderly speakers on poor lines. Translation of legal or scheme text between Indian languages is not reliable enough to leave unreviewed, so anything a citizen will act on is rendered from department-approved wording rather than machine-translated at runtime. And an agent uncertain of what it heard should ask again or transfer, never guess — a wrong subsidy amount confidently delivered in a citizen's mother tongue is more damaging than no answer at all.
Call a live agent before you decide
These are running agents, not recordings. Open one, press call and speak to it in Hindi or English — the same stack that runs the deployments described above.
Yojana Didi — Hindi
Hindi scheme eligibility with the register a north-Indian caller actually uses.
Open the demo →Yojana Tai — Marathi
Marathi, tuned to Maharashtra's schemes and to the way callers there phrase a question.
Open the demo →Yojana Baisa — Rajasthan
Marwari and Mewari handling — the clearest demonstration of register versus language.
Open the demo →Van Dhan — Santhali
A tribal-livelihood line in a language most voice stacks do not support at all.
Open the demo →Kumbh Sahayak — mass-event line
A visitor helpline built for a crowd that arrives speaking every language in the country at once.
Open the demo →Siya — open-ended questions
Unbounded Hindi questions answered over a retrieval-backed knowledge base.
Open the demo →Go deeper
Multilingual voice AI and the Bhashini advantage
What the MeitY stack provides, which languages it covers and how it compares in practice.
The Indian language stack, end to end
How recognition, translation and synthesis fit together for an Indian deployment.
Why Hindi-first is not the same as multilingual
The register problem set out with examples, and what it costs in completion rate.
AI citizen services in India
Where multilingual voice fits in the wider citizen-service picture.
Related pages: The main government page · Automating a government helpline · Grievance intake and closure · State-by-state language pages · Comparing Indian vendors
Frequently asked questions
What is a multilingual AI voice agent?
A conversational AI that handles phone calls in more than one language — recognising the caller's speech, answering from the same knowledge base in their language, and switching when the caller switches. For India that means the 22 scheduled languages and, more importantly, the regional registers beneath them: a caller speaking Bhojpuri-inflected Hindi or Varhadi Marathi needs to be met in that register, not in the textbook form of the state language.
How many Indian languages can it actually handle?
Hindi and English plus the scheduled languages — Marathi, Gujarati, Bengali, Tamil, Telugu, Kannada, Malayalam, Odia, Punjabi, Assamese and the rest — and the regional variants that decide whether a call lands, including Bhojpuri, Awadhi, Magahi, Marwari, Mewari, Varhadi, Malwai, Kosli and Santhali. The honest qualifier is that coverage quality is not uniform: smaller languages and strong regional accents are served less well than Hindi, and that gap should be measured on your own call audio before a language goes live rather than assumed from a list.
Does it use Bhashini or a commercial model?
Either, decided per language rather than once for the platform. The MeitY Bhashini stack matters for departments with data-residency requirements or a policy preference for public digital infrastructure, and for several languages it is the strongest option. Commercial Indic-first models are often ahead on latency and on code-switched speech, and latency is not cosmetic in a voice call. Benchmark both on your own recordings and route each language to whichever wins it, behind a single agent the citizen sees no seams in.
Can it handle a caller mixing English into Hindi or Marathi?
It has to, because that is how most Indian callers speak. Words like application, status, subsidy, KYC and last date arrive inside an otherwise Indian-language sentence constantly. A recogniser locked to a single language transcribes them as nonsense and the agent answers something the caller did not ask. Handling the mixture rather than choosing a side is a deliberate configuration, and it is one of the first things to test on real call audio.
How do you stop the agent mispronouncing scheme and place names?
An explicit pronunciation and terminology list per language, covering scheme names, districts, departments, designations and common abbreviations, maintained as the programme adds new ones. It is unglamorous work and it is decisive — a synthesised voice that mispronounces the scheme tells the caller in the first sentence that it does not know the subject, whatever the rest of the answer contains.
How do you know a language is working before going live?
Each language gets its own acceptance set: a fixed collection of real caller utterances with expected outcomes, scored by someone who speaks that register. A language ships when it passes its own set, never because the English set passed. After launch, completion rate by language is the number to watch — one language sitting well below the others is almost always a register mismatch rather than a model failure, and it is fixable.
Is an AI voice agent legal for outreach in India?
Yes, within a well-defined set of rules. The agent must disclose that it is an AI at the start of the call (ECI's 2024 advisory on synthetic media), outbound calling must respect TRAI TCCCPR 2018 and DLT registration, personal data must be handled under the DPDP Act 2023, and IT Rules 2021 govern the content itself. Election deployments additionally require registration with the ECI or the state CEO as a political advertiser. AiSewak ships these controls switched on by default rather than as an add-on.
Can this run inside our own infrastructure?
On-premise and private-cloud deployments are possible, including fully isolated installations for bodies that cannot send audio outside their own network. State the trade-off in the procurement rather than discovering it at go-live: isolated deployment constrains which models can be used, and that constraint lands hardest on exactly the smaller languages and regional registers that a multilingual programme exists to serve.
Test it on your own call recordings, in your own registers
Send a few hundred real recordings from the line you run — poor network, background noise and all. We will report recognition and comprehension per language and per register, including the ones where the answer is that we are not good enough yet.
Every AiSewak agent identifies itself as an AI at the start of the call, never asks for an OTP or a payment, and honours DND. Election deployments require ECI / state CEO registration as a political advertiser.