Helpline Pilot Acceptance Criteria for One Department

Executive Summary

A helpline pilot ends in a meeting where someone decides whether to extend it. If the pass rule is written after the data arrives, that meeting argues about the rule instead of the result. The department that owns the helpline, through its nodal officer and procurement committee, should file the criteria with the work order, before the first live call.

Executive Callout Sort every pilot measure into one of four classes: stop-the-line events (one occurrence suspends the pilot), must-pass counts (zero failures in a sample), rate gates (judged on the lower bound of a confidence interval, not the headline percentage) and report-only figures. Measure the current line for four weeks first. Let the department, not the vendor, score the calls. The thresholds here are a method, not benchmarks: no published Indian field figure sets a defensible pass mark for a government helpline.

Introduction: A Pilot Without a Pass Rule Is a Demonstration

Today a state department that runs a seated helpline approves a pilot by work order, hears a progress report from the vendor and decides at the end. The vendor picks the calls shown, and the reported percentage has no stated sample size or baseline.

AiSewak's 30-day roadmap sets out the sequence of a pilot. This guide covers the document that makes the roadmap's gates enforceable: the acceptance record, signed before launch. It is design guidance for inbound and overflow pilots, not a report from a deployment.

Step 1: Measure the Current Line First

A pass mark needs something to be compared against. Before the vendor switches anything on, pull four weeks from the telephony or ACD logs:

Answer rate and abandonment by hour, with the busiest hour reported on its own. A monthly average hides the afternoon when the line fails.
Calls that never reached a seat: busy, unanswered, or after hours.
Average handle time by call reason, and the top call reasons by volume.
Repeat calls from the same number within seven days.

If the pilot routes only queue overflow and after-hours calls to the voice layer, which AiSewak's call-centre page describes as the lowest-risk start, the comparison for those calls is a busy tone or no answer, not a human. The baseline is then the unanswered share, and the question is whether an answered call with a correct reply is better than none.

The Four Classes of Gate

Class	Rule	Examples	Failure means
Stop-the-line	One occurrence suspends the pilot	The agent asks for an OTP, PIN or payment; denies being an AI; discloses another citizen's case; writes to a field outside its granted tier; says "registered" without a portal reference	Pause, investigate, fix, re-test that gate
Must-pass	Zero failures in a sample of stated size	No critical failure in 150 scored calls per language	The language or call type does not launch
Rate gate	Lower bound of a 95% interval clears a floor	Correct answers on a known-answer set; handoffs that arrive with context	Extend with a named gap, or stop
Report-only	Reported, never gated	Cost per call, handle time, containment, satisfaction	Informs the scale decision; does not pass or fail the pilot

The first row comes from AiSewak's own standing rules for government deployments, which are to disclose that the agent is an AI and never to ask for an OTP, PIN, card number or payment. The tiers behind the third example are in the scope ladder.

Report-only is not a downgrade. Cost per call and satisfaction matter at the scale decision. They sit outside the pass rule because a first pilot gives no basis for a defensible threshold, and a number picked in advance is a guess with the authority of a contract.

Rate Gates: Judge the Lower Bound

A pilot reports "90% correct". With 100 scored calls, the department can only say the true rate lies somewhere between about 83% and 95%. The lower bound is what the department can rely on, so the gate should be written on it: "the lower bound of the 95% interval is at least 85%".

Use the Wilson interval. Brown, Cai and DasGupta's 2001 review in Statistical Science found the textbook (Wald) interval behaves erratically and recommends Wilson for small samples. The table is arithmetic, not a measurement.

Scored calls	95% margin at a 50% rate	Wilson lower bound if 90% are correct
100	±9.8 points	82.6%
200	±6.9 points	85.1%
400	±4.9 points	86.7%
1,000	±3.1 points	88.0%

Two worked cases. To show 85% or better with an observed 90%, the pilot needs about 184 scored calls. To show 80% or better with an observed 85%, it needs about 230. Observed rates below those need more. For zero-failure gates, the sample arithmetic is in the per-language acceptance guide: about 150 clean calls to say the critical-failure rate is under 2%.

Volume sets the pilot's length. Take an illustration, with an assumed 40 overflow and after-hours calls a day and a language that is 8% of them. That language yields about three calls a day, so 184 scored calls take roughly 58 days. A four-week calendar would pass or fail it on 90 calls. The department's options are to lengthen the pilot, recruit callers for a test set, or launch that language at a lower tier. Calls also cluster by caller and district, so count distinct callers, not only calls.

The Eight Gates

#	Gate	Measured how	Pass rule	Class
1	Reach	Share of baseline-unanswered calls now answered	Lower bound above the baseline answered share	Rate
2	Correct answer	Department-written known-answer set, scored by officers	Lower bound at a floor set from officers' own scores on the same set	Rate
3	Safety	Full review of flagged calls plus a sample	Zero stop-the-line events	Stop
4	Handoff	Officers scoring whether the transfer carried reason, language and details	Lower bound at a floor the cell agrees	Rate
5	Language	Per-language record	Each language passes alone	Must-pass
6	Integration scope	API logs against the granted tier	Zero calls outside scope	Must-pass
7	Repeat calls	Same-number repeats within seven days, against baseline	No worse than baseline	Rate
8	Data	Retention statement signed at pilot end	Complete and matches the contract	Must-pass

Set the floor in gate 2 from the officers' own scores on the same known-answer set. Nothing published offers a defensible target for your callers, and a human floor is the one the cell already understands.

Stages Open on Exit Tests, Not Dates

Shadow. The layer listens to live calls and drafts replies without speaking. Officers score its drafts against what the human said. Exit: gate 2 passes.
Overflow and after-hours. The layer answers calls no seat would have taken. Exit: gates 1, 3, 4 and 6, with gate 5 for each language that goes live.
One call reason at the front door. Status lookups are the usual candidate. Exit: gate 7 for that reason.

Each stage opens only when the one before has passed. Any stop-the-line event returns the pilot to the stage where the gate last passed.

Stop Rules, Incidents and Sign-Off

Keep a register: every stop-the-line event, with call ID, date, finding and fix. MeitY's India AI Governance Guidelines call for monitoring standards, audit trails and human-in-the-loop mechanisms at critical decision points, and recommend an AI incidents mechanism "to report harm and create a feedback loop". A department-level register is that idea at the scale of one helpline. The department should also hold the switch that suspends the layer, without needing the vendor.

Scoring belongs to the department's officers, on a sample the department draws, as in a normal helpline quality audit. The vendor supplies recordings and logs. The nodal officer signs gate by gate, and the head of the department signs the outcome.

The outcome is one of three: pass, extend once with the failed gate named and its re-test defined, or stop. Freeze thresholds on day one. A change after launch is dated, signed, and the results reported under both the old and new threshold.

Data at the End of the Pilot

The Digital Personal Data Protection Act is only partly in force. Under rule 1(4) of the DPDP Rules, 2025, rules 3 and 5 to 16 commence eighteen months after the 13 November 2025 gazette, which is 13 May 2027. Two of them pull against a tidy "all recordings deleted at day 30". Rule 6(1)(e) asks for logs and personal data to be retained for one year for detecting unauthorised access, and rule 8(3) sets a minimum of one year for personal data, traffic data and processing logs, for the purposes in the Seventh Schedule, unless other law requires longer.

Which obligation applies to a pilot recording is a question for the department's counsel. The practical rule is gate 8: the contract states what is deleted, what is kept, for how long, where, and who holds it. The vendor then signs the statement at close.

What Changes in the Helpline's Workflow

	Today	With a signed acceptance record
Who defines success	The vendor's report, after the fact	The nodal officer, before launch
Baseline	None, or a monthly average	Four weeks by hour, by reason, from ACD logs
Who scores	The vendor	Department officers, on a department-drawn sample
A safety failure	Mentioned in the progress report	Suspends the pilot; goes in the register
The end of the pilot	A meeting	Pass, extend once, or stop

Risks and Mitigation

Risk	Mitigation
Too few calls in a thin language	Set length by volume; recruit callers; launch at a lower tier
Floors set from a vendor claim	Derive them from officers' scores on the same set
Tempting to loosen a threshold mid-pilot	Freeze on day one; dated, signed changes only
Calls from one caller or district dominate	Count distinct callers; stratify the sample
A baseline taken in a quiet month	Include a peak week, or label the baseline as off-peak
Retention promised without checking the rules	Gate 8; counsel confirms before signing

Key Takeaways

Write the pass rule before launch, and file it with the work order.
Measure the current line by hour for four weeks. Without that baseline, no result can be judged.
Sort measures into stop-the-line, must-pass, rate gates and report-only.
Gate on the lower bound of the interval. About 184 scored calls are needed to show 85% from an observed 90%.
Pilot length follows the thinnest language's volume, not the calendar.
Officers score; the vendor supplies evidence; the department holds the switch.

Conclusion

A department can approve a pilot without knowing how good the technology is. It cannot judge the result without knowing, in advance, what would count as a pass. The acceptance record is two or three pages: a baseline, eight gates, a register, and a signature.

Government leaders exploring AI-powered citizen engagement can begin with a focused pilot in one department or constituency to validate impact before scaling statewide. Aisewak helps public institutions deploy multilingual Voice AI solutions designed specifically for Indian governance. For the integration and metrics decisions that come before a pilot, the AI call centre automation for government page sets them out in order.

FAQ

What are acceptance criteria for a government helpline pilot?

They are the written pass and fail rules a department signs before launch: a baseline from the current line, measures sorted by type, sample sizes, who scores the calls and what happens on failure. They stop the result being judged against whatever the vendor reports.

How many calls does a helpline pilot need to be meaningful?

It depends on the gate. To show 85% or better correct with an observed 90%, about 184 scored calls. For a zero-failure gate, about 150 clean calls to say the critical-failure rate is under 2%. These apply per language and per call type being claimed.

Why judge a pilot on the lower bound and not the average?

An average from a small sample can look strong by chance. With 100 scored calls, a 90% result is consistent with a true rate near 83%. The lower bound is the figure the department can rely on.

Who should score the calls?

The department's officers, on a sample the department draws. The vendor supplies recordings, transcripts and logs. A vendor that scores its own pilot has a conflict that no threshold can fix.

What should stop a pilot immediately?

One occurrence of a stop-the-line event: asking for an OTP, PIN or payment, denying that it is an AI, disclosing another citizen's case, writing outside the granted access, or claiming a registration the portal did not confirm. Each goes in an incident register.

Should cost per call be a pass or fail gate?

Not in a first pilot. Report it. A small pilot has overheads and volumes that do not predict scale cost, and the scale decision should be informed by it rather than bound by a figure chosen in advance.

Does the DPDP Act apply to pilot recordings?

The Act is only partly in force. The consent and penalty sections and rules 3 and 5 to 16 commence on 13 May 2027. Design to them now, and have the department's counsel confirm how they apply to a pilot.

How long should a helpline pilot run?

Long enough to reach the sample each gate needs, with a peak week included. In a thin language, that can be well beyond four weeks. No benchmark for pilot length exists for Indian government helplines, and this guide offers none.

Schema Markup Suggestions

Article: headline, description, datePublished and dateModified of 2026-10-30, publisher AiSewak.
FAQPage: the eight Q&A pairs above.
Table: the gate classes, eight gates and workflow tables, stated as design guidance and not as measurements.

Suggested External References

Brown, Cai and DasGupta, "Interval Estimation for a Binomial Proportion", Statistical Science 16(2), 2001, 101–133 — doi.org/10.1214/ss/1009213286, recommends the Wilson interval for small samples, abstract read 30 October 2026
MeitY, India AI Governance Guidelines: Enabling Safe and Trusted AI Innovation — static.pib.gov.in, audit trails, human-in-the-loop and AI incidents mechanism, read 30 October 2026
MeitY, Digital Personal Data Protection Rules, 2025 — meity.gov.in, rules 1(4), 6(1)(e) and 8(3), read 30 October 2026

A voice AI helpline pilot ends in a decision, and the pass rule is usually written after the data arrives. A guide for nodal officers: measure the current line first, sort measures into four classes, gate on the lower confidence bound, and let the department score the calls.

LinkedIn Executive Summary

Most helpline pilots are judged at the end, against whatever the vendor reports. The percentage has no sample size, the baseline is missing, and the meeting argues about what counts as a pass.

A department can fix this before launch with a short acceptance record. Measure the current line for four weeks, by hour and by call reason. Sort every measure into one of four classes: stop-the-line events, where one occurrence suspends the pilot; must-pass counts, where the sample must show zero failures; rate gates, judged on the lower bound of a confidence interval; and report-only figures such as cost per call, which inform the scale decision without passing or failing the pilot.

The arithmetic matters. To show 85% correct from an observed 90%, a pilot needs about 184 scored calls, and in a language that carries 8% of traffic that can take two months. Pilot length follows the thinnest language, not the calendar.

Officers score, on a sample the department draws. The department holds the switch, keeps an incident register, and signs the outcome: pass, extend once, or stop. This is a design guide, not a report from a deployment.

AI Search Optimization Summary

Entities: nodal officer, state IT department, ACD, MeitY, India AI Governance Guidelines, DPDP Act, DPDP Rules 2025, Wilson interval, Brown Cai DasGupta, Aisewak

Topics: government helpline pilot acceptance criteria, voice AI pilot success metrics, baseline measurement for helplines, sample size for pilot evaluation, stop conditions for AI in public services, who scores an AI pilot

Semantic keywords: acceptance record, stop-the-line event, must-pass gate, rate gate, lower confidence bound, known-answer set, overflow and after-hours pilot, incident register, frozen thresholds, extend once