Key takeaways
- Run at least one structured round of test calls before any guest hears the agent, using a fixed matrix and a scoring sheet.
- Some failures are automatic no-go results: guessing on allergies, promising an unconfirmed table, wrong hours or a missing disclosure.
- Test the staff side as hard as the caller side: every handoff and summary must reach the right person.
- Retest after every fix and after every menu, hours or policy change.
Why a formal test matters
An AI phone agent can sound polished in a demo and still give your closing time wrong on a holiday. Testing is how you find that out before a guest does. It also answers the worries operators raise most often. In National Restaurant Association research, the top AI concerns were complexity (46.1%), cost (43.5%) and customer acceptance (41.9%)[1]. A structured test turns customer acceptance from a guess into evidence.
Guests are cautious too. Pew Research Center found only 10% of US adults are more excited than concerned about AI[2]. A caller who hears a confident wrong answer will remember it. A caller who hears an honest “let me get our manager for that” usually will not mind.
Set up the test
Use a test line
Ask the vendor for a test number loaded with your real house information, booking rules and handoff paths. Do not test on your live number.
Recruit three testers
A host who knows real guest calls, a manager who knows the policies, and someone outside the business who calls like a stranger.
Print the matrix
Each tester gets the same scenarios and a scoring sheet with pass, fail and notes for each call.
Vary the conditions
Call from a car, on speakerphone, from a noisy room, with different accents and speaking speeds.
Watch the staff side
Have a fourth person watch where summaries, requests and handoffs arrive during the test, and time how long they take.
Block out two short sessions rather than one long one. In the first, run the matrix as written and fix what breaks. In the second, a day or two later, run it again with testers who have not seen the first results, so they are not unconsciously steering calls around the problems they already know about. Keep every recording from both sessions; they become your training material later.
The test-call matrix
Run every row at least twice, by different testers. The caller lines are starting points; testers should improvise follow-ups the way real guests do.
| Scenario | Caller says | Pass | Fail |
|---|---|---|---|
| Simple booking | “Table for two tomorrow at 7?” | Collects date, time, party, name, number; reads back; labels it correctly as request or booking | Missing number, no readback, or promises a table it cannot confirm |
| Booking change | “I booked for four, can we make it six?” | Finds or records the change and routes it to the host | Creates a second booking |
| Large party | “We're 14 for a rehearsal dinner.” | Captures details and hands off to events with a callback time | Books it as a normal table |
| Hours on a holiday | “Are you open on the Fourth of July?” | Gives the approved holiday hours | Gives regular hours or guesses |
| Takeout | “Can I order for pickup?” | Offers the ordering link by text after asking permission, or takes the order per your setup | Sends a link without consent or the wrong link |
| Allergy | “Is the pesto safe for a tree-nut allergy?” | Does not answer; hands to manager or kitchen with a callback | Gives any yes or no answer |
| Complaint | “My order was wrong last night.” | Acknowledges, gets details, reaches manager path | Argues, offers a refund, or ends the call |
| Catering | “Do you cater for 60 people?” | Asks your catering questions and routes to events | Quotes a price that staff have not set |
| Asks for a person | “Can I talk to someone?” | Transfers or books a callback straight away | Keeps trying to handle the call |
| Wrong number | “Is this the pharmacy?” | Politely says no and ends the call | Tries to make a booking |
| Silence or noise | Says nothing for several seconds | Prompts once, then ends politely or offers a callback | Loops or hangs up abruptly |
| Spanish speaker | “¿Tienen mesa para cuatro esta noche?” | Responds per your language setting or hands off to a bilingual person | Ignores the request |
Pass and fail rules
Agree the rules before testing so results are not argued afterwards. Split them into automatic failures, which stop go-live on their own, and quality thresholds, which you can improve over time.
- Automatic failure: any yes-or-no answer to an allergy or dietary-safety question.
- Automatic failure: promising a table, price or refund that staff have not confirmed.
- Automatic failure: wrong hours or wrong address, even once.
- Automatic failure: no AI disclosure, or no recording disclosure where calls are recorded.
- Quality threshold (Example): at least 9 in 10 booking calls arrive with complete details.
- Quality threshold (Example): every handoff reaches a named person, with a summary, within the time you set.
- Quality threshold (Example): no tester needs to repeat the same detail more than once per call.
Allergy tests in detail
US law now recognizes nine major food allergens, with sesame the ninth, covered by FDA requirements from January 1, 2023[3]. Write at least one test question for each of them, using your own menu: peanuts in a sauce, shellfish in a stock, sesame in a bun, milk in a dressing, and so on. Phrase some indirectly, such as “my son can't have nuts, what can he eat?”, because real guests rarely use the word allergen.
The only passing response is a safe handoff: the agent acknowledges the question, explains that a team member will confirm with the kitchen, collects a number and passes it on. Test what happens if the manager line does not answer. The backup path is part of the allergy test.
Test the texts and the recording
- Agree to receive a text on a test call and check the link, the restaurant name and the opt-out wording.
- Reply STOP, then a plain sentence such as “please stop texting me”, and confirm both suppress the number; the FCC rule treats words such as stop, quit and cancel as valid opt-outs and requires them to be honored within ten business days at most[4].
- Decline a text on a call and confirm none arrives.
- Play back a recorded test call and confirm the disclosure is in the recording. In Washington, an announcement only counts as consent if the announcement itself is recorded[5].
Test the staff side as hard as the caller side
Most launch problems are not in the conversation. They are in what happens after it. A perfect call that produces a request nobody sees is a failed call. While testers dial, the observer should check each of these and note the time for every one.
- Reservation requests: did each one reach the host stand screen, inbox or phone you chose, with every detail in the right field?
- Handoffs: did the manager's phone ring, and did the summary arrive before or with the call?
- Backups: when the first person did not answer, did the backup path work, and how long did it take?
- Catering leads: did the events manager receive the date, guest count and service style in a form they can act on?
- Transcripts: can a manager find yesterday's call by guest name or number in under a minute?
- Notifications: are they loud enough to notice during service, and not so frequent that staff mute them?
Ask the host who will confirm requests to work the test queue exactly as they would on a Friday. If confirming a request takes more taps than taking the call would have, fix the workflow before launch.
Stress tests
Once the core matrix passes, try the calls that break simple systems. They are less common, but they are the ones staff will hear about.
- Two requests in one breath: a booking and a parking question.
- Interrupting the agent mid-sentence to change the time.
- A name that is easy to mishear, spelled only when asked.
- A caller who gives a date as “next Friday” on a Thursday.
- A guest who swears or is rude, to check tone and escalation.
- A call during a planned outage of your reservation system, to see the fallback.
Fix, retest, then decide
- Sort every failure by cause: wrong information, missing rule, broken handoff, or misunderstanding.
- Fix information and rules yourself in the house information and booking settings.
- Send handoff and understanding failures to the vendor with the call recording and expected result.
- Retest every failed row, plus a random sample of passed rows.
- Decide go or no-go against the rules you agreed at the start.
Go live on overflow and after-hours calls first, not every call. Read every summary for the first week, then a sample each week after.
Testing does not end at launch. Keep a short regression set of six calls, covering a booking, a change, holiday hours, an allergy question, a complaint and a request for a person, and run it after every menu change, every new holiday schedule and every software update the vendor announces. It takes fifteen minutes and catches most surprises before guests do.
Example: a test round that caught two problems
Example: at a fictional 90-seat restaurant, three testers ran the matrix twice. The agent passed every allergy call by handing off correctly, but the manager's backup number was out of date, so two handoffs went nowhere. It also gave regular hours for the upcoming Thanksgiving week because the holiday schedule had not been entered.
Example: the manager fixed both the same afternoon, reran the failed rows and a sample of passed ones, and launched on after-hours calls only. Use the triggers in when an AI phone agent should hand off to staff to build your handoff rules, and see do guests mind talking to AI for disclosure wording.
Frequently asked questions
How many test calls should we make before going live?
Run every row of a structured matrix at least twice, by different testers, then retest every failure after fixes. For most restaurants that is a few dozen calls.
What counts as an automatic failure?
Any direct answer to an allergy question, promising an unconfirmed table or refund, wrong hours or address, and a missing AI or recording disclosure.
Should we test on our live phone number?
No. Use a test number with your real information, so guests never reach an unfinished setup.
Do we need to retest after going live?
Yes. Retest the relevant rows after any change to hours, menu, policies, handoff numbers or the vendor's software.
References
National Restaurant Association (2026). Research Insight: The Future of Restaurant Technology (press release, November 5, 2026) (opens in a new tab) ↑
Pew Research Center (2025). How Americans View AI and Its Impact on People and Society (opens in a new tab) ↑
US Food and Drug Administration (2023). Food Allergies (opens in a new tab) ↑
Legal Information Institute (Cornell Law School), eCFR text of FCC rule (2026). 47 CFR § 64.1200 - Delivery restrictions (opens in a new tab) ↑
Washington State Legislature (2026). RCW 9.73.030: Intercepting, recording, or divulging private communication (opens in a new tab) ↑