How I built Kate, the AI that calls in my lunch order

The other day I had Kate call Jersey Mike’s and order my sandwich, the way it has several times before. This time, when I walked in to pick it up, the crew behind the counter asked: “Was that AI? How did you do that? That was awesome.” I explained it, left a business card, and walked out with a coupon for a free sandwich.

Nobody on that call was confused or annoyed. A busy sandwich shop took a phone order from an AI, read it back, gave a ready time and a total, and wanted to know how it worked. This post is the answer to their question.

Kate is the outbound assistant I built for my own errands. The receptionists we build for clients answer the phone. Kate places the call, which turns out to be a harder problem in a few specific ways. Here is how it’s put together.

The stack

Retell runs the voice agent: speech in, an LLM in the middle, speech out, plus the phone number it dials from.
One long prompt holds the behavior. Each call fills in three dynamic variables: who Kate is calling (caller_name), what to get done (caller_purpose), and an optional scenario playbook (playbook).
Two key tools on the agent: press_digit for phone menus, and a warm transfer to my cell for the rare call that needs me live.
Three ways to start a call, all hitting the same Retell endpoint. More on those below.

The hardest part: is this a person or a machine?

An inbound receptionist always knows who it’s talking to: a caller. An outbound agent doesn’t. The first thing it hears might be a person, a phone menu, a hold recording, or another AI receptionist. Each one needs a different first move, and getting it wrong ends the call.

Kate’s prompt opens with that decision, before anything else:

The default is human. “Sunny Kitchen, how may I help you?” is a person. That’s how small businesses answer the phone.
A machine lists options. “Press 1 for sales, say billing for billing” is a menu. Only hard evidence like that, or hold music, counts as automated.
Robots get keywords, humans get the message. To a menu, Kate says one routing word or presses one digit and nothing else. No greeting, no explanation of why it’s calling.

The rule that matters most is the tiebreaker. If Kate isn’t sure, it greets a human. Saying “Billing.” to a real person who just said hello makes them hang up. Greeting a menu costs a few seconds, and Kate can still route once the options start playing.

That asymmetry came from a real call. Kate’s first Papa John’s order spent the opening minute navigating a multi-level phone tree on its own, pressing 3, then 2, then 5, before a person picked up. The whole call ran 4 minutes 22 seconds and ended with two large pepperonis, $19.73, ready for pickup. When I shared that call, the phone tree was the part people found most impressive.

press_digit is what makes that possible. Retell exposes DTMF as a tool the agent calls while it’s listening, so the model chooses the digit from what it just heard, and can try again if the menu didn’t take it. A fixed dial string sent when the call connects can’t do that, because it’s decided before anyone knows what the menu says.

Ordering food is its own skill

Most of Kate’s calls are food orders, and they fail in boring, specific ways. The prompt has an ordering section built from those failures:

Let the store lead. Answer what they just asked before raising anything new.
One complete item per turn. Size, bread, every topping and exclusion, then pause.
Correct a wrong read-back immediately. “Small correction: regular lettuce, not extra.” Never agree to a read-back that doesn’t match.
Never guess a number. If a price sounds odd, ask them to repeat it.
End with the ready time, the total, and one final read-back, and wait for the store to confirm it before saying goodbye.

Holds were the other problem: Kate used to talk into the silence after “one second, please.” That fix was mostly in Retell’s speech settings rather than the prompt, and it has its own post: Teaching a voice agent to wait.

A dropped call is a confirm call, not a reorder

One dinner order took four calls. The first connected and dropped after 16 seconds. The obvious move is to redial with the same instructions. But at 16 seconds you don’t know whether the restaurant already had the order, and redialing blind risks two identical orders waiting under the same name.

So the second call went out with a different purpose: you are calling back about an order placed a few minutes ago; do not place a new order; first confirm whether the earlier one is in the system. That worked, and it’s now a standing rule. After any dropped order call, the next call confirms. It never reorders.

Three ways to start a call

1. From my computer. A shell script creates the call through Retell’s API, polls every 10 seconds until it ends, and prints the outcome. Claude Code wraps it in a skill, so “order the usual from McAlister’s” turns into a confirmed call. A saved-places file holds the number, the usual order, and notes for each place, so a repeat order never needs describing again.

2. From an Apple Shortcut. A Python script turns the same places file into a signed Shortcut: pick a place, confirm what Kate will say, and it dials. Two bugs in it could only show up on a real iPhone. Imported Shortcuts don’t automatically pass one action’s output to the next, so every variable came through empty until each action was wired explicitly. And text fields need a different wrapper than the one plain variables accept. That one ate the phone number, and Retell rejected the call for having no number to dial.

3. From Claude on my phone. A Cloudflare Worker exposes Kate as a remote MCP server with two main tools: make_call and get_call_status. I ask Claude for a sandwich, it reads back the number and the order, I say yes, and it dials. When the call ends, Claude reads me the transcript summary.

The first version used a static API token. Then it turned out the iPhone’s connector setup has no field for request headers, so the token could never be sent. The Worker now sits behind an OAuth layer with a one-field sign-in page, and the phone connects through the standard “Requires sign-in” flow.

One detail worth knowing if you build something similar: Retell’s call summary and success flag come back empty if you read them the instant a call ends. The analysis lags a few seconds behind, so get_call_status waits and asks again.

The bug that hid behind another bug

Before Kate ever placed a real call, its setup had two problems that hid each other:

The skill still had a placeholder where the agent ID belonged, so it refused to dial at all.
The agent’s first line was hardcoded to “Hello, how can I help you today?” That’s fine for a receptionist answering a call, but it’s absurd when Kate is the one calling Papa John’s.

The first bug kept the second from ever showing up. Fixing the ID would have produced an agent that called a pizza place and asked how it could help them. Removing the fixed first line lets the model write the opening from the prompt, which already knows to start with “Hey, is this Papa John’s?”

Kate had been “working” for days at that point, but only in the sense that it had been built. It had never actually placed a call. Now nothing counts as done until a real call has gone through, and the first test call for anything new goes to my own cell.

What this means for a business owner

Kate exists to run my errands, but every lesson in it applies to the receptionists we build for clients:

Small businesses answer like people, not like menus. An agent has to sound natural to the person on the other end. Kate’s “was that AI?” moment happened because it did.
Real calls find what tests don’t. The phone tree, the dropped call, and the iPhone bugs were all invisible until a real call went through.
Most of what makes a voice agent good is boring. Read-backs, ready times, totals, and knowing when to stay quiet.

If you’d like to hear what that sounds like, call our demo line at (615) 583-8217. If you want to talk about a receptionist that answers your phone nights, weekends, and holidays, I’m at (629) 217-2310.