Our AI interview was losing facts clients gave it. 18 of 19 came back

An AI interview has one job that matters more than sounding smart: whatever the client said must end up in the requirements. In Structed, some of it didn't.

A client would mention their budget, their deadline, a competitor they hate. The conversation went well. And the requirements at the end were missing one of those facts.

The safety net that matched the wrong thing

There was a check for exactly this: after the requirements were written, a second step looked for facts from the conversation that didn't make it in, and put them back.

It compared facts by their first few characters. So a fact phrased differently in the requirements looked missing, and a different fact that happened to start the same way looked present. We switched to comparing the words that carry the meaning. Of 19 facts that had genuinely gone missing, 18 came back, and the requirements got shorter, because fewer duplicate lines were pushed in.

The question asked until the turn limit

"I'll send it later" is an honest answer. The agent treated it as incomplete and asked again. And again, until the section hit its turn limit.

Now a second identical partial answer settles the field: it's recorded as "will send later" and the interview moves on. Interviews became a quarter shorter, with no topic asked twice, and the facts collected stayed the same.

The minute-long reply

Early on, replies took 20 to 60 seconds. The model wasn't slow. Every turn, the code looked up the trace of the whole conversation so far, and that lookup grew with each turn. Prompts were also fetched fresh on every call.

We removed the lookup and cached the prompts. Replies dropped to 1 to 13 seconds with tracing still on.

What these have in common

None of them was a model problem. Each was ordinary code around the model, failing quietly. You find those by running the interview end to end with test clients, again and again, and reading what came out, not by improving the prompt.