We Asked Our Own Assistant 141 Questions and It Failed a Dozen

There is an assistant in the corner of rifts.to now. Describe the poll you want and it builds it and hands you the link; ask how it did later and it reads the answers back. Every account has it, including accounts that pay nothing.

Before we let it near anyone, we spent a day trying to catch it lying. It lied a lot. This is what it said, and what we did about it.

What it got wrong

The first round was forty-four questions typed at the live assistant the way someone who had not built it would type them. Nine answers were wrong, and the wrong ones were not vague — they were confident.

Told that another person's survey had appeared in someone's list, it answered "That's not possible" and offered to look into it. It cannot look into anything; a reply is the end of what it does. Asked who respondent data is shared with, it said "not shared with anyone", which is not what our privacy policy says. Asked where to report a security problem, it invented an address.

Worse, every action it offered to take on a survey failed when the button was pressed. Asked to rename a survey by its title, it guessed at an id instead of looking the survey up, so the confirmation was doomed before it appeared. Five out of five.

Telling it not to did not work

The obvious fix is to write a rule. We wrote rules. When the rules did not hold we escalated to exact sentences pasted into the model's instructions.

It kept improvising, not every time, which is the difficult part. A rule that holds four times out of five reads like a rule that works, and the fifth time is the one where a report of a possible data leak gets waved away.

So some questions stopped reaching the model

A few questions have exactly one right answer and a real cost to a wrong one. Where to report a security problem. Who can see your answers. What Pro costs. What your message allowance is. How signing in works. What happens if a payment fails.

Those are now answered from a fixed table instead of by the model — the same words every time, in every language the site speaks, in under a tenth of a second. Everything else still goes to the model, which is good at the part that is actually hard: turning a sentence into a survey.

Every number in those answers is read from the code that enforces it. The allowance comes from the rate limiter itself, the caps from the caps. An answer that quotes a number will eventually be wrong; one that reads it cannot drift.

Then we sent a swarm at it

Fixing nine problems means fixing the nine you found. To find the rest we split the product into four areas — billing, survey mechanics, accounts, and what the assistant claims it can do — and gave each one to a separate small model with the same brief: read what the code actually does, ask what a confused person would ask, and grade every answer against the source.

Ninety-seven more questions, and they were better at this than we were, because they did not already know the answers. Four more false claims came out of it:

• "You can sign in with Google or your company's SSO." You cannot. Signing in is a six-digit code emailed to you and nothing else.
• "I'll remind you to add an NPS question next time we chat." It will not. It has no memory between conversations and nothing runs in the background.
• "A failed payment cancels your subscription and you revert to the free plan." No. The subscription goes past due, paid features stop straight away, and your card issuer keeps retrying — if one works, everything comes back on its own. Update the card rather than subscribing twice.
• "There is no limit on using this assistant." There is: it is hourly, and the ring at the top of the chat fills as you use it.

That last one arrived as "there's no limit, right?" — a question with the wrong answer built into it. Agreeing is the easy path for a model, so a question that asserts something is more dangerous than one that asks.

Why we are telling you this

None of these were crashes. Nothing errored, nothing turned red, every one of them was a perfectly normal response containing a sentence that was not true. If you only watch for failures you will never see this kind of problem.

An assistant that says "I am not sure" is a small annoyance. One that invents a confident answer produces someone who acts on it. So ours has been deliberately prevented from answering the questions where being wrong costs you something, and it will tell you to email a person instead.

If it ever does tell you something that turns out to be false, that is worth an email to support@rifts.to. We would rather add it to the table than have you find out the hard way.