My AI is a very confident intern

Most of my days go into AI agents that do things on a computer. They click the button, fill in the form and send the email. Picture the best intern you have ever had: fast, never tired, has read every manual ever written. Now picture that intern also has zero fear of being wrong, and says “done!” with total confidence whether or not anything got done. I work next to one every day.

An agent only knows what you put in front of it. If what it sees is a little off, it will not notice, and it will build a whole confident story on top. Show it a page where the save button turned green, and it will tell you it saved, even if the server quietly threw half of it away. Hand it a summary of what happened an hour ago, and it treats the summary as gospel, including the parts the summary got wrong. I have watched one proudly tell someone their data was “already in the system”. It was, because the agent had put it there itself an hour earlier and forgotten.

So most of my job is making sure what it sees is true: unfinished things look unfinished, and old facts say how old they are. My first instinct, every single time, is to fix the rest with words. I add a line to the instructions, like “please do not make up links” or “only do this if the person asked”. It works great in testing, and then it fails with a real person on the other end. The fix that works is boring. You check it in code. If the AI wants to show you a link, that link has to exist somewhere real, word for word, or it does not show up. If it is about to press Send, something checks that a human asked for that, and the check wants the receipt.

The bugs that hurt the most came from me being clever. A little on/off switch for some special case, a shortcut that skips a step when it “obviously” is not needed. Each one makes total sense on the day you write it. Six weeks later someone uses the thing in a way you never pictured, the switch never flips back, and it all goes wrong in complete silence. A loud bug gets fixed by lunch. A quiet one gets fixed when someone emails you, politely confused. I now get slightly nervous every time I add a yes/no setting, because each one is a little door I am going to forget to close.

At one point we had a run of bad weeks and did the natural thing. We added more safety checks, more rules and more “are you sure?” Within days the AI started refusing perfectly reasonable requests. We had made it so careful it stopped being useful, so we ripped most of it back out. Almost everything that made it better was a deletion: fewer paths, fewer special cases, fewer places for problems to hide.

Watching something you built open an app and do a boring task for someone while they go make coffee still feels a bit like magic to me.