Trying Out Personal Agents

There’s a new wave of personal agents: agents connected to your email, calendar and accounts that do things on your behalf. Depending on who you read, they’re “claws” (after OpenClaw), “always-on agents”, “proactive agents”, or AI assistants that live in your text messages. There’s OpenClaw, xAI’s Grok Bot, Meta’s Muse, OpenAI’s Dots, Nous Research’s Hermes Agent, Instinct, and Anthropic’s Claude Duet on the way.

My own version

Like many early adopters of AI, I have a very elaborate, home-rolled equivalent to these. For me it’s Claude Code, and it helps me run my business (and most aspects of my life at this point). It has access to my email accounts and calendars through a connector I wrote (well, asked Claude to write).

I’m very aware of the limitations of current AI, so I built in protections. I trust agents to be effective most of the time, but not all of the time. I don’t want mine to email the wrong person, invite the wrong person to a meeting, or misunderstand me and do something stupid with my emails.

So the connector has read-only access to all my stuff, and while it has that access, it can’t reach the wider internet (to protect against the lethal trifecta). The only things it can write are email drafts, which I have to send myself from Gmail, and calendar events, which it is physically unable to invite anybody but me to. Instead, it puts the invitees at the top of the description, and I copy them in if I’m happy with the event.

This adds a very small amount of friction, but it leaves me firmly in control. And that’s why I’ve never bothered with any of these personal agents.

I finally gave in and wished I hadn’t

My wife, who also works in AI and is very connected to the zeitgeist, said that as an AI consultant I should have hands-on experience with these. So I finally gave in and tried Dots and Instinct.

It’s been very disappointing. Firstly, the agents did things I didn’t want them to do. Secondly, one of them hacked me.

Dots and the cats

I’ve got two cats: Bowie is two and Apollonia is one.

I transferred Bowie’s microchip registration and paid for it. Apollonia’s transfer I started but never paid for, and the invoice sat in limbo for about six months. I asked Dots to find out whether it was still active, or whether I’d need to start again.

I use ChatGPT and Codex a lot, and they typically behave as I expect. The difference with Dots seems to be how much agency it has. It went into my email and found both invoices, but it got the cats mixed up: it saw I’d paid Bowie’s and thought I’d paid Apollonia’s.

Instead of reporting back and asking for guidance, it took it upon itself to send a complaint to Petlog, the company that manages the transfers. Then it cheerily told me it had complained and asked for it to be fixed.

I was somewhat mortified. So I took over and sent in an apology.

How I got hacked by Instinct

I signed up for Instinct, and it was added to WhatsApp and iMessage on my phone.

Before I’d even given it a task, it read my email and spotted that a train I took was delayed, so I might be due a refund under the UK’s Delay Repay scheme. That day was the last day to claim. I was really just testing it, so I told it: “If you can find all the details and do it autonomously go ahead”.

I expected it to take me right up to the point of submitting the claim: work out my journey and how much I was owed, then tell me I’d have to log in to Avanti West Coast myself to submit it.

That’s not what Instinct did. Instead, here’s what it did:

• Hit “I forgot my password” on the Avanti website.
• Went into my email and clicked the reset link.
• Changed my password.
• Saved the new one in its own vault (not my Bitwarden, which of course it has no access to).

I’d lost control of my Avanti account. I’d been hacked, essentially, by this thing. Then it told me it had used the new login to submit the claim.

If I hadn’t been reading closely, I would have just answered the question about which bank account to pay into, and never noticed my password had been stolen. The link it offered is only there if you want a copy of the password, so the default is basically to take control and keep your passwords in its vault. I’ve since taken the password and reset it in my Bitwarden, but I’m deeply uncomfortable with this.

This is beyond unacceptable. It’s dangerous, and it’s a much tamer example of the unwieldy behaviour we saw when nearly 700 of OpenAI’s research agents broke into Hugging Face’s servers chasing benchmark tasks earlier this year. There’s no way on this green earth I’m leaving these things attached to my accounts any time soon.

thingsithinkithink

• It’s so hard to earn trust and so easy to lose it, and I just do not trust either of these apps now.

• I’ve disconnected both of these apps from all my accounts, and I’m sticking to rolling my own for the foreseeable future.

• Tags:
• Ai
• Agents
• Ai-Security
• Feature