When Should AI Update Your Contacts?

You're talking to your AI assistant about an upcoming meeting. You mention, mid-sentence, that Marcus recently moved from Stripe to a new fintech startup. The AI responds helpfully. The conversation moves on.

But did it update Marcus's contact record?

If it did — silently, immediately — you might not notice for weeks. And when you do notice, you might feel a small, specific discomfort. Not quite alarm. Something more like: I didn't say you could do that.

If it didn't update the record, you'll probably forget to do it yourself. Marcus will sit in your contacts forever as "Head of Risk, Stripe" until that outdated entry causes a real problem — you send him something assuming a role he left eighteen months ago.

Neither outcome is good. So: when should AI update your contacts?

Three reasonable answers

The obvious options are:

Auto-update. The AI extracts the information, verifies its confidence, and writes to the record without asking. Fast, convenient, invisible.

Propose a patch. The AI prepares the update and presents it to you before writing anything. "I noticed Marcus may have changed jobs. Want me to update his record?" You click yes or no. Nothing changes without your involvement.

Ask first. The AI flags the ambiguity in real time. "You mentioned Marcus is at a new company — should I make a note of that?" You confirm, then it writes.

Most discussions of this problem treat it as a trust-level question. How much do you trust the AI? High trust → auto-update. Low trust → always ask. That framing is too simple.

The part that actually matters

The thing I've noticed is that the right answer depends less on trust and more on what kind of data you're talking about.

There's a category of contact data that feels factual and external — job title, employer, phone number, email address. These come from the world. Marcus either works at Stripe or he doesn't. If the AI is confident and has a recent signal (Marcus literally told you, or you read it on a company page), auto-updating feels fine. The field is either right or wrong. Fixing it is just maintenance.

But there's another category of contact data that is your interpretation — notes about how someone communicates, what they care about, how a relationship has evolved, the context of how you met. This data lives inside your model of the person, not in the world. It's not factual in the same way. And having an AI silently rewrite your understanding of a relationship based on a conversation fragment is a different thing entirely.

Marcus's new employer: safe to auto-update with high confidence.

A note you wrote about Marcus being someone to trust with sensitive decisions: not something the AI should touch without being asked.

The distinction isn't confidence level — it's whether the data represents external fact or internal interpretation. Most systems don't make this distinction. They apply a uniform policy across all contact fields, which is why the results feel off in both directions.

What a real policy looks like

In Anthropic's documentation on agentic systems, they describe a "Plan Mode" approach — rather than approving each action one by one, the AI surfaces its intended changes upfront, the user reviews, and execution only begins after sign-off. It's a meaningful step forward from either full autonomy or per-action interruptions.

But even Plan Mode applied to contact updates misses the field-type distinction. You don't need to approve a phone number correction. You probably do need to approve a change to someone's relationship tags or the notes you wrote after a difficult conversation two years ago.

The McKinsey AI in the Workplace survey from 2025 found that organizations with human-in-the-loop confirmation steps reported 2.3x higher adoption rates than fully autonomous deployments. The reason isn't that humans are slow or conservative — it's that visible, reviewable actions build trust faster than invisible ones. People keep using tools they understand. They stop using tools that feel like they're making decisions on their behalf.

For personal contacts — as opposed to enterprise CRM records where data serves a pipeline — the visibility question matters even more. This is your mental model of the people in your life. That's worth keeping under your own authority.

What I actually want

If I'm honest, the answer I want is something like this: the AI should auto-update clearly external, verifiable facts when its confidence is high. It should propose updates to any field I've personally written or categorized. And it should leave interpretation entirely alone unless I explicitly ask.

That's not a simple setting. It requires knowing which fields are "external" and which are "interpretive," which is a harder problem than it sounds. It requires per-field approval policies, not a single toggle. It requires an audit log so that even when updates are automatic, I can see what changed and when.

None of that is technically hard. But it asks the product to make a distinction that most tools have never had to make — between the world's facts about a person and my own understanding of them.

Those are different things. Keeping them separate is one of the things a good personal knowledge base should actually do.

---

Asgeir Albretsen is the founder of Harbor.