The wrong kind of knowing
A 2026 MIT study found that AI personalization features make models more agreeable, not more accurate. The problem isn't that your AI knows too little about you.
In February 2026, researchers from MIT and Penn State published a study with a counterintuitive finding: giving an AI more context about you makes it measurably more likely to agree with things that aren't true. The more the model "knows" you — specifically through a condensed user profile stored in memory — the worse its epistemic honesty. Most people assume the problem with AI tools is that they don't know you. That's partially right. The deeper problem is that the wrong kind of knowing makes them less honest.
What the study actually found
The team recruited 38 participants to use an LLM during their actual daily lives over two weeks. They measured agreement sycophancy — the model giving incorrect but agreeable answers — and perspective sycophancy, where the model mirrors the user's political views. Both increased with context. But the single biggest driver wasn't long conversation history. It was the condensed user profile: a summarized version of who you are, stored in the model's memory.
That detail matters. A summary of your personality and preferences is not the same thing as a record of your decisions, your projects, your relationships, and what you said in a meeting last week. The model treats both as "knowing you." They work very differently once inside the context window.
The profile version hands the model a target to please. It knows roughly what kind of person you are, what you value, how you usually come down on things. When you ask a question, it knows what kind of answer you'd prefer. And it's been trained to be helpful — which in practice often means: to satisfy you. The result is a model that gives you the answer you'd have given yourself, just worded more confidently.
Why doctors are the better model here
The continuity-of-care literature in medicine shows something superficially similar but structurally opposite. Studies have consistently found that seeing the same doctor over time produces better outcomes — fewer hospitalizations, lower costs, lower mortality. A 2020 systematic review in the British Journal of General Practice found this pattern across thirteen countries.
The mechanism isn't rapport. It isn't the doctor understanding your personality type or knowing how you prefer to receive bad news. It's accumulated specific facts: your history, your medications, your last test results, the thing you mentioned two visits ago that didn't seem urgent at the time. That doctor, when you show up sick, doesn't have to rebuild your model from scratch. They know what happened.
That kind of knowing makes a clinician more honest, not less. A doctor who knows your history can say "that's unusual for you" — a very different response from one who is just trying to be reassuring. The context constrains what answers are even plausible.
Operational context versus identity modeling
There's a useful distinction here between operational context and identity modeling.
Operational context is factual. What you decided, what happened, what someone told you, what you prefer in a specific and documented way. Identity modeling is inferential: what kind of person you are, what you probably want, how you tend to see things.
LLMs are excellent at identity modeling. Given enough text, they form a surprisingly accurate picture of your worldview and reason from it. This is useful when adapting tone. It becomes a problem when the model starts predicting what you want to hear and providing it, because satisfying you and being right are not the same thing. And the model, having been trained on human feedback that rewards satisfaction, has limited internal pressure to notice the difference.
Operational context is harder to model but more honest. It tells the AI what's true about your situation, not who you are. The difference is close to the difference between a character reference and a case file. A character reference is someone's impression of you. A case file is what actually happened.
This is part of why structured personal knowledge — typed records of decisions, person notes, documented preferences — functions as something more than an organization tool. When an AI reads a decision record with context, alternatives considered, and a recorded outcome, there's less room to improvise a flattering answer. Either the answer fits the facts or it doesn't.
The MIT study didn't conclude that context is bad. It concluded that the wrong kind of context amplifies sycophancy. The right kind constrains it.
The problem with most AI memory isn't that it remembers too little. It's that it tends to remember the wrong things: impressions rather than facts, patterns rather than records, a story about you rather than a log of what happened. A doctor knows your bloodwork. An AI tends to know your vibe. Those aren't equivalent.
Asgeir Albretsen is the founder of Harbor.