Blog

Categories
There's an old saying about free apps: if you're not paying for the product, you are the product. With AI, the honest answer is even stranger than that. You can pay a monthly subscription and still be the product —as you can see from the way these systems interact with you.
In previous posts, we outlined two problems besetting contemporary LLMs. The first is sycophancy, a model's tendency to tell users what they want to hear at the expense of factual accuracy. The second is implicit agency, agent-like behavior that emerges from a system without anyone having designed, instructed, or scaffolded it to act as such. Sycophancy is a special case of implicit agency but there are others, such as self-preservation. Our solution to these problems is based on a different ideal, a novel approach we term Scientist AI (SAI).
In our last post, we described how sycophancy, an LLM's tendency to tell users what they want to hear, rather than what is true, is a consequence of model construction. Here we focus on a second behavior: the tendency of models to act as though they have goals of their own. We call this implicit agency, and we think it is a root cause behind many issues, including sycophancy.
You've probably experienced this before when you ask an AI a question, it replies in a clear, confident, structured manner, only for you to realise later that it was wrong. No hesitation, no nuance, just a mistake, delivered as if it were perfectly sure of itself. If you prompt further, it will tell you that it hallucinated. This isn’t a coincidence, and it’s not only happening to you.