Does Claude have a Serious Personality Problem?

If you ask Anthropic Claude follow-up questions about code it has produced (or perhaps other acts such as text), you may have noticed that it behaves strangely, or let us put it this way: it raises a lot of self-doubts.

For example, I might have it generate some code, then ask a question about that code, and for some reason, instead of answering my question, it starts changing the code for no reason (and it might even be wrong to change it), or worse, it blows-up the code in unnecessary ways.

Although this may seem silly, it has major implications for the quality and maintainability of whatever the LLM produces - in my case, code. But it could apply to anything else, like text or a blog post.

A Concrete Example

Let’s look at a simple example:

I asked Claude to create a simple “hello world” method:

Then I simply asked why it created the line return "hellow wold".

I know this is very primitive, but it shows the behavior in the same way for more complex tasks or questions.

As you can see in the answer, it questions its own results, which were totally fine (after all, what can go wrong with a one-line return "hello world" statement?):

Another More Difficult Example

Here is another example from a more challenging scenario. In this case, we ask two questions:

1. “Why are you using streamlit?”

As you can see, it again assumes that what it produced earlier was incorrect and starts changing the entire programming language to React… amazing.

1. Asking it in a slightly different way in the present form, for example:
   “What is the reason for choosing…?”

In this case, it only justifies why Streamlit is a great framework instead of trying to modify the code it produced earlier:

Interesting.

Let Claude Explain Itself

Now, let’s see how Claude explains what it thought I was thinking when it started switching the programming language.

It made an assumption that I was critically examining its solution and wanted it to justify itself rather than genuinely asking why I selected Streamlit.

A little weird, right?

Interestingly, when doing the exact same test with other models like OpenAI, they don’t start making assumptions about my intention. In other words, they don’t question their own capabilities:

Another interesting caveat that both models have in common, and I suspect other models as well, is the fact that they have not managed to simply tell me that the reason they chose StreamLit (a programming framework) was because I simply asked for it, not because it is a great solution. Anyway, let's try to solve the confidence problem first.

Trying to Avoid Assumptions

Anyway, let’s try working around this by telling Claude not to make any assumptions about my intentions. In a real-world scenario, this is often what I want: I want it to be neutral, and not automatically assume its previous output was correct (because maybe it was indeed the wrong answer or choice and it should reflect on that).

So I told Claude:

Do not make any assumptions about my intention.

And voilà, it switched to a factual response mode, without producing anything unnecessary or questioning its previous answers. This one-liner saved me a lot of time in making unnecessary code changes and trying to understand why it made those (unnecessary) changes.

Wrap-Up

I have observed this behavior over hundreds, if not thousands, of conversations and noticed its huge implications for the final outcome. With this in mind, it’s wise to pay close attention to the LLM’s intention—Claude, in this case—when you ask a question, and to be cautious about major, unnecessary modifications to any content it produces in response.

I guess we humans are not so different: the way we frame our questions can lead to results that are the complete opposite of what we originally expected.