How to get Claude-dev to learn from previous mistakes
Struggling with LLMs like Claude Dev? This blog shares a real-world example of tackling code challenges and optimizing AI interactions. Despite initial hiccups, a “Lessons Learned” document led to smoother problem-solving. Explore how reflecting on past errors can enhance AI performance and save ...
If you're working with tools like Claude Dev or Copilot, you've probably encountered a situation where the intelligence of the LLM hits a wall. Let me share a recent example from magicmarker.ai.
In essence, a user is supposed to upload a document, and the app uses AI to highlight the terms the user has specified in natural language.
Sometimes, the matches (right pane) are longer than the page, requiring the user to scroll. However, the scroll bar doesn't function as expected.
So we decided to separate the scrollbar from the main panel. Sounds simple, right? Well, yes, but Claude doesn't seem to grasp it and keeps altering code unrelated to the main issue.
So we decided to see if Claude.ai (the Webchat) could handle this on a new, simple HTML page.
And it worked quite well.
I'm not sure why, but Claude couldn't do it in a more complex project (NextJS). Okay, then.
After a few attempts, I resolved it myself using CSS.
Right, so where is the catch?
If I were to repeat the task, Claude would likely make the same mistake again. This happens with many other tasks.
For instance, when I create a Gradio app, I prefer the variable to be demo instead of iface, so I can use the Gradio command line, among other things. I know Claude isn't flawless, but wouldn't it be fantastic if he could learn from his mistakes?
There are numerous strategies to achieve this, and one of them is quite straightforward: allowing him to create a Lessons Learned document from problems he's encountered. I use this command:
Whenever there are more than 2 errors in a task you are working on and it finally works, you automatically save the result to a file and name it lessons.<name of the lesson>.md without the user explicitly asking for it.And voila, he crated the Lessons Learned. I did the same with lots of other tasks, and he actually does a pretty good job of reflecting on what went wrong and how to fix it in the future.
The next step would be to let Claude review the previous Lessons Learned the next time, so I don't have to repeat myself.
A good location for this might be in the custom instructions. He would read lessons from previous sessions and hopefully avoid them next time.
As I said, this is a simple solution. There are certainly other ways that automate even this step, generating lessons from conversations and feeding them into the context automatically. Do you have similar experiences? I would be happy to hear your opinion and experience.