Forget Prompting Engineering. Try this

Prompting has become the holy grail of AI these days. Without this skill, you are lost.

From summarising to generating recipes to writing blog posts, the difference between a good prompt and a bad prompt is night and day. But what if you don't have the perfect prompt?

Before I answer that question, let me mention an important advance in AI: inference-time reasoning.

In a nutshell, this is the ability of models like OpenAI's GPT-4 to reason at inference time (the time you interact with the model), as opposed to the traditional method of "teaching" the model how to reason during training time. This ability is incredibly powerful.

Why?

Because instead of spitting out the answer immediately without "thinking", the model will first "reason" - systematically trying to solve the problem, breaking it down into parts, and then assembling a well thought out answer.

You can observe this phenomenon and its incredible effect on the final answer when you interact with OpenAI o1.

Many compare this type of reasoning to the human thinking process (fast vs. slow), an amazing concept introduced by Daniel Kahneman many years ago.

How does this relate to prompting?.

With this powerful reasoning ability, models can complete tasks (such as answering a difficult question or writing an article) quite well, even if the prompt is not so good.

This works because they fill the gap by breaking down the task and adding any necessary details that were missing from the initial prompt.

So where is the catch??

Most large language models (LLMs) do not have this capability at inference time, including all OpenAI models except the two mentioned above (o1 and o1-mini) as of today.

So you have two options if you want to get the perfect answer:

1. Find the perfect prompt in a prompt library or build your own over time.
   https://docs.anthropic.com/en/prompt-library/library
   https://platform.openai.com/docs/examples
2. Build your own prompt (can take a long time if you want to get it really good).
3. Simulate the reasoning steps yourself (this Blog Post)

In this blog, I will show you the third option with a real-world example.

Suppose we do not yet have a prompt template for creating a blog post, and we want to create an article about openai.

Here is what you might typically do

Try #1:

Here is the weak prompt:

Write a 1000 word blog post about OpenAI.

And here is the result:

As you can see, the blog post looks structured, but is also quite monotonous. It lacks sections, highlighting and engaging elements.

Try #2:

OK, now let's try to ask for a perfect article like this:

Write a PERFECT 1000 word blog post about OpenAI.

And here is the result:

Compared to the first article, there is not much difference.

Try #3: Simulating Reasoning Steps.

Now let's try to simulate reasoning steps by splitting the query into two parts.

Step 1: Defining the characteristics of a perfect blog

What are the characteristics of a perfect blog in detail?

And here we go, the model knows quite well what a good article is. This now becomes a much better reference for the next steps.

Step 2: Create the blog post using the defined characteristics

Now let's use the same prompt again (within the same story), without even mentioning "perfect":

Write a 1000 word blog post about OpenAI.

As you can see, it produced an article that tried to adhere to its own standards set in the previous step, without explicitly saying so, which made the article much more exciting and varied.

Conclusion

This technique can significantly improve the outcome of almost any LLM, simply by using a divide and conquer technique. And just two steps can lead to a significant improvement in the result, even if you have no prompt.