How can you get ChatGPT to think

If someone asked you to write a report or an article, would you sit down and just write it one word at a time until you were finished? Well maybe when you were six years old. However, that is exactly what the current generation of Large Language Models (LLMs) like ChatGPT do. You provide them with a prompt, and they respond one word at a time. This was clearly explained by top AI Researcher Andrew Ng in a very watchable lecture at the start of 2024.

He pointed out that when an adult writes a report, they generally think about what points to include, do some research, write an outline, draft a first version, reflect on it, and finally produce a final version. He showed how he was able to get GPT3 to produce better results than GPT4 by using techniques similar to how a human would break down a task. He calls this “Agentic Workflow”, and it has been a developing area of research in 2024. What if we could get our favourite LLM to think things through like this without fancy software that does “Agentic Workflow“. Well, we can (well almost), and tests show that this produces much better results. Here’s how to easily try this for yourself.

A current frontier LLM (Claude 3.5 Sonnet) was prompted “I would like you to write a 500-word article on the story of Blockbuster and Netflix. The style should be easy to read and informative.“ The results were good, but by using pseudo Agentic Workflow you can get a much better outcome. Here’s the prompt used:

I would like you to write a 500-word article on the story of Blockbuster and Netflix. The style should be easy to read and informative.

Output 10 key events that should be included in this article. Put these between <keyEvents> tags.
Output the structure of the article including any lessons that can be learned. Put these between <structure> tags.
Write the article based on the structure and keyEvents. Put this between <firstAttempt> tags.
Reflect on the firstAttempt and be critical of the content, suggest improvements. Put this between <reflection> tags.
Rewrite the article based on the reflection, put this between <final> tags.

Where tags are mentioned above, this is a common way of labelling sections of a document in HTML and the LLM understands this. It’s fascinating to see how the LLM will produce the sections exactly as specified, build the structure and then first draft, reflect and suggest improvements, and then finally produce the final article. It’s as if it’s thinking things through. The main step missing is the “doing research” step, however you could achieve this with LLMs that allow you to attach a knowledge base (both ChatGPT and Claude do this with the paid version) or retrieve relevant information from the web (Perplexity does this).

Various LLMs were asked to rate the articles produced the normal way and also the Agentic Workflow way. All of them rated the second article as a lot better. Give it a go and see if you think it produces better results for you.

What’s interesting about this is that we are only just discovering how to get the most out of LLMs. The techniques used here are similar to the techniques used by OpenAI o1, the latest LLM from OpenAI which is claimed to have the ability to “reason“. As well as some LLMs becoming bigger and more expensive to train, some are becoming smaller and faster but Agentic Workflow is used to prompt them 20 or 30 times to get them to slowly develop answers and “think” things through.