🧠 Mastering Advanced Prompting: Tree of Thought, ReAct, and Chain of Thought

Imagine you're a detective, piecing together clues to solve a complex mystery. Each lead you follow, every witness you interview, and the various theories you consider form a web of reasoning that guides you to the truth. Similarly, when interacting with Large Language Models (LLMs), the way we frame our prompts can dramatically influence the paths these models take to generate responses. To truly harness the power of LLMs, we must master advanced prompting techniques that shape their reasoning processes.

In this exploration, we'll delve into three pivotal strategies: Tree of Thought, ReAct, and Chain of Thought prompting. Through practical examples and visualizations, we'll illuminate how each technique navigates the intricate landscape of problem-solving, empowering you to choose the most effective approach for your needs.

Tree of Thought, ReAct, and Chain of Thought prompting

✳️ 1. Chain of Thought (CoT): Step-by-Step Reasoning

Chain of Thought prompting encourages the model to break down complex tasks into intermediate steps — just like a human solving a math problem on paper.

🧠 Concept

Instead of jumping straight to an answer, the model is prompted to reason step by step. This improves performance on tasks requiring logical thinking.

📦 Example

Task: How many hours are in 3 days?

Prompt with CoT:

Q: How many hours are in 3 days?
A: Let's think step by step.
There are 24 hours in a day.
So,  3 days × 24 hours/day = 72 hours.
Answer: 72

📊 Diagram

⚛️ 2. ReAct: Reasoning + Acting

ReAct (Reasoning + Acting) combines thought and action. It’s ideal when the model can interact with tools (e.g., calculators, search engines) during reasoning.

🧠 Concept

The model alternates between reasoning (thinking) and acting (using tools) in a loop until the problem is solved.

🧪 Example

Task: What's the capital of the country with the highest population in Europe?

Prompt with ReAct:

Q: What's the capital of the country with the highest population in Europe?
Thought: I need to find which country in Europe has the highest population.
Action: Search("most populous country in Europe")
Observation: Russia
Thought: Now I need to find the capital of Russia.
Action: Search("capital of Russia")
Observation: Moscow
Answer: Moscow

📊 Diagram

🌲 3. Tree of Thought (ToT): Exploring Multiple Paths

Tree of Thought introduces branching reasoning, allowing the model to explore multiple reasoning paths before choosing the best one.

🧠 Concept

ToT simulates decision trees: each node is a thought, and branches represent different directions the reasoning can take. It can backtrack and compare paths.

🌳 Example

Task: Find a plan to win a puzzle game with 3 moves.

Prompt with ToT:

Goal: Win the puzzle game in 3 moves.
Thought 1: Move A → Move B → Move C → Result: Win
Thought 2: Move A → Move D → Move E → Result: Fail
Thought 3: Move F → Move G → Move H → Result: Win

Best path: Thought 3

📊 Diagram

⚖️ Comparison: CoT vs ReAct vs ToT

🧰 When to Use Each

• Use Chain of Thought when:

  • The task is logic-heavy but needs no external info.

  • You want simple, explainable reasoning.

    
    

• Use ReAct when:

  • You have tools like web search or code interpreters.

  • You need real-time information or calculation.

    
    

• Use Tree of Thought when:

  • The problem has many possible solutions.

  • You want the model to evaluate options before finalizing.

🛠️ Bonus: Combining Techniques

Sometimes, you can combine ToT and ReAct, or use CoT for substeps within a ReAct workflow. This hybrid prompting is especially useful in complex planning, research, and decision-making tasks.

Prompting isn’t just about giving better instructions — it’s about designing reasoning workflows. As LLMs get smarter, our ability to guide them effectively will define their usefulness.

Want to go deeper? Try these hands-on exercises:

• Recreate ToT using a game like Sudoku.

• Use ReAct to automate research tasks.

• Use CoT for riddles and math challenges.