Notes: "Reasoning with o1"

Notes from the short course "Reasoning with o1" on DeepLearning.AI, presented by Colin Jarvis of OpenAI -

o1 is a reasoning model for complex tasks that require broad general knowledge, including function calling and image input. It can reason through complex tasks in domains like mathematics, coding, science, strategy, and logistics.

o1 is different from other models because it thinks before it speaks.

o1 requires less context in prompting to produce very effective results.

o1 uses large-scale reinforcement learning to generate a chain of thought before answering.

o1's chain of thought (CoT) is longer and higher quality than what you can typically attain by a prompt alone.

CoT contains behavior like:

- Error correction

- Trying multiple strategies

- Breaking down problems into smaller steps

o1 performs well at understanding images out-of-the-box. It can be used to extract a detailed JSON that describes the image and what's going on in it.

o1 follows a test-and-learn approach in its reasoning, which gives it multiple chances to detect hallucinations before providing an answer.

o1 can be used for customer service and other applications where multi-turn conversations are needed.

The o1 family of models scale compute at inference time by producing tokens to reason through the problem.

o1-mini is a faster, cost-efficient reasoning model tailored to coding, math and science

Weigh the benefits of increased intelligence against the costs and latency when deciding whether to use o1 for a particular task.

Key Principles for Prompting o1 Models

• Be simple and direct: Write straightforward and concise prompts.
• No explicit chain of thought is required: You can fully skip the step-by-step ('Chain of Thought') reasoning prompts
• Use structure: Break complex prompts into sections using delimiters like markdown or XML tags, or quotes. This structured format enhances model accuracy - and simplifies your own troubleshooting.
• Show rather than tell: Give 1 or 2 contextual examples to give an understanding of the domain of your task.

Trade-offs with o1

• Increased intelligence: o1 provides much greater intelligence compared to other models, but this comes at a higher cost and latency.
• Reasoning tokens: o1 generates extra completion tokens that are used for reasoning, which are not visible to the user but are included in the cost and context limit.
• Output truncation: If the output goes over the context limit, it will be truncated.
• Prompt engineering: While o1 is trained to infer and execute chains of thought, there may still be a need for explicit chain-of-thought prompting for specific or nuanced tasks.
• Structured prompts: Providing structured prompts can ensure that o1 follows instructions with greater accuracy, but it requires more effort to create.
• Examples: Giving examples can help o1 learn and perform better, but it requires providing 2-3 examples to achieve consistent performance.