Prompt Engineering Guide Condensed

A concise summary of promptingguide.ai Covers core concepts, techniques, applications, risks, and best practices.

Table of Contents

1. Introduction
2. LLM Settings
3. Elements of a Prompt
4. General Tips for Designing Prompts
5. Basic Task Examples
6. Core Prompting Techniques
7. Advanced Prompting Techniques
8. Agentic & Tool-Augmented Techniques
9. Risks & Misuses
10. Model-Specific Notes (ChatGPT)
11. Tools & Libraries

1. Introduction

Prompt engineering is the discipline of developing and optimizing prompts to efficiently use large language models (LLMs) for a wide variety of applications.

• Researchers use it to improve LLM safety and performance on complex tasks (QA, arithmetic reasoning).
• Developers use it to design robust prompting techniques that interface with LLMs and external tools.
• It encompasses all skills and techniques for interacting with and building on top of LLMs.

Chat model roles: system (sets behavior), user (input/queries), assistant (model response). Most examples use only the user message for simplicity.

2. LLM Settings

Key parameters when calling LLMs via API:

Rule of thumb: Adjust Temperature/Top P or Frequency/Presence penalty, but not both pairs simultaneously.

3. Elements of a Prompt

A prompt can contain any of the following:

1. Instruction — The specific task you want the model to perform.
2. Context — External information to steer the model toward better responses.
3. Input Data — The actual input or question to process.
4. Output Indicator — The desired format or type of output.

Not all elements are required; the format depends on the task.

4. General Tips for Designing Prompts

Start Simple

• Prompt design is iterative. Start simple, then add context and elements.
• Break large tasks into simpler subtasks.

Use Clear Instructions

• Use action verbs: Write, Classify, Summarize, Translate, Order, Extract.
• Place instructions at the beginning; use separators (e.g., ###) between instruction and context.

Be Specific and Descriptive

• The more detailed and relevant the prompt, the better the results.
• Provide examples (few-shot) to specify desired output format.
• Avoid unnecessary details that don’t contribute to the task.

Avoid Impreciseness

• Be direct. Avoid vague or overly clever wording.
• Bad: “Explain prompt engineering. Keep it short.”
• Good: “Explain prompt engineering in 2-3 sentences using simple language.”

Say What TO Do, Not What NOT To Do

• Focus on desired behavior rather than prohibitions.
• Bad: “Don’t recommend horror movies.”
• Good: “Recommend family-friendly movies rated PG or G.”

5. Basic Task Examples

6. Core Prompting Techniques

6.1 Zero-Shot Prompting

Directly instruct the model to perform a task without any examples.

• Modern instruction-tuned models (GPT-3.5, GPT-4, Claude) excel at this.
• Works well for simple, familiar tasks.
• When it fails, move to few-shot prompting.

6.2 Few-Shot Prompting

Provide demonstrations (exemplars) in the prompt to enable in-context learning.

• Format: show input/output pairs before the actual query.
• Even random labels with correct format help more than no labels.
• Label space, input distribution, and format all matter.
• Limitation: Insufficient for complex multi-step reasoning tasks.

6.3 Chain-of-Thought (CoT) Prompting

Prompt the model to show intermediate reasoning steps before giving the final answer.

• Few-shot CoT: Include reasoning steps in exemplars (Wei et al., 2022).
• Zero-shot CoT: Append “Let’s think step by step” to the prompt (Kojima et al., 2022).
• Emerges primarily in sufficiently large models.
• Greatly improves arithmetic, commonsense, and symbolic reasoning.

6.4 Automatic Chain-of-Thought (Auto-CoT)

Eliminates manual example crafting by:

1. Question clustering: Partition dataset questions into clusters.
2. Demonstration sampling: Select representative questions and generate reasoning chains via Zero-shot-CoT.

Uses heuristics (e.g., question length, number of reasoning steps) to encourage simple, accurate demonstrations.

7. Advanced Prompting Techniques

7.1 Self-Consistency

An ensemble method for CoT prompting:

1. Sample multiple diverse reasoning paths via CoT.
2. Take a majority vote over the final answers.

Improves reliability over a single CoT chain by leveraging the fact that complex problems often have multiple valid reasoning paths.

7.2 Generate Knowledge Prompting

For tasks requiring background knowledge:

1. Use the LLM to generate relevant knowledge statements about a question.
2. Use that generated knowledge as additional context to answer the question.

Helps when the model lacks implicit knowledge for the task.

7.3 Prompt Chaining

Break complex tasks into a pipeline of simpler sub-tasks, where the output of one prompt becomes the input of the next.

• Improves reliability and transparency.
• Useful for long-document processing, multi-step transformations, and structured generation.

7.4 Tree of Thoughts (ToT)

Generalizes CoT by maintaining a tree of reasoning paths:

• Allows the model to explore multiple reasoning branches.
• Supports backtracking when a path fails.
• Uses search algorithms (BFS/DFS) over the thought tree.

Best for complex problems requiring planning, exploration, or strategic lookahead.

7.5 Retrieval Augmented Generation (RAG)

Combine an LLM with an external knowledge retrieval system:

• Retrieve relevant documents for a given input.
• Concatenate retrieved documents with the prompt as context.
• Generate the final answer conditioned on both.

Benefits: More factual, up-to-date responses; reduces hallucination; no need to retrain the model.

7.6 Automatic Prompt Engineer (APE)

Automated instruction generation and selection:

1. Use an LLM to generate candidate instructions from output demonstrations.
2. Execute candidates with a target model.
3. Select the best instruction based on evaluation scores.

APE discovered that “Let’s work this out in a step by step way to be sure we have the right answer” outperforms the human-engineered “Let’s think step by step”.

Related approaches: Prompt-OIRL, OPRO, AutoPrompt, Prefix Tuning, Prompt Tuning.

8. Agentic & Tool-Augmented Techniques

8.1 ReAct (Reason + Act)

Interleaves reasoning traces (Thought) and task-specific actions (Act):

• Thought: Plans, tracks progress, handles exceptions.
• Act: Interfaces with external tools (search, APIs, code execution).
• Observation: Result from the action fed back into the reasoning loop.

Pattern: Thought → Act → Observation → Thought → ...

• Outperforms CoT alone on tasks requiring external facts.
• Best results come from combining ReAct with CoT + Self-Consistency.
• Implementations available in LangChain.

8.2 ART (Automatic Reasoning and Tool-use)

Combines CoT with tool use automatically:

1. Select multi-step reasoning + tool-use demonstrations from a task library.
2. At test time, pause generation when external tools are called; integrate their output; resume.
3. Humans can fix mistakes or add new tools by updating libraries.

Enables zero-shot generalization to new tasks with automatic decomposition.

8.3 PAL (Program-Aided Language Models)

Instead of free-text reasoning, the LLM generates executable programs (e.g., Python) as intermediate steps.

• Offloads computation to a programmatic runtime.
• More accurate for arithmetic and structured logic than free-form CoT.
• Can be combined with LangChain for date understanding, math, etc.

8.4 Reflexion

Reinforces agents through linguistic (verbal) feedback:

• Actor: Generates text/actions based on state observations (uses CoT/ReAct).
• Evaluator: Scores the Actor’s output (reward signal).
• Self-Reflection: Generates verbal feedback about mistakes; stores in long-term memory.

Loop: Task → Trajectory → Evaluate → Reflect → Next Trajectory (improved).

• Achieves SOTA on HumanEval, MBPP, and AlfWorld tasks.
• Lightweight alternative to traditional RL (no fine-tuning required).

9. Risks & Misuses

9.1 Prompt Injection

Untrusted user input concatenated with trusted instructions hijacks model behavior.

• Example: User input overrides the system instruction.
• Defense: Add warnings in instructions, parameterize prompt components, quote/escape inputs, use adversarial prompt detectors, or use fine-tuned non-instruction models.

9.2 Prompt Leaking

Attacks designed to extract confidential or proprietary prompt content (exemplars, system instructions).

• Mitigation: Avoid embedding sensitive IP in prompts; test for leak vulnerabilities.

9.3 Jailbreaking

Bypassing safety guardrails to elicit harmful or policy-violating outputs.

Modern models (ChatGPT, Claude) have guardrails, but new jailbreaks continue to emerge.

9.4 Defense Tactics Summary

• Instruction hardening: Warn the model about potential attacks.
• Input parameterization: Separate instructions from inputs (like SQL parameterization).
• Quoting/escaping: Format user inputs to prevent interpretation as instructions.
• Adversarial detector: Use a separate LLM to flag malicious prompts.
• Model choice: Fine-tuned models or k-shot non-instruction models are harder to inject.

9.5 Factuality & Hallucinations

LLMs can generate plausible but false information.

• Mitigations: Use RAG for grounding, ask the model to cite sources, provide context, and verify outputs.

10. Model-Specific Notes (ChatGPT)

• Model: gpt-3.5-turbo / gpt-4 — optimized for chat completions.
• Format: Series of messages with system, user, and assistant roles.
• Cost: ChatGPT API is ~90% cheaper than text-davinci-003.
• Recommendation: For gpt-3.5-turbo-0301, place instructions in the user message rather than system.
• Snapshots: Dated model versions are available for reproducibility.
• Can perform all standard tasks (QA, summarization, classification) via the chat format.

11. Tools & Libraries

Popular tools for prompt engineering and LLM application development:

Quick Reference Cheat Sheet

Source: Condensed from Prompt Engineering Guide by DAIR.AI