Key AI Concepts (3)#

In this chapter, we cover prompt engineering (how to structure questions and instructions) and context engineering (how to provide the background materials the model should use to produce an answer).

Prompt Engineering#

A prompt is the text you give to an AI model as a query (a question or instruction). For example, in ChatGPT, whatever you type into the input box is the prompt. Prompt engineering is the skill of writing prompts in a way that reliably produces the result you want. It may feel like “just typing in a chat box,” but the quality of the output can change dramatically depending on how you phrase and structure the input—so people call it “engineering” because there are principles, trade-offs, and know-how.

Prompt engineering is a practical skill that most AI service users need directly, so below we’ll walk through concrete strategies based on OpenAI’s public documentation.

Write clear instructions#

Be specific about the context and the exact information you need. Instead of “How do I add numbers in Excel?”, it’s better to write something like: “How do I sum a column of dollar amounts in Excel? I want to automatically sum values from all sheets to the left of the ‘Total’ column and apply them to the ‘Total’ column.”

It’s often helpful to assign the model a persona/role. For example: “You are a Python programmer. Given code, add comments and explain it…”

If your input contains multiple parts, clearly separate them using triple quotes or XML tags so the model can tell what’s what.

Summarize the text delimited by triple quotes.
"""
Put the target text here.
"""

You are given a pair of articles on the same topic (delimited by XML tags).
First, summarize the claims of each article.
Then indicate which one is more convincing and explain why.

<article>Article 1 content</article>
<article>Article 2 content</article>

Sometimes it helps to break instructions into explicit steps.

Follow the step-by-step instructions below for the text delimited by triple quotes.
Step 1 - Prefix with “Summary:” and write a one-sentence summary.
Step 2 - Prefix with “Translation:” and translate the summary.
"""
Put the text here.
"""

If it’s hard to describe requirements precisely, you can provide an example and ask the model to respond in the same style.

Answer the question in the same style as the example.
<example>
Q: What is patience?
A: The river that carves the deepest valley begins from a modest spring.
   The most magnificent symphony begins with a single note.
   The most intricate tapestry begins with a single thread.
</example>
<question>What is the sea?</question>

It’s also useful to explicitly specify the desired output length. For example: “Summarize this in one sentence.”, “Suggest 3 titles for this.”, “Write this in about 500 characters.”

Provide reference text#

You can provide reference materials and instruct the model to use them when generating an answer.

To answer the question, refer to the article content delimited by triple quotes.
If you can’t find the answer in the article, write: “Answer not found.”
"""
Article content
"""
Question: (your question)

You are given a document and a question delimited by triple quotes.
Your task is to answer the question using only the provided document, and cite the exact passage used.
If the document does not contain the information needed, write: “Insufficient information.”
If you provide an answer, include citations.
Use the following format to cite relevant passages: ({"citation": …})
"""
Document content
"""
Question: (your question)

Split complex tasks into simpler subtasks#

You can first categorize the user’s request, then route it into a more specific prompt for that category.

Given a customer inquiry, classify it into a primary and a secondary category.
Output JSON with keys "primary" and "secondary".

Primary categories: Billing, Technical Support, Account Management, General Inquiry

Secondary categories for Billing:
- Cancel subscription or upgrade
- Add payment method
- Explain charges
- Dispute billing

Secondary categories for Technical Support:
- Troubleshooting
- Device compatibility
- Software updates

Secondary categories for Account Management:
- Reset password
- Update personal information
- Delete account
- Account security

Secondary categories for General Inquiry:
- Product information
- Pricing
- Feedback
- Talk to an agent

Customer inquiry: I can’t connect to the internet.

A customer inquiry is provided that requires troubleshooting under the Technical Support category.
Help the user as follows:
- Ask them to check whether all cables between the router and modem are connected. Cables commonly loosen over time.
- If all cables are connected but the issue persists, ask which router model they are using.
- Next, instruct them how to restart the device:
  - If the model number is MTD-327J, instruct them to press the red button for 5 seconds, then wait about 5 minutes and test the connection.
  - If the model number is MTD-327S, instruct them to unplug and replug it, then wait about 5 minutes and test the connection.
- If the issue persists after restarting and waiting 5 minutes, output {"IT support requested"} to route to IT support.
- If the user starts asking unrelated questions, check whether they want to end the current troubleshooting chat, then classify the new request using the schema below.
<insert the primary/secondary category definitions above here>

Customer inquiry: I can’t connect to the internet.

In chat-based services like ChatGPT, previous turns in the chat are included as part of the next request, so the model can understand the ongoing context. But if the conversation becomes too long or includes too much irrelevant content, the model may lose the thread. In those cases, it helps to summarize prior messages and include the summary in the prompt.

When summarizing a long document such as a book, it’s often more effective to summarize section by section first, then iteratively summarize the summaries.

Give the model time to “think”#

When the model needs to judge whether a solution is correct, it can be better to ask it to derive its own solution first, then compare it to the provided solution—rather than immediately asking “Is this correct?”

Please determine whether the student’s solution is correct.

Problem statement: You are building a solar power farm and need help with the finances.
- Land costs $100 per square foot.
- You can buy solar panels for $250 per square foot.
- You negotiated a maintenance contract with a fixed annual cost of $100,000 plus $10 per square foot.
What is the total cost for the first year of operation as a function of the number of square feet?

Student’s solution: Let x be the area in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 10x
Total cost: 100x + 250x + 100,000 + 10x = 360x + 100,000

Rather than directly asking whether it’s correct, you can ask the model to first find its own solution and then compare: “First, work out your own solution to the problem. Then compare your solution with the student’s and evaluate whether the student is correct. Do not decide whether the student’s solution is correct until you have solved the problem yourself.”

Use external tools#

As mentioned in the explanation of RAG, you can instruct the model to augment its input using external data. For example, you can provide a specific URL and ask the model to answer based on it, or provide keywords and ask it to summarize search results. In ChatGPT, you can also create custom GPTs (GPTs) or find and use public GPTs that match your purpose. (For tool integrations/search/file-based Q&A via APIs, see OpenAI’s Tools and File Search guides for more details.)

The prompt engineering strategies above are based on OpenAI’s public documentation, so they may not apply identically to every AI model. Different models/services can require different input formats or parameters.

Also, in production, it’s important not only to “write good prompts,” but also to manage the fact that outputs can change as models are updated. For example, a model that produced nearly identical formatting yesterday might respond with a slightly different style or structure after an update (from a service perspective, it can feel like behavior suddenly changed).

That’s why, in production, it’s often recommended to pin a specific model snapshot (version) for predictability, and when changing prompts or switching models, to validate key cases via evals (a prepared “test question set”). (This is primarily a concern for developers running services via OpenAI APIs and similar. For general users, it’s usually enough to know that updates can change results and to double-check important outputs.)

Chat-based AI services like ChatGPT and Google Gemini often share these strategies broadly, and understanding them can help you write prompts for other models too.

Context Engineering#

If prompt engineering is about designing “what to ask the model to do” (instructions/format), context engineering is about designing “what materials (background information) the model should use to produce an answer, and how to provide them.” Even with the same prompt, the accuracy and consistency of the answer can change significantly depending on what context you include.

The key is not “stuffing in a lot of text,” but including the right information in the right amount, and structuring it so the model doesn’t get confused.

What can be included in context#

• Conversation context: previous messages (or a summary when needed)
• Work/domain materials: internal policies, manuals, guides, FAQs
• User/work state: user settings, current step, entered values (minimize personal/sensitive data)
• Tool execution results: outputs from function calling (queries, calculations, system responses)
• Goals/constraints: required rules, forbidden actions, output format

Tips for strong context construction#

• Prioritize and organize: put the most important items first, and make it clear what is “evidence”
• Manage length: overly long context can hide the essentials and quickly consume the context window
• Add clear boundaries: Markdown headers or XML-style tags help separate sections
• Check freshness and consistency: mixing outdated policies can reduce quality
• Force evidence-based answers: rules like “Answer using only the documents below; if missing, say you don’t know” can reduce hallucinations

RAG and function calling are both forms of context engineering#

RAG is a way to “automatically retrieve the right documents and attach them as context.” Including documents directly in the prompt is also a form of context engineering. And using function calling to perform DB lookups / calculations / API calls, then providing the results as context for the final answer, is also an important part of context engineering. In other words, context engineering is a broader concept that includes both RAG and “using tool outputs as context.”