AI Prompt Engineering Demystified: Tokens, Context Windows, and Avoiding Hallucinations

The hidden power of LLMs lie not in their code, but in the way you speak to them.

When you type a question or instruction into an AI like ChatGPT, you’re not just “chatting.” You’re constructing a prompt—a carefully chosen set of words that determine the quality of the response. This process is called prompt engineering, and learning it is like learning to command a new digital language.

Most people still treat AI prompts like Google searches: a quick string of words and nothing more. But unlike a search engine, an AI doesn’t fetch facts from a database—it predicts words, sentence by sentence, based on probabilities. That’s why your input matters so much.

In this post, we’ll break down the essentials of prompt engineering, explain what tokens and context windows really mean, and help you recognize when the AI starts to “hallucinate.”

Prompt Engineering

The Basics of Prompt Engineering

Think of prompt engineering as designing a conversation with AI. Just like in real life, the way you ask matters as much as what you ask. Here are some proven techniques:

1. Role Assignment

   • Example: “You are a financial advisor. Explain to me how compound interest works as if I’m 12 years old.”

   • Why it works: By giving the AI a role, you anchor its response style.

     
     

2. Step-by-Step Instruction

   • Example: “Explain how to cook pasta step by step, starting from boiling water.”

   • Why it works: AI thrives when you break tasks into ordered steps.

     
     

3. Providing Context

   • Example: “I am preparing a presentation for high school students on climate change. Write me a fun introduction.”

   • Why it works: The more background you provide, the more tailored the output.

     
     

4. Few-Shot Prompting

   • Example:

     Q: Translate 'Good Morning' to Spanish.

     A: Buenos Días.

     Q: Translate 'How are you?' to Spanish.

     A:

   • Why it works: Giving examples sets the pattern the AI should follow.

     
     

5. Constraints and Style Guides

   • Example: “Summarize this article in under 100 words, and use bullet points only.”

   • Why it works: AI loves boundaries—they sharpen the output.

     
     

6. Chain-of-Thought Prompting

   • Example: “Think step by step before answering: If a car travels 60 km in 2 hours, what is its average speed?”

   • Why it works: Asking the model to reason out loud often improves accuracy.

What Are Tokens?

AI doesn’t read text the way we do. It breaks every word, number, or even piece of a word into tokens.

• The word “computer” = 1 token.

• The word “unbelievable” might split into “un”, “believe”, “able” = 3 tokens.

• A token is roughly 3–4 characters of English text.

  
  

Why does this matter?

Because tokens are the AI’s currency. Every model has a maximum token limit—shared between your input and the model’s output. Go beyond the limit, and your prompt gets cut off.

The Context Window Explained

The context window is like the AI’s memory span. It’s the maximum number of tokens it can “see” at once.

• If a model has an 8,000-token context window, that means your prompt plus the AI’s answer can’t exceed 8,000 tokens.

• Once you exceed the limit, older parts of the conversation fall off the edge of memory.

  
  

This explains why sometimes the AI “forgets” earlier parts of a long conversation. It’s not ignoring you—it simply can’t remember beyond the window.

Practical Tip: Be concise. Trim unnecessary details. Use summaries instead of pasting full documents. Every token saved is a token earned for better answers.

Tying It All Together: Prompts + Tokens + Context

Think of it this way:

• Prompt engineering = how you phrase the question.

• Tokens = the raw units of cost and memory.

• Context window = the model’s working memory limit.

  
  

To get the most out of AI:

• Write clear, structured prompts.

• Watch your token usage (especially if you’re working with large documents).

• Be mindful of the model’s context window. Don’t overload it with fluff.

The Hallucination Problem

AI can sound confident while being completely wrong. This is called hallucination—when the model fabricates information that doesn’t exist.

• Example: Asking for a biography of a little-known person, and the AI invents achievements or fake citations.

• Why it happens: Remember, AI predicts words based on patterns. If it lacks data, it “fills in the blanks” with its best guess.

How to Spot It:

• If an answer seems oddly specific, double-check it.

• Always verify facts with a reliable source.

• Use constraints: “If you don’t know, say you don’t know.”

Final Thoughts

Prompting isn’t just typing—it’s an art. Tokens aren’t just technical—they’re the building blocks of communication with AI. And the context window isn’t just a number—it’s the boundary of the AI’s memory.

Learn these three concepts together, and you’ll unlock the true potential of AI while steering clear of hallucinations.

AI is powerful, but it’s also literal. The better you learn to speak its language, the more value you’ll get in return.