Chapter 7: Prompt Engineering Case Studies

Theory is essential, but seeing principles in action solidifies understanding. This chapter presents practical case studies demonstrating how prompt engineering techniques (from core principles to advanced strategies) are applied to solve real-world problems across different domains. Each case study follows a similar structure:

• Scenario: The context and the problem to be solved.
• Objective: The desired outcome from using AI.
• Initial Prompt Idea: A first attempt at prompting.
• Challenges & Refinements: Issues encountered and how the prompt was improved using techniques discussed earlier.
• Final Prompt Strategy: The more effective prompt structure.
• Outcome: The results achieved.

Case Study 1: Customer Service Chatbot for Order Tracking

Scenario: An e-commerce company wants to automate responses to common customer inquiries about order status using an LLM-powered chatbot integrated into their website chat.

Objective: Provide accurate, real-time order status updates, reduce human agent workload for simple queries, and maintain a friendly, helpful brand voice.

Initial Prompt Idea:

User: Where is my order? AI: Please provide your order number. User: 12345XYZ AI: [Looks up order 12345XYZ] Your order status is [Status].

(This relies heavily on external logic to interpret “Where is my order?” and handle the lookup).

Challenges & Refinements:

Ambiguity: “Where is my order?” is vague. The bot needs the order number.

Integration: The LLM itself can’t directly query the order database. It needs to know when to ask for the number and how to request the lookup.

Tone: The initial interaction is purely functional, lacking brand voice.

Solution: Implement Function Calling (Chapter 4) and use a System Message (Chapter 4) for persona and instructions.

Final Prompt Strategy:

System Message:

You are 'OrderBot', a friendly and efficient customer service assistant for 'MegaMart'. Your primary goal is to help customers track their orders. Be polite and ask for the order number if the user asks about their order status without providing one. You have access to a function called 'getOrderStatus'. Only use this function when you have an order number. If the user asks about anything other than order status, politely state that you can only help with order tracking.

Function Definition (Provided to LLM):

{
  "name": "getOrderStatus",
  "description": "Gets the current status and estimated delivery date for a given order number.",
  "parameters": {
    "type": "object",
    "properties": {
      "orderNumber": {
        "type": "string",
        "description": "The unique identifier for the customer's order (e.g., '12345XYZ')."
      }
    },
    "required": ["orderNumber"]
  }
}

Interaction Flow:

• User: Hi, I wanted to check on my recent order.
• AI (Guided by System Message): Hello! I can certainly help you with that. Could you please provide your order number?
• User: It's 67890ABC.
• AI (Decides to use function): Generates JSON: {"function": "getOrderStatus", "arguments": {"orderNumber": "67890ABC"}}
• (Application executes getOrderStatus('67890ABC') and gets back: {"status": "Shipped", "estimated_delivery": "May 5th, 2025"})
• AI (Receives function result): Great news! Your order 67890ABC has shipped and is estimated to arrive by May 5th, 2025. Let me know if there's anything else!

%%{ init: { 'theme': 'base', 'themeVariables': { 'primaryColor': '#EDE9FE', 'primaryTextColor': '#5B21B6', 'lineColor': '#A78BFA', 'textColor': '#1F2937', 'fontSize': '18px' }}}%%
graph TD
    A[User: 'Check my order'] --> B(AI: Guided by System Message);
    B --> C{Order Number Provided?};
    C -- No --> D[AI: 'Please provide order number'];
    D --> E[User: Provides Order Number];
    C -- Yes --> E;
    E --> F(AI: Decides to call 'getOrderStatus');
    F --> G["App: Executes getOrderStatus(Number)"];
    G --> H{"Receives Result<br/><i>(e.g., Shipped, May 5th)</i>"};
    H --> I(AI: Formulates response using result);
    I --> J[AI: 'Your order 123 has shipped...'];
    J --> K(End Interaction);

    style A fill:#F3F4F6,stroke:#6B7280
    style E fill:#F3F4F6,stroke:#6B7280
    style J fill:#DBEAFE,stroke:#2563EB
    style D fill:#DBEAFE,stroke:#2563EB
    style B fill:#EDE9FE,stroke:#5B21B6
    style F fill:#EDE9FE,stroke:#5B21B6
    style I fill:#EDE9FE,stroke:#5B21B6
    style G fill:#FEF3C7,stroke:#D97706
    style H fill:#FEF3C7,stroke:#D97706
    style C fill:#FEE2E2,stroke:#DC2626
    style K fill:#D1FAE5,stroke:#059669

Outcome: The chatbot handles ~60% of order status inquiries automatically. Human agents are freed up for complex issues. Customer satisfaction with chat support improves due to instant responses. The system message ensures consistent tone and behavior, while function calling enables real-time data access.

Case Study 2: Code Generation Assistant for Data Scientists

Scenario: A team of data scientists frequently needs to write repetitive Python code for data visualization using libraries like Matplotlib and Seaborn. They want an AI assistant to speed up this process.

Objective: Generate accurate, customizable Python code for common plot types based on natural language descriptions, adhering to team coding standards (e.g., including comments, type hints).

Initial Prompt Idea:

<code>Make a scatter plot in Python showing the relationship between column 'A' and column 'B' from my dataframe 'df'.</code>

Challenges & Refinements:

Lack of Specificity: Doesn’t specify libraries, labels, title, or customization options. Output might be basic or use an undesired library.

No Standards: Doesn’t enforce comments, type hints, or function structure.

Context: Assumes the AI knows about a dataframe named ‘df’.

Solution: Use a more structured prompt defining the role, task, required libraries, input data structure (conceptually), desired output format, and team standards. Leverage Few-Shot examples (Chapter 3) implicitly or explicitly for style.

Final Prompt Strategy:

### Role
You are an expert Python data visualization assistant. You generate clean, well-commented code using Matplotlib and Seaborn.

### Context
Assume the user has a pandas DataFrame named `data_df` with columns specified in the task.
Code should include:
- Necessary library imports.
- Clear comments explaining the code.
- Type hints for function arguments and return values (if generating a function).
- Axis labels and a descriptive title for the plot.

### Task
Generate Python code to create a scatter plot showing the relationship between the 'AnnualIncome' (x-axis) and 'SpendingScore' (y-axis) columns in the `data_df`.
- Set the title to 'Income vs. Spending Score'.
- Label the x-axis 'Annual Income ($)'.
- Label the y-axis 'Spending Score (1-100)'.
- Use Seaborn for plotting.
- Display the plot.

### Output Format
Provide only the complete, runnable Python code block.

Outcome: Data scientists save significant time on routine plotting tasks. The generated code is more consistent with team standards, reducing review time. The assistant helps newer team members learn library usage and best practices through generated examples.

Case Study 3: Image Generation for Marketing Campaign Concepts

Scenario: A marketing agency needs to quickly generate diverse visual concepts for a new eco-friendly soda brand campaign. They are using an image generation model like Midjourney or Stable Diffusion.

Objective: Create visually appealing images showcasing the product in natural, vibrant settings, conveying freshness and sustainability, with a consistent artistic style.

Initial Prompt Idea:

A can of soda in nature.

Challenges & Refinements:

Too Vague: Leads to generic, uninspired images. Doesn’t specify soda appearance, setting details, mood, or style.

Inconsistent Style: Different generations might look completely unrelated.

Lack of Brand Elements: Doesn’t incorporate brand identity (e.g., colors, feeling).

Solution: Develop a detailed base prompt incorporating keywords for the subject, setting, mood, style, and technical aspects. Use negative prompts and platform parameters. Iterate by adding/modifying keywords.

Final Prompt Strategy (Example for Midjourney):

Vibrant product shot of a sleek aluminum can labeled "Evergreen Soda" condensation glistening, nestled amongst lush green moss and ferns in a sun-dappled forest clearing, sparkling stream nearby, soft natural morning light, earthy color palette with pops of bright green, photorealistic style, high detail, shot on Canon EOS R5 --ar 16:9 --style raw --no plastic, litter, people

Breakdown:

Subject: Vibrant product shot of a sleek aluminum can labeled "Evergreen Soda" condensation glistening

Setting: nestled amongst lush green moss and ferns in a sun-dappled forest clearing, sparkling stream nearby

Lighting/Mood: soft natural morning light

Color/Style: earthy color palette with pops of bright green, photorealistic style, high detail

Technical: shot on Canon EOS R5 (simulates camera type)

Parameters: --ar 16:9 (aspect ratio), --style raw (less stylized Midjourney look)

Negative Prompt: --no plastic, litter, people (things to avoid)

Iteration: The team might then create variations by changing keywords: swapping forest clearing for mountain meadow, changing morning light to golden hour, or trying watercolor illustration instead of photorealistic style.

Outcome: The agency rapidly generates a wide range of high-quality, on-brand visual concepts for client review, significantly speeding up the creative ideation phase. They build a library of effective prompt components (keywords for lighting, style, setting) for future campaigns.

Case Study 4: Summarizing Research Papers using RAG

Scenario: A medical researcher needs to quickly understand the key findings from dozens of lengthy research papers on a specific topic.

Objective: Generate concise, accurate summaries highlighting the objectives, methods, key results, and conclusions of each paper, grounded in the paper’s actual content.

Initial Prompt Idea:

Summarize the main findings of the attached research paper about [Topic].

(Assumes the model has access to the full paper, which standard models don’t unless fine-tuned or using specific integrations).

Challenges & Refinements:

Context Limit: Full research papers often exceed model context windows.

Hallucination Risk: Without the actual text, the model might hallucinate findings based on the title or abstract alone.

Lack of Structure: A simple summary might miss key sections like methodology or specific results.

Solution: Implement a RAG system. The system retrieves relevant sections (e.g., Abstract, Introduction, Results, Conclusion) from the paper’s text (stored in a vector database or similar). The prompt then instructs the LLM to synthesize a structured summary based only on these retrieved sections.

Final Prompt Strategy (Used in the Generation step of RAG):

### Role
You are a research assistant specializing in summarizing medical studies.

### Provided Context from Research Paper "[Paper Title]"
--- BEGIN RETRIEVED SECTIONS ---
[Text snippet from Abstract]
...
[Text snippet from Introduction/Objectives]
...
[Text snippet from Methods section]
...
[Key text snippets from Results section]
...
[Text snippet from Conclusion section]
--- END RETRIEVED SECTIONS ---

### Task
Based *strictly* on the provided context sections from the research paper:
1. State the main objective(s) of the study.
2. Briefly describe the primary methods used.
3. List the 2-3 most significant results reported.
4. State the main conclusion(s) drawn by the authors.

### Output Format
Respond using clear headings for each of the four points above (Objective, Methods, Results, Conclusion). Be concise and accurate according to the provided text. If information for a point is missing in the context, state "Information not found in provided context".

%%{ init: { 'theme': 'base', 'themeVariables': { 'primaryColor': '#EDE9FE', 'primaryTextColor': '#5B21B6', 'lineColor': '#A78BFA', 'textColor': '#1F2937', 'fontSize': '18px' }}}%%
graph TD
    A[User: Request Summary for Paper X] --> B(RAG System);
    B --> C["1- Retrieve Relevant Sections<br/><i>(Abstract, Intro, Methods, Results, Conclusion)</i><br/>from Paper X Vector Store/DB"];
    C --> D{Retrieved Context Chunks};
    A --> E(LLM Prompt Construction);
    D --> E;
    E --> F["2- Combine: <br/> - User Request<br/> - Instructions (Role, Task, Format)<br/> - Retrieved Context Chunks"];
    F --> G(LLM Generation);
    G --> H[3- Generate Structured Summary<br/><b>Based ONLY on Provided Context</b>];
    H --> I["Output: Structured Summary<br/><i>(Objective, Methods, Results, Conclusion)</i>"];
    I --> J(User);

    style A fill:#F3F4F6,stroke:#6B7280
    style J fill:#F3F4F6,stroke:#6B7280
    style B fill:#E0E7FF,stroke:#4338CA
    style C fill:#FEF3C7,stroke:#D97706
    style D fill:#FEF3C7,stroke:#D97706
    style E fill:#EDE9FE,stroke:#5B21B6
    style F fill:#EDE9FE,stroke:#5B21B6
    style G fill:#EDE9FE,stroke:#5B21B6
    style H fill:#DBEAFE,stroke:#2563EB
    style I fill:#D1FAE5,stroke:#059669

Outcome: The researcher can process papers much faster. The RAG approach ensures summaries are grounded in the actual paper content, significantly reducing hallucinations. The structured output makes it easy to quickly grasp the essential information from each study.

Summary

These case studies illustrate the practical power of applying core and advanced prompt engineering techniques. Success often comes from:

• Clearly defining the objective and desired output.
• Starting simple and iteratively refining the prompt based on observed outputs.
• Choosing the right techniques for the task (e.g., Function Calling for external data, RAG for grounding, detailed descriptions for images, structure for code).
• Understanding and mitigating potential challenges like ambiguity, lack of context, or unwanted model behaviors.

By analyzing these examples, you can gain insights into how to approach your own prompt engineering challenges across various domains.

In the next chapter of the course, we will delve into Evaluation and Iteration, formalizing the process of testing and improving your prompts systematically.

External Sources:

• Prompt Engineering Guide – Applications (Examples): https://www.promptingguide.ai/applications
• LangChain – Use cases: https://python.langchain.com/v0.2/docs/use_cases/
• OpenAI – Customer Stories: https://openai.com/customers/