Assignments Overview

HW 1 – Prompt Engineering - Test 1

Overview

The purpose of this assignment is to introduce you to prompt engineering, a crucial skill when working with AI models, especially large language models (LLMs). By understanding and practicing prompt engineering, you’ll learn how to effectively communicate with AI systems to obtain accurate and relevant outputs. This assignment will deepen your understanding of how AI models interpret inputs and how you can influence their behavior to solve business problems.

Learning Objectives

• Understand the principles of prompt engineering.
• Develop skills to create and refine prompts for AI models.
• Apply prompt engineering to solve a realistic business problem.
• Evaluate the quality and effectiveness of AI-generated responses.

Business Scenario

Imagine you are working for a company that provides customer support services. The company has recently integrated an AI chatbot to assist with answering common customer inquiries. However, the chatbot’s responses are not always accurate or helpful. Your task is to improve the chatbot’s performance by designing effective prompts that guide it to generate better responses.

Instructions

Step 1: Set Up Your Environment

Ensure you have access to the OpenAI GPT model. You may use the OpenAI API or any other platform that provides access to a similar LLM. Make sure your Python environment is set up with necessary packages such as openai and pandas.

Step 2: Design Initial Prompts

Start by designing a set of initial prompts to test the AI’s response to common customer inquiries. Use the following sample inquiries:

1. “What are your store hours?”
2. “How can I reset my password?”
3. “What is your return policy?”

import openai

# Replace 'your-api-key' with your actual OpenAI API key
openai.api_key = 'your-api-key'

def get_response(prompt):
    response = openai.Completion.create(
        engine="text-davinci-003", 
        prompt=prompt, 
        max_tokens=50
    )
    return response.choices[0].text.strip()

# Sample prompts
prompts = [
    "What are your store hours?",
    "How can I reset my password?",
    "What is your return policy?"
]

# Fetch responses
responses = [get_response(prompt) for prompt in prompts]
for i, response in enumerate(responses):
    print(f"Prompt: {prompts[i]}\nResponse: {response}\n")

Step 3: Analyze and Refine Prompts

Evaluate the responses generated by the initial prompts. Consider the following:

• Are the responses accurate and complete?
• Do they address the customer’s inquiry effectively?

Refine your prompts based on this analysis. For example, you might add context or specify the format of the response.

# Refined prompts
refined_prompts = [
    "Please provide detailed store hours for each day of the week.",
    "Guide a user through the steps to reset their password.",
    "Explain the return policy clearly, including any exceptions."
]

# Fetch refined responses
refined_responses = [get_response(prompt) for prompt in refined_prompts]
for i, response in enumerate(refined_responses):
    print(f"Refined Prompt: {refined_prompts[i]}\nResponse: {response}\n")

Step 4: Document Your Findings

Prepare a report documenting your process. Include:

• The initial prompts and their responses.
• The refined prompts and their responses.
• An analysis of the differences in responses.
• Insights gained about prompt engineering.

Deliverables

Submit a report (PDF or Word document) containing:

1. Initial prompts and responses.
2. Refined prompts and responses.
3. Analysis of the effectiveness of each set of prompts.
4. Insights and lessons learned from the exercise.

Grading Rubric (40 Points Total)

• Initial Prompts and Responses (10 points): Clarity and relevance of initial prompts and analysis of their responses.
• Refined Prompts and Responses (10 points): Quality and improvement of refined prompts and analysis of their responses.
• Analysis and Insights (15 points): Depth of analysis and understanding of prompt engineering principles.
• Presentation and Clarity (5 points): Overall clarity, organization, and presentation of the report.

HW 2 – Retrieval-Augmented Generation

HW 2 – Retrieval-Augmented Generation

Overview

The purpose of this assignment is to deepen your understanding of the concept of Retrieval-Augmented Generation (RAG) and its practical applications in business contexts. By the end of this assignment, you will be able to:

• Understand the basic principles of RAG.
• Implement a simple RAG pipeline using Python.
• Analyze the business value of RAG in providing contextually relevant information.
• Evaluate the effectiveness of RAG solutions in a business scenario.

Business Scenario

Continuing from the prior homework, imagine you are working for a company that provides customer support via an AI chatbot. The chatbot is designed to answer customer queries by accessing a large database of company documentation and generating relevant responses. However, the current system struggles to provide accurate and contextually relevant answers quickly.

To enhance the chatbot’s performance, your team has decided to implement a Retrieval-Augmented Generation approach. By retrieving the most relevant documents and using them to generate precise responses, the chatbot will be able to provide more accurate and helpful information to customers, improving their overall experience and satisfaction.

Instructions

Step 1: Setup and Data

1. Install Required Libraries: Ensure you have the necessary Python libraries installed. You will need transformers, torch, and faiss-cpu. You can install them using pip:

   pip install transformers torch faiss-cpu

2. Download or Generate Synthetic Data: Use the following code to generate a synthetic dataset of FAQs and answers. This will simulate the company documentation.

   ::: {#ce8ae573 .cell execution_count=3} ``` {.python .cell-code} import pandas as pd

   data = { “question”: [ “How can I reset my password?”, “What is the refund policy?”, “How do I update my billing information?”, “Where can I download the mobile app?”, “What are the customer support hours?” ], “answer”: [ “To reset your password, go to the login page and click on ‘Forgot Password’.”, “Our refund policy allows returns within 30 days of purchase with a receipt.”, “To update billing information, navigate to ‘Account Settings’ and select ‘Billing’.”, “The mobile app can be downloaded from the App Store or Google Play.”, “Customer support is available from 9 AM to 5 PM, Monday to Friday.” ] }

   df = pd.DataFrame(data) df.to_csv(“synthetic_faq.csv”, index=False) ``` :::

Step 2: Implement the RAG Pipeline

1. Load Data: Load the synthetic FAQ data into your Python environment.

   ::: {#8639754c .cell execution_count=4} {.python .cell-code} df = pd.read_csv("synthetic_faq.csv") :::

2. Create a Simple Retrieval System: Use FAISS to index the FAQ questions for fast retrieval.

   ::: {#ef769245 .cell execution_count=5} ``` {.python .cell-code} import faiss from sklearn.feature_extraction.text import TfidfVectorizer

   vectorizer = TfidfVectorizer() X = vectorizer.fit_transform(df[‘question’])

   index = faiss.IndexFlatL2(X.shape[1]) index.add(X.toarray()) ``` :::

3. Implement a Generation Model: Use a pre-trained language model from transformers to generate answers.

   ::: {#827caf52 .cell execution_count=6} ``` {.python .cell-code} from transformers import pipeline

   generator = pipeline(“text-generation”, model=“gpt2”) ``` :::

4. Combine Retrieval and Generation: Write a function to retrieve the most relevant FAQ and generate a response.

   ::: {#499a4c1b .cell execution_count=7} ``` {.python .cell-code} import numpy as np

   def retrieve_and_generate(query): query_vec = vectorizer.transform([query]).toarray() D, I = index.search(query_vec, k=1) retrieved_answer = df.iloc[I[0][0]][‘answer’]

    # Generate response based on retrieved answer
    generated_response = generator(retrieved_answer, max_length=50, num_return_sequences=1)
    return generated_response[0]['generated_text']

   # Example usage query = “How do I change my password?” response = retrieve_and_generate(query) print(response) ``` :::

Step 3: Evaluation

• Test the RAG system with at least 3 different queries.
• Analyze the relevance and accuracy of the generated responses.

Deliverables

1. A Python script implementing the RAG pipeline.
2. A short report (1-2 pages) discussing:
   • The effectiveness of the RAG approach.
   • Potential improvements for the system.
   • The business implications of using RAG in the given scenario.

Grading Rubric (40 Points Total)

• Implementation of RAG Pipeline (20 points):
  • Correct setup and indexing of the retrieval system (10 points).
  • Successful integration of the generation model (10 points).
• Evaluation and Analysis (10 points):
  • Quality and relevance of generated responses (5 points).
  • Depth of analysis in the report (5 points).
• Report Quality (10 points):
  • Clarity and organization of the report (5 points).
  • Insight into business implications and potential improvements (5 points).

Ensure that your submission is comprehensive and demonstrates a clear understanding of Retrieval-Augmented Generation and its business applications.

HW 3 – Fine-Tuning

HW 3 – Fine-Tuning

Overview

In this assignment, you will learn how to fine-tune a pre-trained AI model to adapt it to a specific business context. Fine-tuning is a critical step in making AI models more effective for particular applications by leveraging existing models and adjusting them to fit new data. The main objectives of this assignment are to:

• Understand the concept of fine-tuning in the context of AI models.
• Apply fine-tuning techniques to adapt a model to a specific business problem.
• Evaluate the performance of the fine-tuned model.

Business Scenario

Imagine you are working for a retail company that wants to improve its customer service by deploying an AI chatbot to handle customer inquiries. The company has decided to use a pre-trained language model but needs to fine-tune it with its specific customer service data to ensure the chatbot understands and responds accurately to customer queries.

Instructions

1. Data Preparation

   • Download the synthetic customer service dataset from the following link. This dataset contains pairs of customer inquiries and appropriate responses.
   • Load the dataset into your Python environment using pandas.

   ::: {#9bc0a75d .cell execution_count=8} ``` {.python .cell-code} import pandas as pd

   data = pd.read_csv(‘synthetic-customer-service-data.csv’) print(data.head()) ``` :::

2. Model Selection

   • Choose a pre-trained language model from the Hugging Face Transformers library, such as distilbert-base-uncased.
   • Install the transformers library if you haven’t already:

   pip install transformers

3. Fine-Tuning the Model

   • Use the Trainer API from the Transformers library to fine-tune the model on the customer service dataset. You will need to tokenize the text data and define a training loop.
   • Example code snippet for fine-tuning:

   ::: {#6b5aa7c0 .cell execution_count=9} ``` {.python .cell-code} from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments, DistilBertTokenizer

   tokenizer = DistilBertTokenizer.from_pretrained(‘distilbert-base-uncased’) model = DistilBertForSequenceClassification.from_pretrained(‘distilbert-base-uncased’)

   # Tokenize the dataset def tokenize_function(examples): return tokenizer(examples[‘inquiry’], truncation=True, padding=True)

   tokenized_data = data.map(tokenize_function, batched=True)

   # Define training arguments training_args = TrainingArguments( output_dir=‘./results’, num_train_epochs=3, per_device_train_batch_size=8, evaluation_strategy=“epoch” )

   # Initialize Trainer trainer = Trainer( model=model, args=training_args, train_dataset=tokenized_data[‘train’], eval_dataset=tokenized_data[‘test’] )

   # Train the model trainer.train() ``` :::

4. Evaluation

   • Evaluate the fine-tuned model using a test set from the dataset. Report the accuracy and any other relevant metrics.

   ::: {#f7a9b8de .cell execution_count=10} {.python .cell-code} results = trainer.evaluate() print(results) :::

5. Deliverables

   • A Python script or Jupyter Notebook with your complete fine-tuning process.
   • A short report (1-2 pages) summarizing your approach, the results, and any challenges faced during the assignment. Include visualizations of the model’s performance if possible.

Grading Rubric

Criteria	Points
Data Preparation	5
- Correctly loads and inspects the dataset
Model Selection	5
- Appropriately selects and loads a pre-trained model
Fine-Tuning Process	15
- Correctly tokenizes data and sets up training loop
- Successfully fine-tunes the model
Evaluation	10
- Properly evaluates model performance
- Accurately reports and interprets results
Report Quality	5
- Clearly summarizes approach, results, and challenges
- Includes visualizations and insights

Total: 40 points

Ensure your submission is clear, well-documented, and demonstrates a solid understanding of fine-tuning AI models in a business context.

HW 4 – Agents

HW 4 – Agents

Overview

In this assignment, you will explore the concept of intelligent agents, a core component of AI systems that can perceive their environment and take actions to achieve specific goals. By the end of this assignment, you will be able to design, implement, and evaluate a simple intelligent agent using Python. You will also gain insights into how such agents can be utilized in business scenarios to enhance operational efficiency and decision-making.

Learning Objectives

• Understand the fundamental concepts of intelligent agents in AI.
• Design a simple agent-based system to solve a business problem.
• Implement the agent using Python.
• Evaluate the agent’s performance and suggest improvements.

Business Scenario

Imagine you are working for a retail company that wants to optimize its inventory management system. The company aims to use an intelligent agent to monitor stock levels and automatically reorder products when they fall below a certain threshold. This will help reduce stockouts and overstock situations, ultimately improving customer satisfaction and reducing storage costs.

Instructions

Step 1: Understand the Problem

1. Research Intelligent Agents: Start by reading about intelligent agents in AI. Focus on their characteristics, types, and applications in business.

2. Scenario Analysis: Analyze the given business scenario. Identify the key components and actions that an intelligent agent would need to perform to solve the problem.

Step 2: Design the Agent

1. Define Environment: Outline the environment in which the agent operates. This includes the stock levels, reorder thresholds, and any constraints (e.g., maximum storage capacity).

2. Agent Architecture: Design the architecture of your agent. Specify how it will perceive the environment, make decisions, and act.

3. Pseudocode: Draft pseudocode for your agent’s decision-making process. This should include conditions for reordering and any logic for handling exceptions.

Step 3: Implement the Agent

1. Set Up Environment: Use Python to set up the environment. You can generate synthetic data for stock levels and reorder points.

   import random
   
   # Generate synthetic data
   products = ['Product_A', 'Product_B', 'Product_C']
   stock_levels = {product: random.randint(20, 100) for product in products}
   reorder_thresholds = {product: 30 for product in products}
   
   print("Initial Stock Levels:", stock_levels)

2. Implement Agent: Write Python code to implement the agent. The agent should monitor stock levels and reorder products when necessary.

   def check_and_reorder(stock_levels, reorder_thresholds):
       for product, stock in stock_levels.items():
           if stock < reorder_thresholds[product]:
               print(f"Reordering {product} as stock is {stock}.")
               stock_levels[product] += 50  # Assume each reorder adds 50 units
               print(f"New stock level for {product}: {stock_levels[product]}")
   
   check_and_reorder(stock_levels, reorder_thresholds)

Step 4: Evaluate and Report

1. Test the Agent: Run your agent with different initial stock levels and thresholds. Record its performance in terms of successful reorders and any missed opportunities.

2. Suggest Improvements: Based on your tests, suggest at least two improvements to the agent’s design or logic.

3. Write Report: Prepare a report summarizing your design, implementation, and evaluation process. Include your pseudocode, Python code, test results, and improvement suggestions.

Deliverables

• A Python script (.py file) with the implemented agent.
• A report (PDF or Word document) including:
  • Description of the agent design and architecture.
  • Pseudocode and Python code.
  • Evaluation results and suggested improvements.

Grading Rubric (40 Points Total)

• Understanding and Analysis (10 points): Demonstrates a clear understanding of intelligent agents and the business scenario.
• Design and Pseudocode (10 points): Well-structured agent design and clear pseudocode.
• Implementation (10 points): Correct and efficient Python code implementation.
• Evaluation and Reporting (10 points): Comprehensive evaluation, insightful improvements, and clear presentation in the report.

Please ensure your submission is complete and thorough. Good luck!

HW 5 – Strategy Memo

HW 5 – Strategy Memo

Overview

The purpose of this assignment is to enhance your ability to craft strategic recommendations based on AI-driven insights. By the end of this assignment, you will be able to:

• Analyze data to extract actionable insights.
• Develop strategic recommendations for a business scenario.
• Communicate your findings and strategy effectively in a professional memo format.

Business Scenario

You are a strategic consultant for a retail company, “ShopSmart”, which is looking to optimize its inventory management using AI. ShopSmart has been facing challenges with overstocking and stockouts, leading to increased costs and lost sales opportunities. Your task is to analyze their sales data, identify patterns and trends, and propose a strategic AI solution to improve their inventory management.

Instructions

Step 1: Data Analysis

1. Data Acquisition: Download the sales data from the following link: ShopSmart Sales Data. If the link is unavailable, use the synthetic data generated below:

   import pandas as pd
   import numpy as np
   
   np.random.seed(0)
   dates = pd.date_range('2023-01-01', periods=100)
   data = {
       'date': np.random.choice(dates, 1000),
       'product_id': np.random.randint(1, 50, 1000),
       'units_sold': np.random.poisson(20, 1000),
       'price_per_unit': np.random.uniform(5.0, 20.0, 1000)
   }
   sales_data = pd.DataFrame(data)
   sales_data.to_csv('sales_data.csv', index=False)

2. Data Exploration: Load the data into a Pandas DataFrame and perform exploratory data analysis (EDA) to understand the sales trends.

   import pandas as pd
   
   # Load data
   sales_data = pd.read_csv('sales_data.csv')
   
   # Perform EDA
   print(sales_data.describe())
   print(sales_data.groupby('product_id')['units_sold'].sum().sort_values(ascending=False).head(10))

3. Pattern Identification: Identify patterns such as seasonal trends, high-demand products, and periods of overstocking or stockouts.

Step 2: Develop Strategic Recommendations

4. AI Solution Proposal: Based on your analysis, propose a strategy for implementing an AI solution to optimize inventory. Consider using predictive analytics to forecast demand and adjust inventory levels accordingly.

5. Strategic Memo Writing: Write a strategic memo to the ShopSmart executive team. Your memo should include:

   • An executive summary of your findings.
   • Key insights from your data analysis.
   • Your proposed AI strategy and its expected impact on inventory management.
   • Any assumptions or limitations in your analysis.

Deliverables

• A Python script (strategy_analysis.py) containing your data analysis code.
• A strategic memo (strategy_memo.pdf) addressed to the ShopSmart executive team.

Grading Rubric (Total: 40 Points)

• Data Analysis (10 Points)
  • Thoroughness of EDA (5 Points)
  • Correct identification of patterns and trends (5 Points)
• AI Strategy Proposal (10 Points)
  • Clarity and feasibility of the proposed AI solution (5 Points)
  • Expected impact and benefits clearly articulated (5 Points)
• Strategic Memo (15 Points)
  • Clarity and professionalism of the memo (5 Points)
  • Comprehensive executive summary and key insights (5 Points)
  • Well-reasoned recommendations and strategic alignment (5 Points)
• Code Quality (5 Points)
  • Code readability and documentation (3 Points)
  • Correct implementation of data analysis (2 Points)

Ensure your submissions are clear, concise, and demonstrate a deep understanding of the strategic use of AI in business contexts.

Group Project

STRAT 490R – Building Strategic AI Solutions

Assignment: Group Project

Overview

The purpose of this group project is to provide you with hands-on experience in designing, developing, and evaluating a strategic AI solution for a real-world business scenario. By engaging in this project, you will deepen your understanding of the strategic application of AI technologies in business contexts, enhance your teamwork and problem-solving skills, and appreciate the practical value AI brings to business operations.

Learning Objectives:

• Apply AI methodologies to solve business problems.
• Develop a strategic AI solution aligned with business objectives.
• Evaluate the effectiveness and impact of AI solutions.
• Collaborate effectively within a team to deliver a comprehensive project.

Business Scenario

You are part of a consulting team hired by a retail company, “ShopSmart”, which is facing challenges in managing its inventory efficiently. ShopSmart wants to leverage AI to predict product demand and optimize stock levels to reduce waste and improve customer satisfaction. Your task is to develop an AI-driven demand forecasting solution that can provide accurate predictions to inform inventory decisions.

Instructions

1. Form Groups:

   • Form groups of 4-5 members. Ensure diverse skill sets within your team to cover different aspects of the project, such as data analysis, programming, and presentation.

2. Understand the Business Context:

   • Research the retail industry and the specific challenges related to inventory management.
   • Identify key factors that influence demand forecasting in retail.

3. Data Collection:

   • Use the provided synthetic dataset, which simulates ShopSmart’s historical sales data: Download Synthetic Data.
   • The dataset includes columns such as Date, Product_ID, Sales_Quantity, Price, Promotion, and Store_Location.

4. Data Preprocessing:

   • Clean and preprocess the data to handle missing values, outliers, and categorical variables.

   • Example Python code for preprocessing:

     import pandas as pd
     
     # Load data
     data = pd.read_csv('synthetic-sales-data.csv')
     
     # Handle missing values
     data.fillna(method='ffill', inplace=True)
     
     # Convert categorical columns
     data['Promotion'] = data['Promotion'].astype('category').cat.codes

5. Model Development:

   • Choose an appropriate machine learning model for demand forecasting (e.g., Linear Regression, Random Forest, or LSTM).
   • Split the data into training and testing sets.
   • Train your model and evaluate its performance using appropriate metrics (e.g., RMSE, MAE).

6. Example Python Code for Model Development:

   from sklearn.model_selection import train_test_split
   from sklearn.ensemble import RandomForestRegressor
   from sklearn.metrics import mean_squared_error
   
   # Feature selection
   features = data[['Price', 'Promotion']]
   target = data['Sales_Quantity']
   
   # Split data
   X_train, X_test, y_train, y_test = train_test_split(features, target, test_size=0.2, random_state=42)
   
   # Train model
   model = RandomForestRegressor(n_estimators=100, random_state=42)
   model.fit(X_train, y_train)
   
   # Evaluate model
   predictions = model.predict(X_test)
   print(f'RMSE: {mean_squared_error(y_test, predictions, squared=False)}')

7. Solution Presentation:

   • Prepare a presentation (10-15 slides) summarizing your approach, findings, and recommendations for ShopSmart.
   • Include visualizations of your data analysis and model predictions.
   • Discuss the strategic implications of your AI solution for ShopSmart.

8. Deliverables:

   • A written report (5-7 pages) detailing your methodology, analysis, and conclusions.
   • Python scripts used for data processing and model development.
   • A presentation file summarizing your project.

Grading Rubric (40 Points Total)

• Understanding of Business Context (5 points): Demonstrates a clear understanding of the retail industry and inventory management challenges.
• Data Preprocessing (5 points): Effectively cleans and preprocesses the data for analysis.
• Model Development (10 points): Chooses an appropriate model, trains it effectively, and evaluates its performance.
• Solution Presentation (10 points): Clearly communicates findings and recommendations with well-designed slides and visualizations.
• Team Collaboration (5 points): Evidence of effective teamwork and contribution from all group members.
• Report Quality (5 points): Presents a comprehensive and well-structured report with clear methodology and conclusions.

By completing this project, you will gain valuable experience in applying AI to solve real-world business problems and develop strategic insights that can drive business success.