Generative AL Prompt Engineering Project

Files needed for input:

#2 Inventory Process Narrative.docx#3 Inventory Control Listing.xlsx Cased Background: Generative AI Used in Auditing
The business landscape has been inundated with talk of artificial intelligence, specifically generative AI, over
the course of the last several months of 2023. The Merriam-Webster dictionary defines generative AI as
“artificial intelligence that is capable of generating new content (such as images or text) in response to a
submitted prompt by learning from a large reference database of examples (Generative AI, MerriamWebster). LLM tools such as ChatGPT and Bard are text-based machine learning models – they are trained
using various AI and machine learning techniques designed to analyze patterns. These tools are fed copious
amounts of text/language and to recognize, translate, predict, and generate text or other content.
The LLM platforms that exist nowadays are quite user friendly, limiting the barrier to entry to using these
technologies. By inserting background information and the ultimate prompt or question into the LLM, if
prompted appropriately, it will generate a response in a matter of seconds.
Industry experts believe that generative artificial intelligence has the potential to be the biggest disruptor to
businesses since the Internet. Professionals are quickly realizing the ground-breaking capabilities of LLMs
and are working to leverage these technologies within their organizations.
A few of the recognized benefits of AI to businesses are as follows (Pratt, M. (2023, June 21) 12 key benefits
of AI for business, TechTarget):
•
Efficiency and productivity gains
•
Improved customer service
•
New capabilities and business model expansion
•
Faster product development
•
Improved monitoring and better quality
•
Better talent management
Why Does it Matter?
OpenAI, the creators behind ChatGPT, included a model in their published research indicating that
accountants and auditors are 100 percent exposed to AI disruption in their industry (Eloundou, T. et al. (2023,
August 22) GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models,
Open AI, Open Research, University of Pennsylvania). Already, firms are investing billions of dollars into AI
relationships.
As such, it is up to incoming accounting professionals to equip themselves with knowledge and exposure to
these technologies as they begin their careers. The purpose of this case study is to introduce students to the
basic generative AI terminology and use cases. The case begins with an introductory training on LLMs and
prompt engineering, or put simply, crafting the most appropriate question to receive the desired result.
Technology
Consider ways in which you can leverage AI within the audit—brainstorm and practice various use cases using
provided data/information
You will need access to a generative AI LLM of your choice – some options include Open AI’s ChatGPT,
Google’s Bard, Anthropic’s Claude, among others.
Prompt engineering
Countless experts point to the importance of understanding effective prompt engineering as we enter this AI
journey. Again, prompt engineering is the concept of crafting the most appropriate question to receive the
desired result. It boils down to using effective and clear communication. Prompt engineering allows us to
derive value from LLMs and refers to how users ask a question or request a task from AI. Engaging with AI in
an effective and efficient manner will be increasingly crucial as capabilities advance.
Notably, while LLMs have tremendous processing capabilities, they are not foolproof. Sometimes when asked
a question the technology does not know the answer to, it will “hallucinate” by confidently providing the wrong
answer, perhaps even citing a source that does not exist. They also often have issues performing basic
mathematical functions.
NOTE: As with any technology application, it is necessary to appropriately evaluate data integrity,
completeness, and accuracy. Therefore, it is important to review the LLM’s responses and build an effective
method for evaluation. Also like any technology the axiom garbage in, garbage out applies. Understanding the
impact of LLM’s inputs and outputs is critical to their effective use. High quality prompt engineering (input) and
output validation are critical when working with generative AI.
Tips when working with LLMs:
•
Telling the LLM to assume a certain role, that of an auditor or accountant for example, can improve the
response.
•
Clearer prompts don’t necessarily mean shorter.
•
Prompt the tool to work through its own solution first before providing a conclusion; tell it to say “I don’t
know’ if it doesn’t know the answer to your question.
•
Break down tasks into steps to give the model time to think, which leads to more accurate responses.
Consider follow-up prompts to drive effective reasoning.
•
Some LLMs have limitations around the amount of data you can submit in the prompt text box. If you are
receiving an error, try copying and pasting the information in chunks and ensure you are only using the
information relevant to the question at hand.
[recommended] Prompt Engineering Tutorial – Master ChatGPT and LLM Responses
•

Inventory Process Narratives
1. Procurement of raw materials
The procurement team sends purchase orders to approved suppliers for necessary
raw materials such as fabric, insulation materials, buttons, zippers, and other
accessories. The team maintains a detailed record of these purchase orders and
regularly communicates with suppliers to help ensure timely delivery.
2. Reception of raw materials
Upon arrival of the raw materials, the warehouse team inspects and verifies the
quality and quantity of the received goods against the purchase orders. The team
records any discrepancies and reports them to the procurement team. Once the raw
materials are inspected and approved, they are stored in designated areas within the
warehouse.
3. Production and work-in-progress
The production team receives job orders and works accordingly to manufacture
winter hats, gloves, and coats using the raw materials. Throughout the production
process, the work-in-progress (WIP) inventory is tracked and monitored to help
ensure efficiency and proper resource allocation.
4. Finished goods inventory
Upon completion, the finished products undergo a quality inspection and then are
labeled and stored in the warehouse’s designated area for finished goods. The
warehouse team updates the inventory records to reflect the addition of the finished
goods, adjusting the WIP inventory accordingly.
5. Sales and order processing
The sales team processes customer orders and communicates with the warehouse
team to prepare and pick the required items for shipment. Once the orders are
shipped, the sales team generates an invoice and forwards a copy to the accounting
department.
6. Inventory valuation and financial reporting
The accounting team is responsible for maintaining accurate inventory records and
valuing the inventory using a consistent method, such as first-in-first-out or weighted
average. The inventory valuation is incorporated into the company’s financial
statements, adhering to US generally accepted accounting principles and Financial
Accounting Standards Board requirements.
7. Periodic inventory counts and reconciliation
Keepin’ You Warm performs periodic inventory counts, such as cycle counts or
annual physical counts, to verify the accuracy of inventory records. Any
discrepancies are investigated, and appropriate adjustments are made to the
inventory records, helping ensure alignment with the physical inventory count.
8. Obsolete and slow-moving inventory management
The company regularly reviews its inventory to identify slow-moving or obsolete
items. Management devises strategies for managing such inventory, including
promoting sales, offering discounts, or disposing of the items as needed. This helps
ensure that inventory remains relevant and contributes to the company’s profitability
Example:
X. Sub-process
– Control 1 …
– Control 2 …
1. Procurement of Raw Materials
– Approved supplier list to ensure purchases are made from reliable sources.
– Segregation of duties between those who create, authorize, and record purchase orders.
2. Receiving of Raw Materials
– Independent verification of quality and quantity of received goods against purchase orders.
– Proper storage of raw materials in designated areas within the warehouse.
3. Production and Work-in-Progress
– Standardized job orders to guide the production process.
– Monitoring and tracking of WIP inventory to ensure production efficiency.
4. Finished Goods Inventory
– Quality inspection of finished goods before adding them to inventory.
– Accurate labeling and storage of finished goods in designated areas.
5. Sales and Order Processing
– Authorization and review of customer orders before processing.
– Confirmation of shipped orders and issuance of invoices.
6. Inventory Valuation and Financial Reporting
– Consistent application of inventory valuation method (e.g., FIFO or weighted average).
– Reconciliation of inventory records to general ledger.
7. Periodic Inventory Counts and Reconciliation
– Independent personnel conducting inventory counts to ensure accuracy.
– Investigation of discrepancies and adjustments to inventory records, as necessary.
8. Obsolete and Slow-Moving Inventory Management
– Regular review and identification of slow-moving or obsolete inventory.
– Approval of strategies for managing slow-moving or obsolete inventory.
Files needed for input:
•
•
#2 Inventory Process Narrative.docx
#3 Inventory Control Listing.xlsx
IMPORTANT: So far, we have reiterated that sensitive client information is not to be uploaded to
public instances of AI technology platforms. This may seem counterintuitive when we are uploading
client business processes in this portion of the case. However, it is important to point out that most
audit firms have their own proprietary AI technology that is internally confined and monitored,
allowing employees to ask more client-specific questions (subject to audit engagement letter
allowable data use considerations). These are not to be posted to public instances.
As the tool does not have specific information about account balances, risk of material
misstatements, industry factors, the client’s business environment, nor other risk and fraud factors,
including those external to the company (i.e., industry-specific), auditors must exercise their
professional judgment to determine the appropriate procedures.
Requirements (total 100):
Please consider the following hypothetical situation: your engagement team was recently engaged to
conduct the audit of a national manufacturing company, Keeping You Warm, LLC, that designs and
manufactures cold-weather accessories including wool hats, gloves, and socks.
You are an audit associate assigned to the inventory process for this year’s audit. You have just
completed walk-through meetings with the client and are currently working through the initial risk
assessment of the inventory process. Read through the Inventory Process Narrative.docx
supplemental file received from the company’s key inventory contact.
1) [30 points] Use the company’s inventory process narrative in the LLM, along with the current
list of inventory controls in place (Inventory Control Listing.xlsx). Where are there potential
gaps identified in controls? Identify additional controls that can be added to each step of the
process. Please list the LLM’s response below (10 points) and comment on the proposed
controls. In your opinion, which are critical for the company to add (20 pints)?
2) [30 points] Based on the company’s current process narrative and controls already in place (not
including the suggested controls in your response to #3), have the tool provide an initial risk
assessment of the company’s current-year inventory process in the aggregate (not separated by
subprocess) (10 points). Use your professional judgment to comment on any oversights within
the tool’s response (20 points).
3) [15 points] As your team is finalizing its audit plan for the year, ask the LLM what substantive
procedures it recommends performing over the various inventory account balances, given all the
above information. What procedures does it recommend?
4) [15 points] Based on the recommended substantive procedures from the previous question, ask
your LLM to generate an initial Prepared-by-Client (PBC) list to deliver to the client for interim
and year-end procedures. A PBC list includes all of the various schedules, documents, or sample
support needed in order to complete your audit procedures.
5) [10 points] Please complete the below survey: https://forms.gle/JrgS8zWdCpvZh58HA