What's Next For Accounting? - Issue #84

Author

Jason Staats
May 25, 2023

💡Building Better Prompts

Buried in the news that GPT-4 aced every accounting certification, was the fact that chain-of-thought prompting yielded a 16% better score on the CPA exam.

Simply by changing how the prompt was written, it scored 16% better. The study is a fascinating read, and a great advertisement for the value of prompt engineering. In their tests vanilla GPT-4 scored 16% better than GPT-3.5, and with some prompt engineering they got GPT-4 to score another 16% better.

We discussed it on my daily pod today, and if your initial reaction is I’m not going to that level of effort to work with a machine, consider the efforts we go through with our human colleagues. The time we spend with interns, the cross-training, the effort that’s poured into every individual.

I’ll admit, a positive culture that grows fellow human beings has a different appeal than becoming a robot-whisperer, but who says we can’t have both!

I’ve built a swipeable prompt that uses chain-of-thought prompting for research. It could be tax research, software research, any question you’d normally Google. My initial results have performed higher than any other research utility I’ve used.

I’m using ChatGPT Plus, specifically the new Bing Search GPT-4 model (select GPT-4, Browse with Bing). I tried the same with the free version of Bing and the results were much more sporadic. ChatGPT broke around 25% of the time, and Bing gave up 75% of the time 🤷

  1. Paste the full prompt into ChatGPT

  2. Provide it with an explicit URL to research from. This ensures it’s only referencing up-to-date, authoritative source material. Currently this doesn’t work with PDF files, so in the below example of an IRS publication (I know this isn’t authoritative you tax nerds) I linked it to the HTML version.

  3. Provide it with a question.

Rather than simply asking a question, the prompt has it fetch the 10 most relevant excerpts from the source material, then based on those excerpts forms an answer to the question. This basic example of chain-of-thought prompting focuses the model on the most relevant material, and provides a more logically-coherent answer.

💎This Week's Sponsor

Have a product or job opening to advertise? Reply to this email for more info.

📅 This Week

😍From The Community

My private peer networking community Realize for firm leaders, now capped at 500 members.

Upcoming Events:

▮▮▮▮▮▮▮▮▮▮▮▮▯▯▯▯▯▯▯▯ 308/500 members