Introduction to Automating Data Analysis
As data analysts, we’ve all been there – the dreaded request for the monthly or yearly report, an essential task that’s also a massive time sink. The question is, can AI just do this? Can it whip up a simple data analysis report?
The Prerequisites
Before diving in, a word on the setup is necessary. Running Claude Code in a local environment is a must, but to give Claude Code true data analyst powers, a tool for Claude Code to securely read the database and run queries is also needed. There are many great open-source examples of this online, and finding one that works for the setup is crucial.
Attempt 1: The No-Context Data Analyst
The first attempt was an optimistic one, giving Claude Code full decision-making power on its own. A simple prompt was thrown into the chat, asking Claude Code to provide a monthly analysis of why users are contacting us. However, Claude Code struggled. The SQL queries were generating errors because Claude Code had no context on what the correct tables to reference were, what the columns meant, or what the values meant. It made guesses everywhere and was forced to constantly fix its errors.
Improving the Context
The key is to think about the context that would be given to a brand new data analyst who joined the team and needed to write the report. What would be shared with them? The answer is information about the tables, a few SQL queries to get going, and an example structure of the report. This includes table documentation, a list of the main queries to run, a few example SQL queries, a chart style guide, and an analysis blueprint.
Improving the Instructions to Follow
We needed a way to control Claude Code’s attention so that it focused on the right task step by step. It was time to enforce Claude’s todo list. The prompt created was a step-by-step workflow for it to follow, which baked in an iterative loop over each of the ‘pending tasks.’ Once the pending task was done, it moved to ‘completed tasks’ in the todo list.
Attempt 2: A Much More Detailed Prompt
After all the trial and error, the final prompt looked like a detailed, step-by-step guide. It included a first message – query writer, and a second round – writing the analysis. The first message asked Claude Code to create SQL queries, execute them, and create charts based on the results. The second round asked Claude Code to provide clear analysis about trends in data and write data reports based on charts.
How Did It Do?
It did well. Ultimately, the AI was able to produce all the correct queries/charts that it needed to do and write an analysis of the data trends. The key principles learned were to give it its onboarding and control Claude Code’s todo list. Taking the time to set up the documentation for it to follow can take a bit of time, but if the report is written regularly, the work pays dividends quickly.
Conclusion
In conclusion, automating data analysis with Claude Code is possible, but it requires giving it the right context and controlling its workflow. By providing the necessary documentation and enforcing a todo list, Claude Code can produce accurate queries and charts and write a good analysis of the data trends.
FAQs
Q: What is Claude Code?
A: Claude Code is an AI tool that can be used for various tasks, including data analysis.
Q: What are the prerequisites for using Claude Code for data analysis?
A: The prerequisites include running Claude Code in a local environment and having a tool for Claude Code to securely read the database and run queries.
Q: What are the key principles for automating data analysis with Claude Code?
A: The key principles are to give it its onboarding and control Claude Code’s todo list.
Q: Can Claude Code produce accurate queries and charts?
A: Yes, Claude Code can produce accurate queries and charts if given the right context and controlled workflow.
Q: What’s next for automating data analysis with Claude Code?
A: The next step is to give Claude Code more freedom to decide what queries to deep-dive into and upgrade it from a junior intern to a self-sufficient mid-level analyst.
