datapitfalls - A Tool for Spotting Pitfalls Across the Entire Data Analysis Process, Not Just Charts

What the tool does

datapitfalls finds common errors that people make when they work with data. Many tools only check a chart's looks, such as a cut-off axis or weak colors. This tool also checks earlier steps, like how a question was asked, how data was collected, and how results were explained. You can give it a chart, some code, a plain description of an analysis, or a full report. It then gives a report that names each problem, explains why it matters, and suggests a fix.

Key facts

• The tool sorts every mistake into one of 8 groups, called domains.
• The checks cover 7 steps: question, data collection, transformation, analysis, visualization, interpretation, and communication.
• You can try it in a web browser with no install needed.
• The tool uses the Claude API, a service from Anthropic, an AI company.
• The code is shared under the MIT license. The pitfall list is shared under a CC BY-NC 4.0 license, which does not allow commercial use.

Where the ideas come from

The tool is based on the pitfall list from the book Avoiding Data Pitfalls by Ben Jones. The book was published by Wiley. Today the tool scans one item at a time. Checking a full, linked workflow as one chain is planned for later and is listed on the project's roadmap.

Why it matters

Good charts can still mislead. For example, a neat chart that uses only a short, chosen time period can give a false picture. A perfect database query can still drop missing values and hide them. By checking the whole process, this tool helps people find these hidden errors before they share results. It is meant to be a helper that encourages honest data work, not just a style checker.

What to watch

• Whether the tool expands to check a full, connected workflow as one chain.
• How accurate the checks are on real projects.
• How the project grows, since it accepts contributions from others.