The Test First, Implement Second Technique

In the last post, we saw how AI can give plausible-sounding instructions that contain hidden errors.  These include filtered data that later steps ignore, column names that don't exist in your file, denominators that don't match your actual dataset, etc.

The traditional approach is to ask AI for instructions, then verify the results after you've implemented everything. But by then, you've already invested time following potentially flawed steps.

The trick is to make AI verify its instructions before you implement anything.

Instead of asking "How do I analyze this data?" tell it to "Test your proposed approach on my actual data, then give me verified instructions." This shifts AI from pattern-matching mode into verification mode. You're forcing it to ground its instructions in your specific file structure before it tells you what to do.

The technique forces AI to examine your file structure, validate its proposed approach against your real data, and check that all steps remain compatible with each other as they build on your data's current state. Only after this verification do you receive step-by-step guidance.

The Practical Prompt Template

Here's what this looks like for our housing survey scenario. Copy this structure and adapt it to your own analysis needs:

Effective Verification Prompt:

I need to analyze housing survey data, but I need you to test everything before giving me instructions.

Before providing any analysis steps:

1. Analyze my file structure and current state

    • Tell me what columns exist and their exact names
    • Identify the data types in each column
    • Note how many total rows I have
    • Check for any obvious data quality issues

2. Test your proposed solution on my actual data

    • Run through your entire proposed workflow using my actual column names
    • Verify that each step can actually be performed with my data structure
    • Check that filtered data from earlier steps is correctly accounted for in later steps

3. Verify all steps are consistent and compatible

    • Ensure that if Step 1 filters data, Steps 2-5 account for that filtered state
    • Confirm that column references in later steps match what exists after earlier transformations
    • Check that any totals or denominators used in calculations reflect the current data state

4. Only then provide verified, step-by-step instructions

    • Number each step clearly
    • Reference my actual column names (not assumed ones)
    • Note any dependencies between steps
    • Include verification checks I should perform after key steps

CRITICAL REQUIREMENT: If at any point you realize your instructions would be contradictory or wouldn't work with my actual data, STOP and re-analyze everything.

Specific things to verify:

    • If you tell me to filter data, ensure ALL subsequent calculations use filtered counts
    • If you reference a column name, make sure that exact column exists in my file
    • If you create a pivot table in Step 2, ensure Step 3 doesn't assume unfiltered source data
    • If you calculate percentages, verify the denominator matches the current data state

My goal: Calculate response rates by age group and identify top 3 housing concerns by region.

[Attach your data file]

Why This Works

"Before giving me any steps" forces front-loaded analysis rather than on-the-fly instruction generation. AI commits to understanding your data first rather than generating generic steps and hoping they fit.

"Test your proposed solution on my actual data" is key. You're asking AI to run the workflow before outputting it. It traces through: "If I filter in Step 1, what columns exist in Step 3? What's the row count in Step 5?"

"Verify all steps are consistent and compatible" addresses the context fragmentation problem directly. AI checks whether Step 5 works given what Step 2 did.

"If you make any errors, stop and re-analyze everything" creates an error-catching checkpoint and prevents cascading mistakes.

The specific verification examples target the most common failure modes: filter consistency, column names, denominator accuracy.

When you ask: "Help me calculate response rates by age group in my survey data" here's what AI gives you: It creates a pivot table with Age_Group and Response columns, counts total responses in each group, then divides by 2,500 to get percentages.

It assumes your column is called "Age_Group" when yours is actually "AgeCategory," doesn't account for incomplete responses you need to exclude, and uses the wrong denominator (should be invited respondents per age group, not the total sample size of 2,500). You discover these issues only after implementing the steps and getting strange results.

Verification-First Prompt

Use the template above with your file attached. In this situation, here's what AI gives you: It analyzes your file structure first and identifies that your age column is called "AgeCategory," not "Age_Group." It finds your response completeness indicator ("Survey_Complete" with Yes/No values) and notes you have 2,847 total rows including 347 incomplete responses. When it tests the proposed workflow—filtering to complete responses, creating a pivot table with the correct column name—it stops at the response rate calculation and asks for clarification. Should the denominator be 2,847, or are you measuring against a different invited population? Are you calculating completion rate within your sample, or response rate versus total invited? Once you clarify, it'll provide verified steps using your actual column names, appropriate filter state, and accurate denominator.

Using the verification-first prompt the AI identifies your actual column names, catches the incomplete response issue, and asks for clarification on metric definition before giving you potentially wrong instructions. You have a dialogue based on your actual data, not assumptions.

Adapting the Template

The core template works for most quantitative analysis, but you can add context-specific verification for different scenarios.

For example, for budget or financial analysis, you might add: "Verify that any formulas account for fiscal year boundaries, that currency amounts are in the correct units (thousands vs. millions), and that totals match across different aggregation levels."

For statistical comparisons, you could add: "Confirm that sample sizes are adequate for any significance tests, that comparison groups are properly defined, and that any percentages use the appropriate base."

And for time-series data, I would probably add: "Ensure that date ranges are consistently applied across all calculations, that any growth rates use the correct time periods, and that seasonal adjustments are appropriate."

What to Do With the Response

Once AI gives you verified instructions, review the file analysis section—does it accurately describe your data? Check any clarifying questions. If AI asks for clarification, it's being careful rather than making assumptions. Implement steps incrementally. Don't do all ten steps at once—do Step 1, verify it worked, then move to Step 2. After major transformations like filters or aggregations, spot-check a few rows manually.

This technique adds two or three minutes to your initial prompt writing, but it can save you hours of debugging and rework. It increases the likelihood that your analysis is correct.

In government work, you're accountable for the numbers you produce. This technique helps ensure you can confidently stand behind your results.

Coming up next: Post 3 covers the six most common ways AI gets your numbers wrong—the specific error patterns to watch for even when using verified prompts.