Thinking Through AI 🤖

A survey draft comes back for review, and one question stops you. It's well-written and clear, and you can't work out why it's there. So you ask the program area: what will you do differently depending on how people answer this? There's a pause. Then: "We just thought it would be interesting to know." Interesting to know is not nothing. But it's not a reason to put a question in front of the public, take up their time, and then carry the answer forward into a report that shapes a decision. If you can't say what the answer would change, you can't tell, when the results come in, whether the question did its job. You've collected data you have no standard for evaluating. This is a companion piece to my "Confident and Wrong" series, which is about a gap in how government adopts AI: we measure whether tools get used, but not whether anyone can tell when they're working. This post explores one part of it. The practice has an unglamorous...

A piece of guidance crosses your desk. It tells you, as the person using an AI tool, to check the output for accuracy and bias before you rely on it. It even gives you questions to ask: are some perspectives portrayed more favourably than others? Are any important viewpoints missing? Does the wording lean on assumptions about a group? These are good questions. They reflect real awareness of how these tools go wrong. Then comes the line that does the work: "... if the output feels biased, do not use the content." Read that again. "If it feels biased." That's where the guidance hands the hardest part back to you and walks away. This is the last of three posts ("Confident and Wrong") adapted from a talk I gave at the joint Canadian Conference on AI, Robots & Vision (AI/CRV) held at Simon Fraser University in May 2026. The first two looked at what careful AI use took on my own team — analyzing survey responses after the fact ( Post 1 ), and stress-tes...

You've drafted a survey. Before it goes out to the public, you want to know where it'll trip people up: the question that reads clearly to you but lands oddly for someone encountering it cold, the response option you forgot to include, the wording that quietly nudges people toward an answer. Normally you'd find this out by testing the draft with real respondents. But testing costs time and money you don't always have, so you ask an AI to play the respondent instead — adopt a persona, work through the survey as that person would, and flag whatever causes confusion. It works, up to a point. The trouble is that the point where it stops working doesn't announce itself either. This is the second of three posts ("Confident and Wrong") adapted from a talk I gave at the joint Canadian Conference on AI, Robots & Vision (AI/CRV) held at Simon Fraser University in May 2026. The first post looked at AI-assisted analysis of survey data after the fact. This one move...

A thematic summary lands on your desk. Three hundred and forty-two people answered an open-ended survey question, and an AI tool has sorted their responses into five clean themes, each with a count and a percentage. Twenty-five percent supported the changes. Twenty-two percent had concerns about implementation. It's tidy and fast, and it looks finished. You could drop it into a briefing note this afternoon. Here's the problem: a wrong answer and a right answer look almost identical coming out of an AI tool. The output itself gives you little purchase on which one you're holding. That table could be an accurate read of what people said, or it could be quietly off, there's really nothing on the page telling you which. The gap between how an output looks and what it actually contains is what this series is about. This is the first of three posts ("Confident and Wrong") about a gap I've come to think is the most important and least discussed problem in how lar...

The core technique in Post 1 is straightforward: set up the observation, do your work, ask the AI to reflect back what it noticed. Do that consistently and you'll build a description of your own learning needs that makes each session sharper than the last. That loop is worth getting comfortable with before adding anything else. But once it's working, there are a handful of techniques that extend it — ways to catch your assumptions before they lead you astray, test whether you've actually understood something, and build a longer-term record of how your thinking on a topic develops over time. None of them require much extra effort, and they all work the same way the core loop does: by turning the AI's attention toward how you're thinking, not just what you're asking. The first technique is what you might call an Assumption Auditor . Partway through a complex piece of work, pause and ask the AI: "What assumptions am I making that I should verify? What have I ...

Most people use AI to get answers. That's useful, but it's only half of what's available. If you're using AI regularly for work, you're sitting on a source of feedback about your own thinking that most people never tap — a record of what you asked, how you framed it, and what that reveals about where your understanding is solid and where it isn't. This post is about how to use that. The technique is simple. Instead of just asking AI questions and moving on, you ask it to pay attention to how you're asking — and at the end of a session, to tell you what that reveals about your understanding. Then you carry those insights forward as context for your next session. AI tools don't retain memory of you between sessions by default, though some tools and configurations are starting to change that. Either way, the value here isn't about working around a technical limitation — it's about developing a clearer picture of yourself as a learner. This isn't...

Who Did We Actually Ask? A program area receives survey results. The report says: 73% of people in B.C. support this policy direction. It sounds like evidence. It lands in a ministerial briefing, gets cited in committee, and shapes how the next budget is framed. Nobody in that chain of handoffs — from panel provider to consulting firm to program area — is set up to ask who those people were, or how they ended up in the survey. That question might seem procedural. It isn't. It's where the credibility of the finding rests. Most government program-area surveys use commercial research panels: pools of people who signed up to take surveys in exchange for small rewards. Nobody randomly selected them from the provincial population. They opted in. That single fact — how participants were chosen — determines whether calling them "people in B.C." is accurate or whether it's a claim the research can't support. There's also a structural reason this ...

You've seen how personas work. You've used them to catch problems in your own work. The harder question is how to make this a regular practice instead of a one-time exercise. Start small. Three to five personas representing populations your ministry actually serves — or populations who struggle with your current services — is enough to start. Resist the urge to cover every possible BC resident before you've used any personas at all. In the first month, build your core personas from data you already have: service usage stats, consultation records, demographic information, feedback from frontline staff. Focus on diversity of access needs rather than demographic variety — a persona representing someone without reliable internet reveals different barriers than one representing someone without English fluency, even if both are "underserved." In months two and three, put those personas to work on existing documents and services. Note which insights were most valuable. R...

You've built your personas. Here's what using them actually looks like. The three scenarios below are based on real government projects. In each case, standard review processes checked the logic and missed the barriers. A 30-minute persona review caught what weeks of internal testing didn't. Testing a New Online Application Service BC was redesigning their BC Services Card renewal process. The new system looked streamlined and modern. Internal testing went smoothly. Then someone asked how Margaret Chen would experience it. Margaret is 73, lives in Victoria, uses her smartphone for most things but gets frustrated when technology doesn't behave predictably. She's willing to learn but needs clear instructions. The team prompted their AI tool: Review this BC Services Card renewal process as Margaret Chen. Margaret: 73, Victoria, smartphone user, no computer, gets frustrated with unpredictable tech, doesn't know technical terms like "cache" or "sessio...