If an AI answer changes, can you explain why?
Track the model, prompt, evidence, tools and policies behind an AI answer so changes can be investigated and reproduced.
Field notes
Things I’m
exploring.
Practical field notes on using AI, making product decisions, understanding data, and noticing the invisible structures behind everyday experiences.
An agent handoff is a contract, not a forwarded chat
How to pass goals, evidence, permissions and failure states between AI agents without losing the task.
Which decisions should the AI make—and which belong in code?
A practical way to decide which parts of an AI workflow need model judgment, deterministic code or human review.
A human in the loop is not a safety control by itself
Human approval only helps when reviewers have independent evidence, expertise, time and genuine authority.
When AI agrees with you, what exactly has it checked?
What sycophancy research reveals about AI agreement—and how to test whether your opinion changes the answer.
AI can have the answer in front of it—and still miss it
What the Lost in the Middle research reveals about long AI context—and a practical way to check whether evidence survives the extra material.
You changed the prompt. Did the AI actually get better?
A practical introduction to AI evaluations: compare prompt changes, check the judges and catch regressions before release.
Prompt injection: when the document tries to become the instruction
How instructions hidden in documents can redirect an AI agent—and how to test the boundary between evidence and authority.
An AI agent needs a budget, not just a goal
How execution budgets, completion rules and retry limits keep agent workflows useful, reliable and worth their cost.
How an AI agent keeps track of its work
State, checkpoints and recovery: how an agent records progress, resumes safely and distinguishes a plan from completed work.
How AI uses tools—and who actually runs them
Follow a tool call from the model’s request to execution, evidence and an answer—and see where the application stays in control.
The numbers were right. The recommendation wasn’t.
A correct calculation can hide an unfair comparison. Check the audience mix before trusting an AI recommendation.
Your AI found a policy. Was it the right one?
Finding a relevant passage is only the beginning. Check the version, context and exceptions before trusting the answer.
Give AI a brief you would give a colleague
A clearer assignment helps the assistant preserve the purpose of your work, not merely polish its wording.
Before you ask AI for an answer, ask what it can see
A polished explanation depends on what the assistant had available when it produced it.
AI should remember what matters—and let the rest expire
A useful preference can make the next answer better. An old assumption can make it narrower.
The five-star restaurant might not be your restaurant
A rating summarizes other people’s evenings. How do you decide whether it fits yours?
The rollout is part of the product
A feature’s journey to users needs eligibility, exposure, guardrails and a recovery plan.
Before an AI agent gets a button, give it a boundary
A plausible answer can be corrected. An action may already have changed something.
Not everything in your wardrobe has to earn its place the same way
Cost per wear is useful. It cannot explain every kind of value.
A family recipe is a model, not the meal
The recipe gives a starting point. The food gives feedback.
The AI returned valid JSON. The answer was still wrong.
A schema can verify the shape of an AI answer. It cannot verify its truth.
A launch is not an experiment just because the chart moved
The chart moved after launch. Here’s how to decide what the evidence actually lets you claim.
I learned faster when I stopped trying to sound technical
A practical way to expose the edge of a mental model, test it, and use AI for correction rather than reassurance.
The word “Parmesan” is not the proof. The supply chain is.
What a protected cheese can teach us about definitions, traceability, and the difference between an attribute and provenance.
The best part of a trip is often the part you didn’t schedule
Choose one anchor, understand a neighbourhood, and leave enough room for the place to surprise you.
The wine was already good. The label was getting in the way.
A favourite Napa Cabernet, a blind tasting, and a lesson about evaluating the product rather than its reputation.
The tire company that gave people somewhere to go
Michelin did not simply sell tires. It created more reasons to need them.
Before you trust an AI analysis, check its grain
AI can write a plausible query. This workflow checks whether it counted the right thing.
A dashboard can be fresh and still be too late
A pipeline can finish on time while the decision arrives too late. Measure the full journey from event to action.
One impression, five paths: when more supply isn’t actually more supply
More bid requests can describe the same opportunity. Explore what each supply path actually adds.
Why a loan balance shouldn’t be just one number
Settle a payment, retry it, and reverse it. Follow how principal, interest, and financial history explain every dollar.
The payment happened once. Why is it in your data twice?
Retries, late events and reversals are normal. Here’s how to keep one financial action from becoming two—and still preserve the history.
Agent orchestration: supervisors, specialists & learning loops
How a supervisor coordinates micro-agents, a validator checks evidence, and evals and feedback improve the system.
Use AI to challenge your thinking—not just finish your sentence
A five-prompt workflow for finding weak assumptions before they become expensive decisions.
A metric can improve while the product gets worse
How to pair a success metric with downstream quality and unintended-harm guardrails.
The queue at your coffee shop is telling you where the bottleneck lives
A practical way to map throughput, constraints, and failure using an ordinary morning ritual.
Visibility is infrastructure, not vanity
A tiny practice for making valuable work legible without becoming performative.
Your closet has an inventory problem
What utilization, optionality, and feedback loops can teach us about buying fewer, better things.