What does it mean that isolate variables when optimizing?

Change one thing at a time so you can attribute improvements to specific changes.

Example: You run an AI agent that summarizes customer support tickets and routes them to the right team. Response accuracy is 72 percent, and you want to reach 90. You have three ideas: rewrite the system prompt with clearer routing criteria, switch from GPT-4o to Claude, and add a retrieval step that pulls the five most similar resolved tickets as context. You implement all three changes at once. Accuracy jumps to 88 percent. Progress — but which change drove the improvement? Maybe the new prompt alone would have reached 88 percent and the model switch added nothing. Maybe the retrieval step actually hurt accuracy by injecting irrelevant examples, and the gains came entirely from the prompt and model changes. Maybe the model switch introduced a regression that the retrieval step masked. You cannot know, because you changed three variables simultaneously. Now imagine you made one change per week. Week one: rewrite the prompt. Accuracy moves from 72 to 81 percent. You now know the prompt rewrite is worth nine points. Week two: switch the model. Accuracy drops from 81 to 78 percent. You now know this model performs worse with your prompt — revert the change, save the API cost. Week three: add the retrieval step. Accuracy moves from 81 to 91 percent. You exceeded your target, and you know exactly which component delivered each increment. More importantly, you avoided shipping a model change that actually degraded performance — a regression you would never have detected if you had changed everything at once.

Try this: Identify something you are currently trying to improve — an AI agent, a workflow, a habit, a creative process, a system. List every change you are considering making. Now rank them by your best guess of impact. Take only the top-ranked change and implement it in isolation. Define your measurement: what metric or observation will tell you whether this change helped, hurt, or made no difference? Run the single change for a defined period — one week for a digital system, two weeks for a habit or process. Record the result. Only then move to the second change. After implementing two changes sequentially, write a brief log entry: what did change one do, what did change two do, and would you have been able to determine this if you had made both changes simultaneously?