Skip to content
How to ThinkIn the Age of AI

Question

What is AI alignment reward specification values?

intermediate

Quick Answer

Core values are ends in themselves — they define what a good life means to you. Instrumental values are means — they are valuable because they serve core values. Confusing the two leads to optimizing for the wrong things.

Example: A product manager spends twelve years climbing the corporate ladder, optimizing relentlessly for promotion and title. She values ambition, discipline, and competitiveness — or so she believes. After reaching VP, she feels empty. The title was never the value. It was an instrument she mistook for the thing itself. Underneath, her core values were creative expression and intellectual challenge — the things that made her love product work before she started chasing rank. The promotions were supposed to give her more autonomy to do creative work, but somewhere along the way, the pursuit of promotion became its own end. She was running a means-ends inversion: the instrument had consumed the value it was supposed to serve.

This concept is part of Value Identification in the How to Think curriculum.

Learn more in these lessons