How to Measure Anything

If you can observe it in any way at all, it lends itself to some type of measurement method.

If you can observe it in any way at all, it lends itself to some type of measurement method. Measurement is uncertainty reduction. Contrary to popular belief, measurement doesn't require perfect precision or certainty. It simply means reducing uncertainty about a quantity of interest. This applies to tangible things like physical objects and intangible concepts like customer satisfaction or project risk. Observable consequences. Any intangible that matters must have observable consequences. For example, if you claim employee morale affects productivity, there must be some way to detect changes in morale through productivity metrics. By identifying these observable effects, we can indirectly measure the intangible. Practical methods exist. Many seemingly immeasurable things have already been measured by someone, often using surprisingly simple methods. Examples include: Estimating fish populations in lakes without draining them Measuring the economic impact of brand damage Quantifying the value of a human life for policy decisions

Measurement matters because it must have some conceivable effect on decisions and behavior. Decision-driven measurement. The purpose of measurement is to inform decisions. Before measuring, clearly define the decision at stake and how additional information would affect it. This helps prioritize what to measure and how precisely. Uncertainty and risk. Decisions involve uncertainty, which creates risk. Measurement reduces uncertainty, thereby mitigating risk. Key concepts: Uncertainty: Lack of complete certainty; existence of more than one possibility Risk: A state of uncertainty where some possibilities involve loss or undesirable outcomes Value of information. Not all measurements are equally valuable. Calculate the Expected Value of Perfect Information (EVPI) to determine how much a measurement is worth: Identify the decision and possible outcomes Estimate probabilities and consequences of each outcome Calculate the expected value with and without perfect information The difference is the EVPI, the maximum you should spend on measurement

Success is a function of persistence and doggedness and the willingness to work hard for twenty-two minutes to make sense of something that most people would give up on after thirty seconds. Overconfidence is common. Most people are overconfident in their estimates, providing ranges that are too narrow. This leads to poor decision-making based on unrealistic expectations. Calibration training. Through practice and feedback, people can learn to provide more accurate probability estimates. Techniques include: Equivalent bet test: Compare your estimate to a bet with known odds Consider the opposite: Actively look for reasons why you might be wrong Use reference classes: Compare to similar, known quantities Practice with feedback: Take calibration tests and review results Benefits of calibration. Well-calibrated estimators: Provide more reliable inputs for decision models Are more open to new information and changing…