You Can’t Manage What You Can’t Measure

The numbers don’t always tell the whole story by themselves, and it’s up to analysts – or anybody who has access to the numbers – to discover the meaning hidden in the digits.

There’s a very interesting story about how understanding the implications of a data set made a real difference in a real world situation.

During the Second World War, aerial bombing was a major component of the Allied strategy. Bomber aircraft had to fly straight and level when approaching their targets to have a hope of hitting anything, but this made them relatively easy pickings for fighters and anti-aircraft guns. Consequently, Allied aircrew losses were dreadfully high.

But you can’t put armor all over a plane; it will quickly be too heavy to get off the ground. There’s a reason tanks don’t have wings!

With this in mind, the Center for Naval Analyses studied damaged bombers to see where armor was most desperately needed.

A researcher named Abraham Wald, created a simple diagram of a plane and shaded in the areas where returning bombers showed damage. The thinking by military planners was that the damaged areas identified where the aircraft should be better armored.

The problem: Pretty much all of Wald’s diagram was heavily blackened, so they were back to the problem of getting a tank to fly.

Then Wald showed what human insight can do. The data came from aircraft returning from battle. He reasoned; the damage by definition was not fatal to the aircraft, even though some crew members still died. But if a plane went down, the entire crew and the aircraft was lost.

Wald flipped the model and looked at the undamaged parts of the bombers. He reasoned that those were the important components. Armoring these pristine areas, Wald argued, would improve the rate of survival. He was right, and because he refused to limit himself to visible figures and conventional ideas, more Allied bomber crews made it back.

In other words, it’s not just the people with rifles who win wars. Nerds do, too.

The sort of thinking that Wald applied to the bomber problem is a great example of looking beyond the data. Sometimes the holes we don’t see are the most damaging, and realizing that requires an understanding that goes beyond a list of numbers.