Are We Measuring the Right Things?
One of the reasons that airline passengers sit on the tarmac for hours before takeoff is how the FAA Department of Transportation measures “on time departures.” The on time departure is measured by push-back from the gate, not wheels leaving the tarmac. (Airlines argue that the former is in their control.) If you measure the wrong things, you create incentives for bizarre behavior.
Which is why I was fascinated to read the new GAO report, “Information Security: Although Progress Reported, Federal Agencies Need to Resolve Significant Deficiencies.”
While progress may be reported, PogoWasRight calls out:
The number of security breaches on government computers has quadrupled in the last 2 years – from just over 3,500 in fiscal 2005 to just over 13,000 in fiscal 2007.
If that’s progress, maybe we need some regression?
More seriously, I think it’s great progress that we are talking about the failure rates. Now we need to start to question the things being measured that allow the GAO to summarize that state of affairs as progress.
I wonder, where else are we measuring the wrong things?
[Update: I was measuring the wrong agency.]
The FAA measures its portion of the on-time system that it controls on the ground – – from the moment we tell an aircraft to back off from its gate until it departs off the runway. If an aircraft initially sits at a gate past its scheduled departure time, we do not keep track of this data.
However, the DOT, in its monthly Consumer Report, DOES keep track of all portions of the on-time data and makes this information public.