Shostack + Friends Blog Archive


Fanning the flames, security metrics style

Amidst the to and fro over insider v. outsider threats, whether security metrics can be “gamed”, and so on, and in recognition of the best buddies that security geeks and economists have now become, I offer the following. 

The saying often quoted from Lord Kelvin (though the substance, I believe, is
much older) that “where you cannot measure your knowledge is meagre and
unsatisfactory,” as applied in mental and social science, is misleading and pernicious.
This is another way of saying that these sciences are not sciences in the sense
of physical science, and cannot attempt to be such, without forfeiting their
proper nature and function. Insistence on a concretely quantitative economics
means the use of statistics of physical magnitudes, whose economic meaning and
significance is uncertain and dubious. (Even “wheat” is approximately
homogeneous only if measured in economic terms.) And a similar statement would
apply even more to other social sciences. In this field, the Kelvin dictum very largely means in practice, “if you cannot measure, measure anyhow!” That is,one either performs some other operation and calls it measurement or measures something else instead of what is ostensibly under discussion, and usually not a social phenomena. To call averaging estimates, or guesses, measurement seems to be merely embezzling a word for its prestige value.

Frank H. Knight, “‘What is Truth’ in Economics?”
The Journal of Political Economy
Vol. 48, No. 1, Feb. 1940, 1-32.