Wikipedia vs Britannica tested

In Wikipedia vs. Britannica Smackdown, Ed Felten takes my challenge. In the meanwhile, I’d done some hypothesizing, here.
So how’d I do?
Hypothesis 1 is spot on.
#2 is more challenging to assess: The errors in Britannica are smaller, and I think I’ll judge myself wrong.
#3 I think is accurate, if only because of the long entry on Microsoft.
#4 Ed did not assess, or comment on.
#5 Ed didn’t check Encarta.
So, I think I get 2 out of 3 for the tested hypothesis.

Wikipedia vs Britannica

A few days ago, I challenged Ed Felten to do some more comparison work. In the spirit of Milgram, I didn’t propose a theory. (This was mostly because I was trying to make a good joke about assigning the professor homework, but couldn’t come up with one.) However, on consideration, I think that I should propose some theories, and also not influence the experiment.

So, hypothesis 1:
Wikipedia will have 30-50% more entry coverage than the others.
In particular, I don’t expect Ed Felten will have an entry, and I
expect one of his two computer science entries to not be in each
comparison encyclopedia.

Hypothesis 2:
The quality of Wikipedia, measured by errors detected, will meet
that of the others.

Building a large encyclopedia is a lot of work, and I don’t expect that the quality assurance and fact checking will be great anywhere.

Hypothesis 3:
The quality of Wikipedia, measured by the depth of the entries,
will be substantially greater than the comparison.

Techies aren’t noted for brevity and conciseness, and the web doesn’t
have physical constraints holding down the size of the entries,
whereas each DVD you ship may add $2 to the cost of a product. I
expect that the difference would be largest against the print or CD
versions.

Hypothesis 4:
The quality of Wikipedia, as measured by the accessability of
entries, will be lower.

By accessability, I mean how good the
basic introduction and contextualization are, and how well the entry
takes you from no knowledge to some.

Hypothosis 5:
Ed will believe that Encarta’s entry on the Microsoft trial is
biased towards Microsoft.

Analysis:

An encyclopedia must be measured first on accuracy, and secondly on
breadth. A roomful of monkeys writing entries does not get you a
useful encyclopedia, but neither does one with one entry. (There are
a great many useful topical encyclopedias which address this by
constraining themselves to one subject.

I expect that Wikipedia’s accuracy will be roughly that of the others,
and it will win, hands down, on breadth and depth. However, this test
is biased by the selection of terms, where they are known to a
computer science professor. If my hypotheses pan out, it would be
fascinating to see if we could recruit from across the Princeton
faculty, to see if the same tests hold true across wider disciplines.

(I did two short tests, on Rabbi Akiba, and Brillat-Savarin.
Wikipedia spells it Akiva. But I
don’t have a comparison document to compare to.)

Science is easier from the outside

As part of a larger project on security configuration issues, I’m doing a lot of learning about taxonomies and typographies right now. (A taxonomy is a hierarchical typography.)
I am often jealous of the world of biology, where there are underlying realities that can be used for categorization purposes. (A taxonomy needs a decision tree. Any trained person using this tree should classify the same items the same way.)
A new type of shark has recently been discovered, in the Sea Star Aquarium, in Coburg, Germany. This is (at least) the second zoo that the shark has been in.
We are not embarrassed,” said [Schonbrunn Zoo] spokesman Dr Ekkehard Wolf. “We get thousands of exotic animals every year. It is not possible to categorize them all. (From The Telegraph.)
See a picture (and read the article) at Unterwasser.de or read Google’s translation
Even the lucky biologists run into difficulty classifying their species. I feel better trying to classify minimum time between password changes.

Airline Security

In Educated Guesswork, Eric Rescorla writes about one way tickets and the search criteria.
The CAPPS program was created by Northwest airlines, who set the criteria for inclusion. They included one way tickets to enforce their bizarre pricing schemes. This is the same reason they started asking for ID: to cut down on the resale of the other half of a round-trip ticket, which cost the same as a one-way.
CAPPS, incidentally, has been renamed the “free wheelchairs for paraplegic children” program, to make it harder to argue against, and to get around a congressional mandate that the program not be deployed until someone actually thinks it through.
In his comment, Kevin Dick gets it mostly right–there are other items that you want to keep off the planes (pepper spray, for example), but the right technique in a free society involves enabling passengers to fight for their lives, and fortifying the flight deck. There’s a lot that could be done that hasn’t been. For example, consider an “airlock” system, with two doors at the front of the plane, with a restroom inside. The doors open one at a time. There may be an air marshall inside. (A curtain prevents anyone from seeing.) Now hijackers need to get through two doors. They can’t storm the cockpit while the pilots are being fed or using the restroom. This is very expensive. It would require a new bathroom for the high-revenue business travelers up front. It makes a section of plane unusable for reveune generation. But it is entirely free of civil liberties implications for fliers.

Wikipedia

Over at Freedom To Tinker, Ed Felten writes about the Wikipedia quality debate.
He takes a sampling of six entries where he’s competent to judge their quality, and assesses them. Two were excellent, one was slightly inaccurate, two were more in depth, but perhaps less accessible than a standard encyclopedia, and one (on the US Microsoft anti-trust case) was error-prone.
Ed writes: “Until I read the Microsoft-case page, I was ready to declare Wikipedia a clear success.” However, I think his experiment is only one-third to one-half done. I think that Ed ought to look up the same 6 entries in another encyclopedia or two, and report back. I’d suggest the Britannica, which is usually considered the gold standard, and perhaps Microsoft’s Encarta, which may be the most widely used.
I can’t do this experiment the way Ed can, because firstly, I don’t have an EB account, and second, because I don’t know all the topics to the depth he does. I could pretend, and perhaps miss errors that he’d catch, or sample six other articles, and perhaps I will over the weekend.

Lock 'em up!

Over at TaoSecurity, Richard writes:

Remember that one of the best ways to prevent intrusions is to help put criminals behind bars by collecting evidence and supporting the prosecution of offenders. The only way to ensure a specific Internet-based threat never bothers your organization is to separate him from his keyboard!

Firstly, I’m very glad that the second, qualifying sentence is there. It provides some context. However, I’m not sure that I care that a specific threat stops, what I care about is that the class of threats go away.
If the odds that a specific criminal hacker goes to jail are low, then the penalties need to be exceptionally severe and well publicised to create a deterrent effect. (This is roughly a criminal attack loss expectancy, which someone smart has done work on.)
We can see that the odds that an attacker goes to jail are relatively small because there is clearly a large attacker population, and very few criminal sentencings. I’m curious how many attacker convictions we’d need each year to change the economics of this and deter 15 year olds from bringing down CNN?

The Man Who Shocked the World

I’ve recently finished The Man Who Shocked the World, a biography of Stanley Milgram. The book’s title refers to the “Authority Experiments,” wherein a researcher pressured a subject to deliver shocks to a victim. The subjects of the experiments, despite expressing feelings that what they were doing was wrong, were generally willing to continue.
Other work Milgram did lead to the “six degrees of separation” meme, insight into mental maps of cities, the “lost letter” technique of assessing public opinion, and the concept of the “familiar stranger.” He was outstanding at creating illuminating experiments in social science.
I learned in reading this book that Milgram had enormous difficulty getting grants. The review committees who essentially gatekeep over government grants wanted him to work from a theory. (Its not clear from the book if they thought research should support a theory, or correctly understood that great research involves undercutting a theory.)
There’s an interesting tie to computer security here, in that there is a group of researchers who do nothing but interesting experiments, whose results and replicability are shared through what is variously called demonstration code, “POC” (proof-of-concept), or “sploit” (short for exploit) code. Many of these researchers use pseudonyms in their publication, and are considered annoying by the computer security establishment (both commercial and academic), whose work they poke holes in.
In contrast, I think these researchers do an important service by demonstrating how security can be broken. If you consider the hypothesis “This software is resistant to attack,” a few bytes of exploit code is an elegant refutation.

Unrecoverable Damage?

I’m reading through NIST SP-800-70 (pdf), the NIST guide to producing security configuration guides. Let me get more coffee before I continue. Thanks for waiting.

“If home users and other users without deep security expertise attempt to apply High Security checklists to their systems, they would typically experience unwanted limitations on system functionality and possibly unrecoverable system damage.”

Can someone explain to me how you can break a system that badly? I mean, sure, it can be hard to get a new boot block, or a new kernel in place, but once you do, you can recover things.
I’m very down on a system message that implies that modifying your computer can cause unrecoverable damage. It inherently inhibits tinkering, perhaps even more than laws do. After all, we see how effective laws against sharing music or drugs are. But scaring someone into not touching that config file with the threatened loss of all their data? There’s a security measure for you!

Lewis Carroll

Or, if you prefer, the original can be found elsewhere. It’s always nice when things I want to abuse like that are in the public domain. (Obligatory Lessig link.)
But beyond that, think how much poorer literature in the computer science field would be if we didn’t have Alice In Wonderland to freely quote from, adapt, and re-imagine.
On the other hand, I think we might have ended up with Adam and Bob talking instead of Alice and Bob (pdf). (For both non-cryptographers in the audience, very early in the public academic study of cryptography, the paper that introduced the RSA system used “Alice” and “Bob” to represet the two people trying to communicate in secret. Alice and Bob, and their ongoing attempts to have a conversation, plot a rebellion, communicate while in jail, and play poker long distance.)