Genome Biology volume 4, Article number: 108 (2003)
It was Hiawatha Bray, a columnist for The Boston Globe who writes about information technology, who broke the story, at least in my town. In a column on 3 February this year, he recounted what happened to the Litchfield brothers, a pair of British computer security experts whose company, Next Generation Software, specializes in developing programs to help businesses and governments defend against computer viruses and hackers. Mark and David Litchfield are experts at finding weaknesses in widely used software, and last year they found a bug in Microsoft's SQL Server database software. To establish that the bug could be used to cripple computer networks, David Litchfield wrote an exploit program, a kind of dummy virus. The existence of the bug, a buffer overflow problem, was then communicated to Microsoft, which published a fix in July. Other security experts had already been asking for copies of the exploit program so they could test their own systems and become familiar with this new potential method of attacking networks, and David Litchfield eventually released his code, but only after the patch to fix the program had been published. He thought, quite reasonably, that once the fix was available, SQL Server users would no longer be vulnerable.
He was wrong. On the last weekend in January, the Slammer worm struck. Millions of computers were attacked, and the entire worldwide web was crippled for some time. Businesses are estimated to have lost tens of billions of dollars. It seems that many corporations and individuals hadn't yet bothered to install the patch. And David Litchfield discovered, to his horror, that his code was used as the template for the worm. To be sure, any good programmer could have worked out the code without help, but because of the Litchfields' well-meaning publication of their exploit program, that hadn't been necessary. Now Mark and David Litchfield must decide whether they will ever again publish or distribute their exploit programs. They have received hundreds of e-mails from colleagues begging them to continue releasing such code, because many in the computer business believe that the best way to deal with such weaknesses in widely disseminated programs is to publicize them and make examples widely available. But these colleagues don't have to live with the feelings of responsibility that the Litchfields now have. So from now on, every bug they discover - and they've already found others, in Microsoft's Windows XP and 2000 operating systems - will call for their judgement.
A similar judgement call was made a few weeks ago by the editors of a number of life-science journals, who released a statement of a new policy regarding the publication of reports of scientific research in areas such as microbiology and genomics that could potentially be of use to bioterrorists. The policy, which has received widespread attention, permits the journals to request that experimental details be omitted from papers if, in the view of the editors or a panel of experts (depending on the journal), the information could be misused. In some cases, publication of the work could be embargoed altogether.
The policy has been sharply criticized. The arguments against it would be familiar to the Litchfields: that any form of censorship runs contrary to the spirit of science; that it creates a slippery slope leading to government control; that free exchange of ideas and data represents the best way to anticipate possible misuses of science and technology and to generate the methods to counter them.
The demand for freedom of inquiry and the open exchange of information always wars with the demand for security. While recognizing that absolute security is impossible, governments have a legitimate right - and a duty - to take reasonable steps to protect the lives of their citizens. In a repressive society, such steps are merely part of the general curtailing of liberty. But in a free society the trick is to strike a balance between protection and oppression. There is always a danger, as the witch-hunting excesses of the Cold War remind us, that one can destroy a free society in the name of saving it.
I am completely in agreement with those who worry that this danger is near. The US government has recently shown disturbing tendencies to ignore the fundamental freedoms of its citizens for the sake of what many of us believe is a false sense of security. Other governments are unlikely to resist the temptation at least to control the public's access to information, a goal that many in power have secretly harbored for years.
But I am not in favor of doing nothing. At the heart of many of the objections to the policy of the journal editors, I believe, is a basic sense that the corruption of biology for evil purposes is unlikely. I recognize, and share, the concerns of my colleagues that even self-censorship, as a concept, sets a dangerous precedent. But I wonder if many scientists - who after all tend to have a mostly positive view of human nature as a result of largely associating with other scientists, a class of humanity not known for crimes or acts of violence - have a realistic perspective on the existence of evil.
I once taught a course in the social history of the detective story to a class of extremely bright, well-read college freshmen. As part of the discussion one afternoon, I asked them if they thought that evil existed. Almost to a person, they argued that real evil was a literary abstraction. Historical examples I offered were dismissed as illustrating madness, not evil. The notion that someone could be technically sane yet delight in human suffering and death was something they were neither prepared nor willing to accept. I suspect that many scientists may feel the same way.
Professional writers, whose careers depend on understanding the human condition, tend to be less starry-eyed. Two noted science-fiction authors have explored this subject in stories that are eerily similar. The older story, 'The Supreme Moment', was unpublished during the lifetime of its author, Robert E. Howard, best known for his fantasy tales of Conan the Barbarian. It was eventually published in 1984 in Crypt of Cthulhu magazine #25 and reprinted in The New Howard Reader #1, edited by Joseph W. Marek. The story concerns five wealthy, powerful men who are trying to convince a crippled scientist to save the human race from a fungus that is spreading across the earth, destroying all vegetation. The scientist, Zan Uller, knows how to make a fungicide but refuses to reveal the formula. As a justification, he explains that he had a tormented childhood and a career bedeviled by sabotage from rival scientists, persecution by religious fanatics, and ridicule for his discoveries. His five visitors threaten to force him to reveal the formula by torture, believing it justified to save the planet. Before they can act, to forestall them and to take his vengeance on a world that has given him nothing but misery, Uller commits suicide.
'Judgement Day', by L. Sprague de Camp (who, interestingly, was chosen to complete Howard's unfinished Conan stories after the latter's early death), was published in the August 1955 issue of Astounding Science Fiction magazine and later reprinted by Ballantine Books in The Best of L. Sprague de Camp. A physicist, Wade Ormont, has developed a formula for a nuclear reaction using iron, a cheap, widely available material. If he publishes the details of his discovery, he realizes that the probability is high that someone will eventually use it to destroy the world. Although he doubts that the US government would do that he is convinced that if he turns over his information to them it will eventually become more widely known: as the theft of the atom-bomb secrets proved, nothing can be kept hidden forever. In thoughts that chillingly echo our concerns about rogue states and terrorist organizations with weapons of mass destruction, Ormont decides that sooner or later some 'crackpot' head of state will use this capability to wreak havoc on a planet-wide scale. He then reflects on his own life history. Like Uller in The Supreme Moment, he had a childhood filled with physical and verbal abuse. His marriage failed and most of his other human contacts have been brief and hostile. An attempt at therapy quickly ended when he resented the psychiatrist's description of his personality as schizoid. Now in his mid-50s, enfeebled by a heart condition and with little will to live, he has become thoroughly misanthropic. He guesses that even if he publishes his formula in an obscure place, it would be discovered and used by a madman within a decade or two, and it is unlikely that he would live to see the end of the world. Finally, he reaches a decision: "There is one way I can be happy during my remaining years, and that is by the knowledge that all these bastards will get theirs someday... I hate everybody... I shall write my report."
Both Howard and de Camp recognize that there can be people who are capable of great acts of malice and vindictiveness. It used to be argued that no one would be likely to use a chemical, biological or nuclear weapon since to do so would lead to far too great a risk of the user's own destruction, either through retaliation or the failure to control the damage from one's own use. The advent of suicide bombers and eschatological cults like Aum Shinrikyo should dispel that notion. There are people who would not blanch at the end of the world, or at least the destruction of large parts of it, and who hold their own lives (or, more commonly, the lives of their followers) very cheaply. Some of them are neither so insane as to be incapable of cunning nor so technically inept as to be unable to adapt 'peaceful' discoveries to their own ends. It is true that most of their activities up to now have been of the low-tech kind, and they probably will continue to be for the near future, but it is indisputable that at least some of them have tried to obtain or develop biological weapons, and it seems certain that such attempts will continue.
"We often forget that our actions...can have very real consequences in real life," David Litchfield wrote shortly after the Slammer incident. I think the biological community has to face the fact that software experts - and physicists and chemists - are no longer alone in their nightmares over the possible misuse of their discoveries. Until human evolution, ethically speaking, catches up with technological evolution, we will all have to live with this possibility. The policy adopted by the journal editors seems to me a wise attempt to seize the initiative from those in government who would use the public's increasing fear of biology as a license for repressive control of scientific research and publication. As a community, I think we should adhere to this policy for the time being while continuing to debate its merits and considering alternatives. During these discussions, we will no doubt also be asking ourselves how we would feel, and what the consequences to our profession would be, if one of our publications were to form the blueprint for a terrorist act. If we have trouble imagining the answers, Mark and David Litchfield do not.
Rights and permissions
About this article
Cite this article
Petsko, G.A. Judgement call. Genome Biol 4, 108 (2003). https://doi.org/10.1186/gb-2003-4-4-108
- Free Society
- Suicide Bomber
- Security Expert
- Judgement Call