The Declassification Engine: What History Reveals About America's Top Secrets

SHORTLISTED FOR THE CUNDILL HISTORY PRIZE • Every day, thousands of new secrets are created by the United States government. What is all this secrecy really for? And whom does it benefit?

“A brilliant, deeply unsettling look at the history and inner workings of ‘the dark state'.... At a time when federal agencies are increasingly classifying or destroying documents with historical significance, this book could not be more important.” —Eric Schlosser, New York Times best-selling author of Command and Control

Before World War II, transparent government was a proud tradition in the United States. In all but the most serious of circumstances, classification, covert operations, and spying were considered deeply un-American. But after the war, the power to decide what could be kept secret proved too tempting to give up. Since then, we have radically departed from that open tradition, allowing intelligence agencies, black sites, and classified laboratories to grow unchecked. Officials insist that only secrecy can keep us safe, but its true costs have gone unacknowledged for too long.

Using the latest techniques in data science, historian Matthew Connelly analyzes a vast trove of state secrets to unearth not only what the government really did not want us to know but also why they didn’t want us to know it. Culling this research and carefully examining a series of pivotal moments in recent history, from Pearl Harbor to drone warfare, Connelly sheds light on the drivers of state secrecy— especially incompetence and criminality—and how rampant overclassification makes it impossible to protect truly vital information.

What results is an astonishing study of power: of the greed it enables, of the negligence it protects, and of what we lose as citizens when our leaders cannot be held to account. A crucial examination of the self-defeating nature of secrecy and the dire state of our nation’s archives, The Declassification Engine is a powerful reminder of the importance of preserving the past so that we may secure our future.

MATTHEW CONNELLY is a professor of international and global history at Columbia University, codirector of its social science institute, and the principal investigator at History Lab, a project to apply data science to the problem of preserving the public record and accelerating its release. He received his BA from Columbia and his PhD from Yale. His previous publications include A Diplomatic Revolution: Algeria’s Fight for Independence and the Origins of the Post–Cold War Era and Fatal Misconception: The Struggle to Control World Population.