Defense News just posted an article by Jim Hendler and me about the need for the intelligence community to embrace cutting edge information technology that can help analysts process the enormous quantities of information collected and connect the dots.
Jim has been working on this stuff for over a decade and when I worked with him at UMIACS where we built an online portal to highlight this potential.
Info Systems Must 'Connect Dots' on Terrorism
By AARON MANNES and JAMES HENDLER
Published: 19 April 2010
The near success of the Christmas Day bomber on a flight from Amsterdam to Detroit has been characterized as a failure to "connect the dots." Now, additional teams of specialists are being formed to run down clues.
Throwing resources at a problem is an understandable knee-jerk reaction, but human capacity to absorb and process enormous quantities of information quickly is limited. Connecting the dots requires cutting-edge information technologies that augment human capabilities. Without revamping the information systems used in the intelligence community, more eyeballs will, at best, yield diminishing returns and, at worst, exacerbate problems and increase stovepiping.
The American intelligence community has invested heavily in building unparalleled tools for collecting information - generating dots - while slighting tools to connect them. The most commonly reported weakness in the information systems used in the intelligence community is that analysts have to search multiple databases to access all available information and that they cannot integrate the data they find from their searches.
Although technologies to amalgamate databases exist, they have not been widely deployed within the intelligence community. Giving intelligence analysts the same search capabilities that are commonplace for most Web surfers is necessary, but on its own will not help "connect the dots."
Since 9/11, intelligence-sharing has improved, but the data deluge caused by increased collection capabilities has negated these gains. Data is now measured in the petabyte, 1 million gigabytes, roughly equivalent to 20 million four-drawer filing cabinets filled with text.
A public search engine might process many petabytes of data daily, and presumably the intelligence community collects data on a comparable scale. Without knowing what to look for, searching these enormous quantities of data will only increase information inundation. An effective system not only helps analysts find information, it helps them make connections.
A search on variants of the Christmas bomber's name, Umar Farouk Adbulmutallab of Nigeria, would have resulted in hundreds or thousands of responses. One person could not examine this volume of material in a timely manner. Even if a team of analysts exam-ined the material in a reasonable time frame, without knowing what to search for in advance, team members might focus on different issues and not connect the dots.
Information systems that can draw basic conclusions, such as indicating probable links, would help overwhelmed analysts identify critical information. This is not the stuff of science fiction. Businesses are investing in the Semantic Web, in which information is given some context by facilitating smart or semantic searches. This approach is just starting to be embraced by industry, with some search engines, such as Microsoft's bing.com, starting to link searches together.
An information system with these capabilities might understand Umar Farouk Abdulmutallab not only as a string of characters but as a person who possesses certain inherent characteristics, such as parents and a last known location. Nigeria and Yemen can be understood as places where a person might be present at a given time. With this context, the info system could automatically identify disparate warning signs as potentially relating to the same person.
Despite the tremendous efforts of intelligence-community analysts, the current ways of doing business have proved inadequate. While there is no guarantee that better information technology would have allowed U.S. intelligence to act before Abdulmutallab boarded the plane, increasing personnel represents more of the same. Adopting revolutionary IT could be a much-needed game-changer.
Ironically, many of the technologies underpinning these revolutions in information systems were funded by the Defense Advanced Research Projects Agency, and were motivated by the need to enhance data-sharing in the intel community. Now, the Semantic Web is used by governments to increase the public's access to data and by businesses to connect the dots. But efforts to deploy this technology within the national security system have moved more slowly.
There are understandable reasons why adopting new information systems have not been prioritized. The infrastructure for building and launching an intelligence-collection satellite is proven and the additional data gathered is easily measured. But it can be difficult to quantify the impact of a new information system and its implementation is an enormous technical and bureaucratic challenge.
But without better information systems, the intelligence community will be hamstrung in its efforts to transform information into intelligence while America's enemies continue to systematically identify and exploit these blind spots. To connect the dots and prevent the next incident before it happens, the intelligence community needs to implement technologies that were specifically developed to address these challenges.
---
Aaron Mannes is a researcher at the University of Maryland's Laboratory for Computational Cultural Dynamics. James Hendler is a professor at Rensselaer Polytechnic University, N.Y., and a former chief scientist for information systems at the U.S. Defense Advanced Research Projects Agency.
