Reports and Events Archive Scholars and Experts Newsroom Blog About NMRC Home



Keyword Search

E-mail Updates
Get E-mail updates on NMRC publications, events and activities.


print page button
Ideas and Trends for the New Millennium

June 2007
Data Retention Questions and Mandates:
Will Online User Data Storage Requirements Protect Us Or Create More Problems?

From the Executive Director
Matt Bennett, NMRC Executive Director

The New Millennium Research Council (NMRC) continues to explore critical public policy issues that affect how new technologies are deployed, used, monitored, and governed.

In this second edition of NMRC Milestones, we examine data retention mandates and discuss the numerous privacy and security concerns associated with the storage of online user data.   

Data retention mandates require communications services providers and/or governments to store records of email, visited websites, instant message or chat room conversations, and VoIP call logs, among other information sent or received online. The purpose of a data retention system is to keep a detailed record of every individual's digital movement should it be needed for a criminal investigation.  Legislative and legal mandates for data retention are already in place in the European Union, where directives require that communication providers retain certain records for at least six months. Several EU member states mandate data storage for considerably longer periods of time, with Ireland retaining information for three years and Italy for four.

Now, the United States is considering whether to impose data retention mandates.  Attorney General Alberto Gonzales, the Internal Revenue Service, and law enforcement agencies have called for a variety of retention policies and Congress could consider data retention legislation this year. Beyond the policy debate, corporate data retention is already common practice at some of America's biggest firms, including Amazon and Google.

Proposed data retention mandates have generated numerous questions about the privacy and security of the information stored.  Who will have access to the data?  What precautions will communications service providers be required to take to protect the information and at what costs? 

To explore this important and complex issue, including the balance between personal privacy and law enforcement needs, the NMRC sought the opinions of two telecommunications policy and privacy experts.  Hance Haney, a Senior Fellow at the Discovery Institute, questions whether there is a certain degree of futility attempting to track individual digital movement and explores the utility of such records for law enforcement agencies. Peter Swire, a professor of law at the Ohio State University and a fellow at the Center for American Progress raises important points about the security of data retained by Internet service providers or the government. These articles offer two important voices on the issue of data retention and pose critical questions of data retention legislation.

NMRC Milestones was created as an information resource for policymakers, consumers, industry executives, and thought leaders.  As a regularly occurring update, Milestones is a platform for new voices and a discussion vehicle for finding workable policy solutions.

Do Data Retention Risks Outweigh the Rewards?
Hance Haney, The Discovery Institute

Last year when Congress conducted a series of hearings on how to prevent crimes against children on the Internet, Lt. Anthony Ritter of the computer crimes bureau at the New Jersey State Police recommended requiring Internet Service Providers to store customer data for not less than two years.  The data should “include, but not be limited to, subscriber information, method of payment, types of devices connected and all in and out IP logging records,” he added.

Legislation pending in Congress would allow the Attorney General to issue regulations governing the retention of records by ISPs.  A spokesman for the Justice Department stated last year that the actual content that customers look at on the Web would not be stored.  He added that all of the data would be stored by the companies, not by the government, and that the government would have access to the data only by current means, such as warrants and subpoenas.  But the legislation before Congress includes none of these safeguards.  The Attorney General could require anything he wants above and beyond the “name and address of the subscriber or registered user to whom an Internet Protocol address, user identification or telephone number was assigned.”

No one is quite sure how costly mandatory retention would be, but the amount of data that would need to be stored, indexed and formatted would be huge. 

A couple of things are clear.  One is that civil litigants, private investigators, marketers and hackers would seek access to the data, and security breaches would occur.  Another is that there are a number of ways criminals can conceal their online activities.

For example, maintains an encrypted network which allows its subscribers to browse the Web anonymously.  The subscriber’s local ISP can see the connection to the Swedish company’s network but doesn’t have any record of the user’s online activities beyond that point.

There are nearly 46,000 public Wi-Fi access points at cafes, hotels, airports and town squares which anyone can access anonymously.  These hotspots could be modified to require pre-registration and access codes so that network administrators could record both who accesses them and which Web sites are visited.  That would push criminals out into neighborhoods, where wireless networks for home computers are increasingly common and are frequently unsecured.

Will lawmakers see the futility of trying to control offshore proxy servers, encryption software, unsupervised Wi-Fi access and other means of evading data retention, or will mandatory data retention beget more regulation in an effort to ensure its reliability as a law enforcement tool?

Although there was one highly-publicized failure of a Colorado ISP to provide data needed for an investigation – because the ISP didn’t keep the data – law enforcement agencies face unrelated problems that are much more serious.

Congressional inquiries in 2005 and 2006 revealed that over 100,000 sex offenders, or nearly one-fifth of the total were “missing,” meaning that they chose not to comply with sex offender registration requirements.  The median prison sentence for sexual abuse crimes was only 41 months.  Some states do not even have laws making the sexual solicitation of a minor online a felony offense with automatic jail time; nor is the possession of child pornography a felony offense in all 50 states. 

Congress passed the Adam Walsh Child Protection and Safety Act of 2006 to expand the sex offender registry, strengthen federal penalties for crimes against children and to authorize regional task forces to provide training and funding for law enforcement agents.  But scant resources continue to hinder the effort to combat crimes against children online – particularly in the states, where 70 percent of all online child sexual exploitation cases are prosecuted.  In Wyoming, for example, “we have over 250 search warrants we could request if the manpower permitted,” according to Flint Waters, a member of a Task Force.  “Our investigators are averaging over 70 hours per week,” he said at one of the congressional hearings.

The problems of our criminal justice system notwithstanding, parents want to be certain the Internet is safe as possible for their children.  Although nothing can fully replace parental supervision, ISPs and others are working in a number of ways to reduce the potential for online child exploitation.

British ISPs voluntarily block access to Web sites identified by the Internet Watch Foundation as containing images of child pornography, no matter where the sites are hosted.  British officials believe this cooperation partly accounts for the fact that only 0.2 percent of Web sites containing child pornography are currently hosted in Britain — down from 18 percent in 1997.  But instead of taking down U.S. Web sites with child pornography, until recently these sites were left up long enough to allow investigators to gather evidence.  Now some of these sites are taken down immediately – those which are not the subject of a criminal investigation.  This emphasis on gathering evidence and preparing charges for criminal prosecutions may be one reason IWF has found that 51.1 percent of Web sites containing child pornography are hosted in the U.S.

ISPs including AOL, Yahoo, Microsoft and Earthlink record an “electronic signature” of child pornography images that they can use to identify and block the images if they are sent again.  Every piece of e-mail is scanned for files that match the list of signatures, and those that do are forwarded to law enforcement.  Filters and algorithms are also used by ISPs to identify child pornography transmitted over their networks.

Beyond these and other initiatives, most ISPs already retain customer data for at least 60 days.  Once a law enforcement agent sends a data preservation request to an ISP, the ISP is required by law to retain the data for an additional 90 to 180 days. 
On the surface, a government mandate for ISPs to preserve customer data for at least two years sounds like it would make it easier for investigators who are averaging over 70 hours per week to catch hapless criminals.  But given the opportunities for criminals to conceal their identities and the various defects in the criminal justice system, the real danger is that mandatory data retention might give parents and children a false confidence that the Internet is or ever can be a crime-free zone.    

Mandatory data retention won’t eliminate the need for parents to supervise their children’s online activity, nor make it unnecessary for the government to adequately fund the criminal justice system.  Mandatory storage of the online activity of every American would create an irresistible target for troublemakers and potentially subject innocent people to embarrassment, misunderstanding or increase their exposure to identity theft and other crimes.

Is Data Retention Secure?
Peter P. Swire, Ohio State University and the Center for American Progress

Information and national security issues need much more attention while the government decides whether to require data retention by Internet service providers.

Attorney General Alberto Gonzales recently asked major ISPs to store Web traffic information for as long as two years. Although details are still unclear, the attorney general apparently wants to track which Internet addresses contact other addresses in Web surfing, e-mail messaging and chat sessions.

According to Justice Department officials, the proposal’s genesis was a problem that prosecutors faced in tracking child pornography on the Internet. In some cases, investigators were able to determine an IP address involved in the crime. By the time they sought to identify the individual who used that address, however, the ISP no longer retained the record of the Web session. Faced with this lost evidence, Justice is now considering asking for legislation that would require retention of the traffic data.

Press attention to the issue thus far has focused on cost and privacy concerns. The cost to ISPs of storing two years of traffic data for all Americans who are online would be considerable, especially when compared with the small number of child pornography prosecutions, which number in the hundreds per year. In response to that point, Justice officials have discussed expanding use of the retained data to anti-terrorism cases and perhaps considerably further, such as tracking illegal music downloads.

From the privacy perspective, there are significant concerns about how the government -- and others -- might use such a large new dataset on how people communicate via the Internet. The recent reports of warrantless National Security Agency access to telephone calls and call records heighten those concerns.

The government needs to address serious information and national security questions. Although Justice brought only lawyers to the meeting I attended on the subject, federal chief information officers and other technical experts should also play a key role. Consider the potential problems:

  • First, what protections are necessary against inadvertent data breaches? The recent data loss at the home of a Department of Veterans Affairs employee has re-emphasized the message that inadvertent, large breaches will occur. In response, data-retention proposals should consider measures that ISPs must take to reduce the risk of a breach. Greater security comes only with greater expenditures, including for federal systems that would presumably retain data, too.

  • Second, what governmental activities might be compromised by data retention? A major concern, which the attorney general has not addressed, is that the ISPs would also retain traffic information of federal, state and local governments. Classified federal systems, which generally don't use the Internet, may be the exception. Input from federal chief information officers is necessary to resolve this issue. What problems will arise if commercial ISPs hold detailed records for years of all the communications to and from government agencies?

    For example, data retention may increase the risk of the exposure of undercover police and confidential informants. Commercial ISPs would retain detailed records of communications to and from the FBI or the local police department. ISPs would thus become a honeypot for attacks from compromised employees and outside attackers using root kits or other techniques. Organized crime groups, for instance, might find it irresistible to place a spy in the ISPs in cities where they operate. The richness of the database for prosecutors matches its richness for criminals who gain access to it.

  • Third, what are the counterintelligence and national security implications? When commercial ISPs retain many government records, the ISPs become a more attractive target for foreign governments and perhaps terrorist groups. In 2005, the press reported in detail about the Titan Rain attacks, in which it appears the Chinese government hacked into Army and Air Force computers and seized flight-planning software and other military secrets. Looking ahead, large databases of retained information would become a tempting target for foreign adversaries. In light of this risk, the government must address those counterintelligence concerns.

In developing its proposal, Justice should publicly address such information and national security problems. Federal CIOs and other security experts may be able to suggest countermeasures to reduce those security risks. In assessing the overall desirability of data retention, the costs of those countermeasures should be calculated with the costs that private-sector companies and individuals would face if data retention occurs.

In bringing their technical expertise to bear, federal CIOs and other security experts should also consider alternatives to nationwide storage of all Web traffic activity. Current law requires data preservation rather than data retention. That means ISPs must preserve specific data when the government indicates that an investigation is under way. Better technical ways to implement data preservation would reduce costs and privacy concerns in addition to the information and national security problems.

This article was originally published in Federal Computer Week.

For comments, suggestions, or submissions for future NMRC Milestones editions, please contact Matt Bennett at

New Millennium Research Council, 2007. All rights reserved.

Contact | Links of Interest | Internships

© 2004 All Rights Reserved New Millennium Research Council