DECISION AND ORDER STAYING FURTHER DOCUMENT PRODUCTION
BERMAN, District Judge.I. Background
This Decision and Order addresses pending discovery disputes in Plaintiff subway riders’ action, under 42 U.S.C. § 1983, to invalidate the New York City Defendants’ program, begun on July 21, 2005, to randomly search subway riders’ backpacks and other containers in an effort to detect and deter terrorism (“Subway Search Program”).
On August 12, 2005, Defendants moved for a protective order to shield from discovery documents and information relating to the frequency and location of searches under the Subway Search Program. (See Defendants’ Memorandum in Support of Application for Protective Order, dated Aug. 12, 2005, at 3-4 (“The documents at issue fall squarely within” the law enforcement privilege, and “Plaintiffs cannot demonstrate sufficient need to overcome the privilege.”); Plaintiffs’ Memorandum Opposing Motion for Protective Order, dated Aug. 15, 2005, at 1-2 (“defendants fail to explain adequately how disclosure ... would imperil law enforcement procedures,” and the information sought “is central to the issues in this case,” is “already ... publicly known,” and “can be subject to an appropriate confidentiality order”).)
On August 18, 2005, United States Magistrate Judge Frank Maas, to whom this matter had been referred, issued an order (“Discovery Order”) directing that three categories of documents and information be released by the City, namely: (1) “the number of days between July 21 and August 21, 2005, on which no searches have been conducted”; (2) if searches were conducted, “the number of subway stations each day where no searches were conducted”; and (3) “the aggregate number of subway entrances that went ‘unsearched.’ ” (Discovery Order at 9-10.) Judge Maas developed a sampling mechanism for categories 2) and 3) and directed that information be disclosed “for attorneys’ eyes only.”
On August 23, 2005, Defendants submitted objections to the Magistrate Judge’s Discov*381ery Order (“Defendants’ Objections”), arguing that Judge Maas had abused his discretion by: (1) holding that the discovery was relevant to the “effectiveness” of the Subway Search Program; and (2) failing adequately to “weigh the harm of disclosure” to New Yorkers. (Def. Obj. at 5-12.) On August 24, 2005, Plaintiffs responded to Defendants’ Objections and submitted objections of their own (“Plaintiffs’ Objections”) asserting that, among other things: (1) Judge Maas had abused his discretion by limiting production to “sample” data; and (2) Judge Maas’s Discovery Order “does not address at all the issue of the discoverability of information about the number of searches conducted under the program.” (PI. Obj. at 1-2.)
On August 23 and 24, 2005, Defendants submitted for in camera review by this Court the disputed documents and information. On August 24, 2005, the Court heard oral argument from the parties. (See Transcript of Conf., dated Aug. 24, 2005 (“Tr.”).) Three areas of discovery remain in dispute, as follows: (1) the number of subway stations where no searches were conducted on days (and at times) when searches were conducted; (2) the number of station entrances where no searches were conducted on days (and at times) when searches were conducted; and (3) the number of searches conducted on a daily basis. (Tr. at 3-18.) Without waiving its claim of privilege, at the August 24, 2005 conference the City disclosed that subway searches were conducted every day during the period July 21 through August 21, 2005. (Tr. at 3-4, 14.) On August 25, 2005, the parties submitted letters reiterating arguments made at the August 24, 2005 conference.
For the reasons set forth below, the Discovery Order is suspended and no further documents need be disclosed by Defendants pending the hearing on September 7, 2005 at 9:30 a.m.
II. Standard of Review
When considering objections to a discovery order issued by a magistrate judge, the Court “shall modify or set aside any portion of the magistrate judge’s order found to be clearly erroneous or contrary to law.” Fed.R.Civ.P. 72(a); see 28 U.S.C. § 636(b)(1)(A); Thomas E. Hoar, Inc. v. Sara Lee Corp., 900 F.2d 522, 525 (2d Cir. 1990); United States v. Premises Known as 281 Syosset Woodbury Road, 862 F.Supp. 847, 851 (E.D.N.Y.1994), affd, 71 F.3d 1067 (2d Cir.1995). “A finding is ‘clearly erroneous’ when although there is evidence to support it, the reviewing court on the entire evidence is left with the definite and firm conviction that a mistake has been committed.” United States v. United States Gypsum Co., 333 U.S. 364, 395, 68 S.Ct. 525, 92 L.Ed. 746 (1948).
“The purpose of [the law enforcement] privilege is to prevent disclosure of law enforcement techniques and procedures, to preserve the confidentiality of sources, to protect witness and law enforcement personnel, to safeguard the privacy of individuals involved in an investigation, and otherwise to prevent interference with an investigation.” In re Department of Investigation of City of New York, 856 F.2d 481, 484 (2d Cir.1988). The privilege is “qualified, however, and must be balanced against the litigant’s substantial need for information____” Otterson v. National R.R. Passenger Corp., 228 F.R.D. 205, 207 (S.D.N.Y.2005). “To sustain the invocation of such a privilege ..., [Defendants] must make a clear showing of harm if the information is disclosed.” Galvin v. Hoblock, No. 00 Civ. 6058, 2003 WL 22208370, at *3 (S.D.N.Y. Sept.24, 2003).
III. Analysis
Based upon the parties’ submissions and the Court’s review of the disputed material in camera, the Court agrees with Magistrate Judge Maas to the extent he found that the disputed documents and information are covered by the law enforcement privilege. (Discovery Order at 9 (Court’s “challenge” is to “devise a means by which the plaintiffs can pursue their claims without unreasonably jeopardizing the secrecy that the City claims lies at the heart of the program’s effectiveness.”).) At the same time, and for the reasons that follow, the Court concludes it was error to order further disclosure at this time.
First, given the sensitive nature of the documents at issue — which reflect and detail law enforcement techniques and proce*382dures — an in camera review should have been conducted by the Magistrate Judge before ordering disclosure. Such review is “a practice both long-standing and routine in cases involving claims of privilege.” In re Grand Jury Subpoenas Dated March 19, 2002 and August 2, 2002, 318 F.3d 379, 386 (2d Cir.2003) (collecting cases); see Gould, Inc. v. Mitsui Mining & Smelting Co., 825 F.2d 676, 680 (2d Cir.1987).
Second, assuming arguendo that disclosure was warranted, the Magistrate Judge failed to explain how the proposed sample of information was derived and/or how it accurately could reflect the data sought by Plaintiffs. {See Declaration of Christopher Hansen, dated Aug. 24, 2005; PL Obj. at 12 (because Judge Maas “does not explain why or how he picked [the sample] dates ... it is not possible to draw conclusions about whether the ... dates represent a statistically valid sample”).)
Third, it seems premature (and unnecessary) at this stage to conclude, as Judge Maas apparently did, that the documents sought are relevant to the inquiry at hand. (Discovery Order at 6.) Given the dual objectives of the Subway Search Program— i.e., detection and deterrence of terrorists— and the fact that the Court has not yet heard from the public officials and other experts who design these kinds of programs, it serves little purpose to generate (even for attorneys’ eyes only) sensitive data and details identifying where searches are not conducted and the number of searches conducted daily.1 The City may be able to demonstrate that the Subway Search Program effectively deters terrorism precisely because it is random and unpredictable. {See Affidavit of Deputy Police Commissioner David Cohen, dated Aug. 12, 2005 (“Cohen Aff.”), 111117-18 (“[P]laintiff has stated that the number of inspections is vital to the court’s consideration of the effectiveness of the program. This seems to assume that there is a threshold number of stations that must be inspected before there is any deterrent effect .... This view ignores the role that unpredictability plays in creating a deterrent.”).) The Supreme Court has counseled against engaging in “searching examinations” of the effectiveness of such programs. The “choice among ... reasonable alternatives remains with the governmental officials who have a unique understanding of, and a responsibility for, limited public resources .... ” Michigan Dep’t of State Police v. Sitz, 496 U.S. 444, 453-54, 110 S.Ct. 2481, 11.0 L.Ed.2d 412 (1990); see also United States v. Green, 293 F.3d 855, 862 (5th Cir.) (It is “common sense” that “[shopping vehicles at regular intervals, rather than every one, ... reasonably advances the purposes of the checkpoint because it deters individuals from driving while unlicensed and or transporting weapons and ... provides a gauntlet, random as it is, that persons bent on mischief must traverse.”), cert. denied, 537 U.S. 966, 123 S.Ct. 403, 154 L.Ed.2d 324 (2002); United States v. Davis, 270 F.3d 977, 982 (D.C.Cir.2001) (“statistical evidence is not ... essential” in proving the “effectiveness of the checkpoint,” which “may be demonstrated in other ways” (citation omitted)); United States v. Hernandez, 162 F.3d 1174 (table), 1998 WL 764482, at *2 (10th Cir. Oct.28, 1998).
Fourth, the Magistrate Judge’s Discovery Order does not appear adequately to have assessed the unrebutted affidavit of Deputy Police Commissioner Cohen which argues that “[unpredictability introduces uncertainty into both the planning and execution of terrorist attacks ---- The bag inspection program introduces uncertainty by withholding [the disputed discovery] from public dissemination .... The success of the program vitally depends on maintaining that secrecy .... Were the court to require such disclosure, the efficacy of the program’s deterrent effect would be seriously undermine[d]----” (Cohen Aff. 1HI14-16.) That is, in determining whether privileged material should be disclosed, courts must carefully weigh defendants’ interest in- non-disclosure “against [plaintiffs’] substantial need for information.” Otterson, 228 F.R.D. at 207. Simply put, there does not at this time appear to be a “substantial need” for disclosure of the dis*383puted material, see Friedman v. Bache Halsey Stuart Shields, Inc., 738 F.2d 1336, 1341 (D.C.Cir.1984) (“showing of relevance can be viewed as a showing of need”), while there does appear to exist a significant potential danger from (premature) disclosure. See Galvin, 2003 WL 22208370, at *6 (granting protective order based upon law enforcement privilege: “in view of the marginal — and in some cases non-existent — pertinence of these documents to plaintiffs case, plaintiff cannot demonstrate a need for these items that would justify overcoming the asserted interest of the [defendant] in confidentiality”).
The Court recognizes the importance of discovery in all cases, see, e.g., Haus v. City of New York, No. 03 Civ. 4915, 2005 WL 1021173 (S.D.N.Y. April 28, 2005), and is not foreclosing additional disclosure here should circumstances warrant. But given the sensitive nature of the material sought and Plaintiffs’ unproven need for the information, the Court grants Defendants’ motion to shield the disputed documents and information at this time. The parties are directed to address at the September 7, 2005 evidentiary hearing, inter alia, whether or not the disputed documents and information are relevant and “substantially needed” by Plaintiffs to prove their case. If at the hearing it appears that the materials are needed and relevant, Plaintiffs may move to reopen the record and conduct further discovery.
IV. Conclusion and Order
For the foregoing reasons, the Court affirms in part and reverses in part Magistrate Judge Maas’s Discovery Order, dated August 18, 2005, and grants Defendants’ motion [6] for a protective order as to the disputed documents and information.
. The Court is not suggesting that counsel would not keep such information confidential, per Judge Maas's directive.