United States v. Sanchez

EFFRON, Chief Judge

(dissenting):

The majority opinion concludes that the military judge did not err in determining that Dr. Kellogg based her testimony on a reliable methodology under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993). For the reasons set forth below, I respectfully dissent.

I. ADMISSIBILITY OF SCIENTIFIC EVIDENCE UNDER DAUBERT

In Daubert, the Supreme Court placed ultimate responsibility on the trial judge to ensure that scientific evidence is reliable by critically examining the methodology from which the expert’s conclusions are derived. Id. at 588, 592-93, 113 S.Ct. 2786. The Court stated that:

in order to qualify as “scientific knowledge,” an inference or assertion must be derived by the scientific method. Proposed testimony must be supported by appropriate validation — i.e., “good grounds,” based on what is known. In short, the requirement that an expert’s testimony pertain to “scientific knowledge” establishes a standard of evidentiary reliability.

Id. at 590, 113 S.Ct. 2786. The Court reasoned that the trial judge’s inquiry must be “a flexible one [whose] overarching subject is the scientific validity — and thus the evidentiary relevance and reliability — of the principles that underlie a proposed submission.” Id. at 594-95, 113 S.Ct. 2786 (footnote omitted).

Under Daubert, the trial judge must make “a preliminary assessment of whether the reasoning or methodology underlying the testimony is scientifically valid and of whether that reasoning or methodology properly can be applied to the facts in issue.” Id. at 592-93, 113 S.Ct. 2786. The Court provided a nonexclusive list of factors that may be used for assessing reliability, including: (1) whether the theory or technique can be or has been tested through use of scientific methodology; (2) whether the theory or technique *154has been subject to peer review and publication; (3) the known or potential rate of error; (4) the existence and maintenance of standards and controls; and (5) whether the theory or technique has been generally accepted in the expert community. Id. at 593-94, 113 S.Ct. 2786.

In crafting the Daubert test, the Court rejected the previous standard, which asked only whether a scientific theory enjoyed “general acceptance” in the relevant professional community. Id. at 588, 113 S.Ct. 2786; see Frye v. United States, 293 F. 1013, 1014 (D.C.Cir.1923). Although Daubert includes a broader range of scientific or technical evidence than the Frye general acceptance test, it is more restrictive than Frye because it requires a determination of whether that evidence is reliable even if it meets a general acceptance test. See Edward J. Imwinkelried et al., 1 Courtroom Criminal Evidence 222-24 (4th ed.2005).

The Daubert reliability assessment must be narrowly tailored to the precise issue before the court. Kumho Tire Co. v. Carmichael, 526 U.S. 137, 141, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999) (citing Daubert, 509 U.S. at 597, 113 S.Ct. 2786). Evidence of an expert’s qualifications and general approach is not sufficient to establish the reliability of a particular technique used by thé expert to analyze data and draw conclusions. Id. at 153-54, 119 S.Ct. 1167. The specific theory or technique that is the subject of expert testimony must be sufficiently reliable to perform the “task at hand.” Daubert, 509 U.S. at 597, 113 S.Ct. 2786; Kumho Tire, 526 U.S. at 153-54, 119 S.Ct. 1167; see also Weisgram v. Marley Co., 528 U.S. 440, 455, 120 S.Ct. 1011, 145 L.Ed.2d 958 (2000) (opining that “[sjince Daubert ... parties relying on expert evidence have had notice of the exacting standards of reliability such evidence must meet”).

The scientific methodology required by Daubert and its progeny is embodied in Military Rule of Evidence (M.R.E.) 702, which provides:

If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education may testify thereto in the form of an opinion or otherwise if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.

See Daubert, 509 U.S. at 589-90, 113 S.Ct. 2786; Fed.R.Evid. 702. As noted by the drafters of the parallel Federal Rule of Evidence, “[t]he more subjective and controversial the expert’s inquiry, the more likely the testimony should be excluded as unreliable.” Fed.R.Evid. 702, Notes of Advisory Committee on 2000 Amendments.

II. ADMISSION OF DR. KELLOGG’S TESTIMONY

At the time of Appellant’s court-martial, Dr. Nancy Kellogg was a physician and the director of the Alamo Children’s Advocacy Center. Doctors at the Children’s Advocacy Center examined all children referred by the local hospital as potential victims of sexual abuse. Dr. Kellogg estimated that she has examined approximately 6,000 children referred for this reason as well as approximately 2,000 children referred for other conditions. The Government moved to permit Dr. Kellogg to testify as an expert witness in the field of child sexual abuse regarding the conclusions she derived from her physical examination of JA, the victim in this case.

In a hearing on the motion under Article 39(a), Uniform Code of Military Justice (UCMJ), 10 U.S.C. § 839(a) (2000), Dr. Kellogg explained that she would testify about three physical findings that led her to conclude JA was “concerning” for sexual abuse: (1) hymenal measurement; (2) anal dilation; and (3) vaginal white blood cell count. Dr. Kellogg defined “concerning” for sexual abuse as “a finding or a group of findings that I cannot readily explain as being normal or [attributed] to a condition other than sexual abuse____they’re concerning in the sense that they signify possible trauma to the genitals”; however, “it could also be attributed to a nontraumatic event.” Additionally, Dr. *155Kellogg explained that the most important factor in her assessment is a patient’s history, or account of abuse, especially if it remains consistent over time.

Trial defense counsel opposed admission of Dr. Kellogg’s testimony, contesting the reliability of her methodology under M.R.E. 702 and Daubert. Specifically, trial defense counsel challenged reliability under three of the Daubert factors: the failure to calculate an eiTor rate for these studies, the absence of direct peer review and publication, and the lack of general acceptance of Dr. Kellogg’s standards. The defense argued that Dr. Kellogg’s experience at the sexual abuse clinic had not been subjected to “any statistical analysis, any verification, nor has she gone out into the community to determine the extent of these concerning findings in normal children.”

The military judge granted the prosecution’s motion to admit Dr. Kellogg’s testimony, concluding that her methodology was reliable under M.R.E. 702 and Daubert. The military judge determined that Dr. Kellogg’s findings were based on sufficient facts or data under M.R.E. 702(1) because “[e]ven if there is a disagreement on matters within this area of expertise, Dr. Kellogg herself still has a basis of her own 6,000 examinations to fall back upon.... ” With respect to the Daubert factors, the military judge stated that: (1) although Dr. Kellogg’s methods were not universally accepted, lack of general acceptance is not a bar to admissibility; (2) Dr. Kellogg’s factors “have been subject to peer review and publication; apparently, hotly so. But that is still peer review and publication”; and (3) “while there can be no known error rate because of the lack of a normative population, that at least to some extent the use of measurements and more than one measurement is accepted in that field.”

III. DISCUSSION

‘When expert testimony’s factual basis, data, principles, methods, or their application are called sufficiently into question, ... the trial judge must determine whether the testimony has a reliable basis in the knowledge and experience of [the relevant] discipline.” United States v. Billings, 61 M.J. 163, 168 (C.A.A.F.2005) (quoting Kumho Tire, 526 U.S. at 149, 119 S.Ct. 1167) (quotation marks omitted). In the present case, the military judge referred to the Daubert factors but failed to either properly apply these factors or employ adequate alternative factors to assess the reliability of Dr. Kellogg’s methodology.

Dr. Kellogg stated that she had testified previously in approximately 600 cases “on numerous subjects” involving a variety of matters related to child sexual abuse. However, the record does not identify which methodologies were at issue in those eases— which Dr. Kellogg said involved issues such as patterns of child disclosure and conditions confused with sexual abuse; nor does the record indicate that the methodology at issue in the present case — see Part II supra (reliance on hymenal measurement, anal dilation, and vaginal white blood cell count) — was litigated and determined to be rehable under Daubert in the prior cases. Assuming that her prior testimony “on numerous subjects” was sufficient to qualify her as an expert, it was not sufficient to establish the reliability of the specific methodology used to support her testimony in the present case. See Kumho Tire, 526 U.S. at 153, 119 S.Ct. 1167; Margaret A. Berger, The Supreme Court’s Trilogy on the Admissibility of Expert Testimony in Reference Manual on Scientific Evidence 9, 34-35 (Federal Judicial Center, 2d ed.2000) [hereinafter Admissibility of Expert Testimony].

The military judge was required to examine the specific issue of whether Dr. Kellogg’s methodology supported her conclusion that JA was “concerning” for child sexual abuse. Kumho Tire, 526 U.S. at 154-55, 119 S.Ct. 1167. In that regard, the military judge was required to ensure that the methodology not only enabled the expert to ascertain the existence of a physical condition, but also enabled the expert to testify as to the causation of that condition. See, e.g., Berger, Admissibility of Expert Testimony at 34-35; Edward Imwinkelried, Forensic Science: The Relativity of Reliability, 40, No. 4 Crim. *156L. Bull. 386 (2004). Assuming that Dr. Kellogg’s testimony would have been admissible, based on her clinical experience, to describe JA’s physical characteristics, Dr. Kellogg went beyond that scope to draw conclusions about the causation of those characteristics— that they were “concerning” for sexual abuse. See Berger, Admissibility of Expert Testimony at 34-35. The military judge failed to ascertain whether Dr. Kellogg’s methodology reliably supported this precise conclusion. Kumho Tire, 526 U.S. at 154-55, 119 S.Ct. 1167.

As the majority opinion notes, Dr. Kellogg made “objective medical and physical findings at the sites of the alleged abuse.” The conclusions Dr. Kellogg derived from her physical findings, however, were subjective. Dr. Kellogg did not tabulate or verify her data and could not correlate the findings to a concrete likelihood of abuse. Dr. Kellogg’s observations were based primarily on her experience as a clinician, which in the absence of her own or other empirical support does not qualify such observations as evidence “derived by the scientific method.” Daubert, 509 U.S. at 590, 113 S.Ct. 2786. As noted in David L. Faigman et al., 1 Modern Scientific Evidence: The Law and Science of Expert Testimony 182 (2006-2007 ed.) [hereinafter Modern Scientific Evidence]:

For scientists, the key word in the phrase “scientific method” is method ...
[Cjlaims [that do not utilize the scientific method] are likely to be defended by statements that the truth of the assertion rests on “my many years of experience,” ____ [but w]ere the findings based on evidence produced by the scientific method, the expert should be able to present those studies to any audience, including a court, along with the methodology and the results of the studies.

Dr. Kellogg’s conclusions constituted “merely an hypothesis,” not the product of a reliable scientific method. Whiting v. Boston Edison Co., 891 F.Supp. 12, 25 (D.Mass.1995) (finding a methodology unreliable when it could not be tested, was rejected by scientists in peer-reviewed journals, and had no known or potential rate of error).

Dr. Kellogg considered findings to be “concerning” for abuse when there was no other readily attributable cause for “possible trauma.” Dr. Kellogg also explained that the most important factor in her appraisal is the patient’s verbal account of abuse. Dr. Kellogg’s hypothesis appears to be as follows: if the child is telling the truth, and there is no other readily apparent cause for the findings of “possible trauma,” then the findings are “concerning” for sexual abuse, meaning only that the possibility of sexual abuse cannot be ruled out. This is the same type of ungrounded testimony Chief Judge Weinstein rejected in In re Agent Orange Product Liability Litigation, 611 F.Supp. 1223, 1238-39, 1250-51 (E.D.N.Y.1985). Agent Orange foreshadowed Daubert in its critique of the Frye standard and emphasis on reliability of the expert’s methodology. Daubert, 509 U.S. at 586 n. 4, 113 S.Ct. 2786 (citing Michael D. Green, Legal Theory: Expert Witnesses and Sufficiency of Evidence in Toxic Substances Litigation: The Legacy of Agent Orange and Bendectin Litigation, 86 Nw. U.L.Rev. 643 (1992)).

In that case, the court barred proposed expert testimony that was based on the following hypothesis: if the plaintiffs accurately reported symptoms, and if there was no evidence of other causes, then exposure to the Agent Orange chemical was “more likely than not” the proximate cause of the plaintiffs’ symptoms. In re Agent Orange, 611 F.Supp. at 1237-38. In rejecting the proposed testimony, the court found that it was “speculative,” “so guarded as to be worthless,” and lacked “any foundation in fact.” Id. at 1238.

The military judge’s assessment of Dr. Kellogg’s methodology is similar to the assessment rejected by the United States Court of Appeals for the Fifth Circuit in Black v. Food Lion, Inc., 171 F.3d 308 (5th Cir.1999). The plaintiff in Black alleged that she developed fibromyalgia as a result of falling in defendant’s grocery store. Id. at 309-10. The trial judge admitted the plaintiffs diagnosing physician as an expert witness on the issue of causation. Id. The physician’s methodology consisted of: taking *157a patient history; diagnosing fibromyalgia; attempting to eliminate other causes; and concluding that the fall was the only possible remaining cause of the disease. Id. at 313.

In rejecting this testimony as unreliable scientific evidence, the Fifth Circuit stated that the Daubert inquiry required that the expert’s specific conclusion — that the fall could have caused the plaintiffs condition— must be the product of a reliable methodology. Id. at 311. The court determined this methodology was unreliable under Daubert because it had not been tested or peer reviewed, lacked a rate of error, and was not generally accepted in the medical community. Id. 313-14. Although the expert followed the “approved protocol for determining fibromyalgia,” a methodology used in medical practice, it did not constitute reliable scientific evidence. See Berger, Admissibility of Expert Testimony at 34-35 (citing Black, 171 F.3d at 313). The Fifth Circuit determined that the trial judge, in admitting the expert testimony, “fatally erred by applying [Daubert’s] criteria at a standard of meaninglessly high generality rather than boring in on the precise state of scientific knowledge in this case.” Black, 171 F.3d at 314.

The military judge in Appellant’s case committed a similar error. Assuming that Dr. Kellogg’s methodology may be used in clinical practice, such use is not sufficient to establish reliability under Daubert and its progeny. See Berger, Admissibility of Expert Testimony at 34-35.

Examination of the Daubert factors identified by the military judge — error rate, peer review and publication, and general acceptance — further underscores the unreliability of Dr. Kellogg’s findings.

A. Error Rate

Dr. Kellogg’s methodology for assessing physical findings did not utilize the scientific method. Dr. Kellogg did not offer any support from the scientific community for the validity of her observations or the conclusions she drew from them. Despite her recognition that other studies have employed scientific research principles in this area, Dr. Kellogg did not record measurements, tabulate data, or otherwise conduct formal studies with her examination results. Although Dr. Kellogg examined approximately 2,000 children who were not referred for possible sexual abuse, she did not attempt to study them as a control group and record the findings. Further, Dr. Kellogg did not test her conclusion that the findings introduced into evidence were “concerning” for sexual abuse in a blind case study. That is, she did not compare examination results of abused versus non-abused children, nor did she research the prevalence of abused children who did not present physical evidence of abuse versus those who did.

Dr. Kellogg maintained that there could be no measurable rate of error for the predictive value of her findings due to a lack of a normative population of non-abused children despite recognizing that “numerous studies” have calculated error rates for factors that may be indicative of child sexual abuse. The military judge improperly relied on Dr. Kellogg’s claim that child sexual abuse is so rampant and hidden that no normative population could be identified in light of her acknowledgment that such studies are regularly conducted. See, e.g., John McCann et al., Perianal Findings in Prepubertal Children Selected for Nonabuse: A Descriptive Study, 13 Child Abuse & Neglect 179 (1989) [hereinafter Perianal Findings in Prepubertal Children].

Even if Dr. Kellogg’s findings were potentially useful for treatment purposes in her clinic, they were not sufficiently reliable to be admitted in a court of law. See Faigman, 1 Modern Scientific Evidence at 182. Her assessments of what constitutes trauma and when trauma is “concerning” for abuse have not been empirically verified and therefore do not evoke sufficient guarantees of reliability to be admitted as expert testimony before a court-martial panel.

B. Peer Review and Publication

The military judge found that Dr. Kellogg’s methods “have been subject to peer review and publication; apparently, hotly so. But that is still peer review and publication.” However, the peer review and publication factor “does not necessarily correlate with *158reliability.” Daubert, 509 U.S. at 593, 113 S.Ct. 2786; Faigman, 1 Modern Scientific Evidence at 60. Rather, the value of peer review lies in the likelihood that other experts will detect flaws in and refíne the methodology. Daubert, 509 U.S. at 593, 113 S.Ct. 2786. “The courts, no less than the scientific community, should be concerned not with the mere formal act of submission to the scrutiny of the scientific community, but with what the community concluded following such scrutiny.” Faigman, 1 Modern Scientific Evidence at 60.

Dr. Kellogg did not refer to any peer-reviewed article or scientific study that supported her findings. A study cited by the defense directly contradicted her finding that the hymenal rim measurements are significant. Joyce A. Adams, Evolution of a Classification Scale: Medical Evaluation of Suspected Child Sexual Abuse, 6 Child Maltreatment 31, 33 (2001) [hereinafter Evolution of a Classification Scale] (stating that “[t]here are currently no published research studies that show that a smooth but narrow posterior rim of hymen, or an enlarged hymenal opening diameter, or any combination of findings, are any more common in abused than in nonabused children”). The pertinent studies in the record, which were submitted by the defense, underscore the absence of a scientific basis for Dr. Kellogg’s views regarding the significance of focal hymenal thickness or a high vaginal white blood cell count as “concerning” for sexual abuse. Joyce A. Adams et. al., A Proposed System for the Classification of Anogenital Findings in Children with Suspected Sexual Abuse, 5 Adolescent Pediatric Gynecology 73 (1992); Adams, Evolution of a Classification Scale at 31; McCann, Perianal Findings in Prepubertal Children at 179.

C. Support in the Scientific Community

The record demonstrates that the three factors Dr. Kellogg identified as “concerning” for sexual abuse have attracted little support in the scientific community. Dr. Kellogg cited the thickening of JA’s hymen in only a localized area as the reason for classifying the hymen as “concerning,” but she acknowledged that there is no data to support her theory that focal hymenal thickening is “concerning” for sexual abuse. Likewise, Dr. Kellogg was aware of only one study that measured anal dilation, and there is no evidence as to how many physicians employ the method and with what criteria, as its use is “very controversial” in the field. Third, Dr. Kellogg testified that JA’s white blood cell count was “concerning” for abuse due to the finding of lack of adequate hymenal tissue. The high white blood cell count had no independent significance — it was “concerning” for abuse only if the hymenal findings were the rehable product of a proven methodology, which Dr. Kellogg acknowledged was not the case. Lastly, the patient’s consistent history, or account of abuse over time, to which Dr. Kellogg gave the greatest weight in making her assessment, is simply not scientific evidence. It is the victim’s account of what occurred, and in this case, it was clinically unverifiable.

Dr. Kellogg’s testimony was admitted to clarify medical evidence for the panel. M.R.E. 702. In this context, the military judge should have ensured that a rehable scientific methodology supported Dr. Kellogg’s conclusions. Daubert, 509 U.S. at 591-92, 113 S.Ct. 2786 (reasoning that Fed. R.Evid. 702 “requires a vahd scientific connection to the pertinent inquiry as a precondition to admissibility”); Billings, 61 M.J. at 168. Instead, Dr. Kellogg’s findings were based on unverified hypotheses. Even though Dr. Kellogg conducted thousands of examinations for “objective medical and physical findings,” she did not use a rehable scientific methodology to evaluate those findings.

IV. CONCLUSION

In the present case, the military judge was required to determine “whether the expert’s theory can be challenged in some objective sense, or whether it is instead simply a subjective, conclusory approach that cannot reasonably be assessed for reliability.” Fed. R.Evid. 702, Notes of Advisory Committee on 2000 Amendments. Here, the military judge did not recognize that there was no independent scientific support for Dr. Kel*159logo’s findings and Dr. Kellogg had failed to test her observations through a reliable scientific method. Accordingly, I would con-elude that the military judge abused her discretion in admitting Dr. Kellogg’s testimony. See Billings, 61 M.J. at 167-68.