United States v. Hunt

Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 1
FILED
United States Court of Appeals
PUBLISH Tenth Circuit

UNITED STATES COURT OF APPEALS March 24, 2023

Christopher M. Wolpert
FOR THE TENTH CIRCUIT Clerk of Court
_________________________________

UNITED STATES OF AMERICA,

Plaintiff - Appellee,

v. No. 21-6046

DOMINIC EUGENE HUNT, a/k/a Dime
Sack,

Defendant - Appellant.
_________________________________

Appeal from the United States District Court
for the Western District of Oklahoma
(D.C. No. 5:19-CR-00073-R-1)
_________________________________

Grant R. Smith, Assistant Federal Public Defender (Virginia L. Grady, Federal Public
Defender, with him on the briefs), Denver, Colorado, for Defendant-Appellant.

Jacquelyn M. Hutzell, Assistant United States Attorney (Robert J. Troester, United States
Attorney, and David McCrary, Assistant United States Attorney, with her on the brief),
Oklahoma City, Oklahoma, for Plaintiff-Appellee.
_________________________________

Before HARTZ, SEYMOUR, and MORITZ, Circuit Judges.
_________________________________

HARTZ, Circuit Judge.
_________________________________

Defendant Dominic Eugene Hunt appeals his convictions on two charges of being

a felon in possession of ammunition. The ammunition was used in two shootings in early

2019. Investigators found three spent cartridges at the scene of one shooting and one
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 2

spent cartridge at the other. A firearms expert testified that all four cartridges were fired

from the same (undiscovered) weapon. Defendant’s sole complaint on appeal is that the

expert testimony should not have been admitted at trial. He argues that the expert’s field

of firearm toolmark examination is not scientifically valid and that the district court failed

to perform its gatekeeping role in examining the admissibility of expert testimony

because it relied on prior judicial opinions rather than the most up-to-date empirical

evidence when it denied his pretrial motion to exclude the testimony without conducting

a hearing.

Exercising jurisdiction under 28 U.S.C. § 1291, we affirm. We need not declare a

general rule on the admission of firearm toolmark testimony. We hold only that the

district court adequately performed its gatekeeping role and did not err in admitting the

testimony in light of the material presented on the pretrial motion and the expert

testimony at trial.

I. BACKGROUND

On January 20, 2019, a car was stolen from the residence of Defendant’s

cousin, Jimmy Jones. The theft was captured on the surveillance camera at Jones’s

home. It showed that after his daughter started her car to warm it up and went back

inside, a man exited a blue Hyundai that had driven by and then jumped in her car

and drove off following the Hyundai. Jones later thought he saw the same blue

Hyundai down the street, though it turned out that he was mistaken. Based on this

misidentification, however, he, Defendant, Travis Carter (Jones’s brother), and

Christopher Dawson (Defendant’s brother) confronted three men that Jones thought

2
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 3

were involved in the theft. This confrontation led to a fistfight, which ended when

someone shot Del Lavar Brison, one of the men from the group that Defendant

confronted. An officer with the Oklahoma City Police Department (OCPD) was

called to the scene and asked Brison who shot him. Brison said something that the

responding officer understood as indicating that “it was a black male wearing a . . .

maroon jacket.” R., Vol. III at 213. The surveillance video from Jones’s home

showed Defendant wearing a maroon hoodie at the time of the theft. An OCPD

crime-scene investigator recovered one spent Blazer 9mm Luger cartridge case from

the scene—it was the only cartridge case that was found.

Less than two weeks later, in the early morning of February 2, 2019, a man

named Conilius Wright was found unconscious in his truck after being mortally

wounded in a drive-by shooting. One of Wright’s companions testified that

Defendant and Wright had issues with one another, and Defendant’s then girlfriend

testified that on the night of the shooting Wright spoke with her about possibly being

the father of her child. Defendant’s cell-phone location data indicated that he was

near Wright’s shooting one minute before it was reported on a 911 call. An OCPD

crime-scene investigator recovered three spent cartridge cases near Wright’s vehicle:

one Blazer 9mm Luger cartridge case and two Winchester 9mm Luger cartridge

cases.

The four 9mm Luger cartridge cases recovered from the January and February

2019 incidents were submitted to the OCPD Firearms Laboratory and were analyzed

3
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 4

by Ronald Jones, a firearm and toolmark1 examiner. Jones compared the cases using

a microscope and reported on November 4, 2019, that (1) the three cartridge cases

recovered at the scene of Wright’s homicide were all fired from the same unknown

firearm and (2) that same unknown firearm fired the one cartridge case recovered

from the scene of Brison’s shooting.

On November 6, 2019, a federal grand jury returned a nine-count third

superseding indictment against Defendant. The first seven counts arose from

Defendant’s unlawful possession of a firearm and drug-trafficking activities five

years before the shooting. This appeal concerns only the last two counts. Count 8

charged that Defendant violated 18 U.S.C. § 922(g)(1) by being a felon in possession

of ammunition—the Blazer 9mm Luger cartridge case recovered from the scene of

the January 2019 incident. Count 9, which was added in the third superseding

indictment, charged that Defendant violated the same provision in February 2019 by

possessing the one Blazer 9mm Luger cartridge case and two Winchester 9mm Luger

cartridge cases recovered from the scene of Wright’s homicide.

After the government disclosed that it intended to present firearms-expert

testimony to show that the spent cartridge cases recovered from the January and

February incidents were fired from the same gun, Defendant filed in March 2020 his

1
“Toolmarks are generated when a hard object (tool) comes into contact with a
relatively softer object. Such toolmarks may occur in the commission of a crime
when an instrument such as a screwdriver, crowbar, or wire cutter is used or when the
internal parts of a firearm make contact with the brass and lead that comprise
ammunition.” Nat’l Rsch. Council, Strengthening Forensic Science in the United
States: A Path Forward 150 (2009).
4
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 5

“Motion in Limine to Exclude Ballistics Evidence, or, Alternatively, for a Daubert

Hearing.” R., Vol. I at 106.

Before our discussion of the arguments advanced in Defendant’s motion to

exclude the firearms-expert testimony, an overview of firearm toolmark examination

is in order. The method of comparing marks left on different pieces of ammunition to

determine whether the ammunition was expended from the same firearm has been

used for over a century. See, e.g., Commonwealth v. Best, 62 N.E. 748, 750 (Mass.

1902) (Holmes, C.J.) (rejecting a challenge to the admission of an expert’s firearms-

identification testimony). While advances in science and technology have refined the

field of firearm toolmark examination, its object remains the same: “to determine

whether ammunition is or is not associated with a specific firearm based on toolmarks

produced by guns on the ammunition.” President’s Council of Advisors on Sci. and

Tech., Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-

Comparison Methods 104 (2016) [hereinafter PCAST Report]. The underlying

concept is that different equipment and processes used in manufacturing firearms

produce unique marks on the internal parts of the firearm. Firearm examiners theorize

that even the same manufacturing tool will produce “microscopically different”

marks on consecutively produced firearms, since “[m]anufacturing tools experience

wear and abrasion as they cut, scrape, and otherwise shape metal.” Nat’l Rsch.

Council, Strengthening Forensic Science in the United States: A Path Forward 150

(2009) [hereinafter NRC Report]. Also, the internal parts of the firearm may undergo

individualized changes through use. Firearm toolmarks are then produced “when the

5
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 6

internal parts of a firearm make contact with the brass and lead that comprise

ammunition.” Id. Specifically with respect to cartridge cases (which house the

primer, gunpowder, and bullet):2

The brass exterior of cartridge cases receive[s] . . . toolmarks during the
process of gun firing: the firing pin dents the soft primer surface at the
base of the cartridge to commence firing, the primer area is forced
backward by the buildup of gas pressure (so that the texture of the gun’s
breech face is impressed on the cartridge), and extractors and ejectors
leave marks as they expel used cartridges and cycle in new ammunition.

Id. at 151.

The experts in this case used the Association of Firearms and Tool Mark

Examiners (AFTE) method to compare the toolmarks left on the four cartridge cases.

See United States v. Hunt, 464 F. Supp. 3d 1252, 1256 (W.D. Okla. 2020). Under this

method, examiners begin by evaluating class characteristics, “which are features that

are permanent and predetermined before manufacture.” PCAST Report at 104. They

are “family resemblances which will be present in all weapons of the same make and

2
At oral argument Defendant’s appellate counsel suggested for the first time
that the district court erred to the extent that it found firearm toolmark examination
reliable based on studies involving toolmarks left on bullets (as opposed to cartridge
cases). We decline to consider this argument. See United States v. Malone, 10 F.4th
1120, 1124–25 (10th Cir. 2021) (“[I]ssues may not be raised for the first time at oral
argument.” (internal quotation marks omitted)). We note, however, that while the
processes that transfer toolmarks to bullets are not the same as those leaving marks
on cartridge cases, see NRC Report at 151, the examinations of each “involve[] many
of the same concepts,” United States v. Williams, 506 F.3d 151, 158 n.4 (2d Cir.
2007); see also Alfred Biasotti, John Murdock & Bruce R. Moran, Introductory
Discussion of the Science—The Scientific Methods Applied in Firearms and
Toolmark Examination, in 4 Modern Scientific Evidence: The Law and Science of
Expert Testimony § 34:9 (David L. Faigman et al. eds., 2021) (discussing the general
method of toolmark examination).

6
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 7

model.” United States v. Willock, 696 F. Supp. 2d 536, 557–58 (D. Md. 2010)

(internal quotation marks omitted). For cartridge cases this includes caliber and the

shape of the firing pin (which affects the firing pin’s impression on the cartridge). “If

the class characteristics are similar, the examination proceeds to identify and

compare individual characteristics, such as the striae that arise during firing from a

particular gun.” PCAST Report at 104. Examiners can make an “Identification”—that

is, conclude that two specimens were derived from the same source (such as a

firearm)—when there is:

Agreement of all discernible class characteristics and sufficient
agreement of a combination of individual characteristics where the
extent of agreement exceeds that which can occur in the comparison of
toolmarks made by different tools and is consistent with the agreement
demonstrated by toolmarks known to have been produced by the same
tool.

Ass’n of Firearm and Tool Mark Exam’rs, Glossary 94 (6th ed. 2013) (defining

Identification) [hereinafter AFTE Glossary]. In arriving at an identification the

examiner tries to avoid confusing individual characteristics with subclass

characteristics—features produced during manufacture, not by design, that are

“consistent among items fabricated by the same tool in the same approximate state of

wear.” Id. at 121.3 On the opposite end of the spectrum, when there is “[s]ignificant

3
For example, “an imperfection on a rifling tool” can “impart[] similar tool
marks on a number of barrels before being modified either through use or
refinishing.” Ronald G. Nichols, Defending the Scientific Foundations of the
Firearms and Tool Mark Identification Discipline: Responding to Recent Challenges,
52 J. Forensic Sci. 586, 587 (2007). An examiner who is not attuned to this
possibility or lacks adequate training or experience may make a misidentification of
two bullets as having come from the same weapon, when the two weapons that fired
7
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 8

disagreement of discernible class characteristics and/or individual characteristics,” an

examiner can determine that the specimens came from different sources—this is

known as an “Elimination” conclusion. Id. at 94 (defining Elimination).

In recent years the field of firearm toolmark examination has been subjected to

closer scrutiny as part of a general program to examine the validity of feature-

comparison forensic methods used in the courts. We discuss three efforts which were

of special importance in the district court’s resolution of Defendant’s pretrial motion:

(1) the 2009 NRC Report, (2) a 2014 study by the Ames Laboratory, and (3) the 2016

PCAST Report.

In 2005 Congress authorized the National Academy of Sciences to create a

committee to study the needs and practices of the forensic-science community. See

NRC Report at 1–2. The committee issued a report in 2009 in which it recommended

that further research be conducted “to address issues of accuracy, reliability, and

validity in the forensic science disciplines.” Id. at 22. With respect to firearm

toolmark examination, the report found that “[s]ufficient studies have not been done

to understand the reliability and repeatability of the methods.” Id. at 154. It also

criticized the discipline’s “lack of a precisely defined process,” stating that “AFTE

has adopted a theory of identification, but it does not provide a specific protocol.”

the different bullets merely share a subclass similarity. See Adina Schwartz, A
Systemic Challenge to the Reliability and Admissibility of Firearms and Toolmark
Identification, 6 Colum. Sci. & Tech. L. Rev. 1, 8–10 (2005).
8
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 9

Id. at 155. The NRC Report recognized, however, that firearm toolmark evidence can

be valuable:

The committee agrees that class characteristics are helpful in narrowing
the pool of tools that may have left a distinctive mark. Individual
patterns from manufacture or from wear might, in some cases, be
distinctive enough to suggest one particular source, but additional
studies should be performed to make the process of individualization
more precise and repeatable.

Id. at 154.

In response to the NRC Report’s recommendations, research was conducted to

better assess the validity of firearm toolmark examination. We discuss here a study

by the Ames Laboratory. See David P. Baldwin et al., A Study of False-Positive and

False-Negative Error Rates in Cartridge Case Comparisons, Ames Laboratory,

USDOE Technical Report # IS-5207 (2014) [hereinafter Ames Study]. The Ames

Study tested 218 firearm examiners by sending each of them 15 sets of four spent

cartridge cases. See Ames Study at 3–5, 8–10. The researchers generated the spent

cartridge cases using 25 Ruger SR9 semiautomatic 9mm handguns.4 See id. at 5, 9–

10. The examiners were to determine whether the fourth cartridge case in each set

(the questioned case) came from the same firearm as the three other cartridge cases in

that set (the known cases). See id. at 4, 10, 15–16. For each examiner the three

cartridge cases came from a different firearm than the fourth case in 10 of the 15 sets.

See id. at 10. The examiners thus conducted 2,180 “true different-source

4
While the PCAST Report approved of the Ames Study’s research design, it
did note one forensic scientist’s criticism that “the study did not involve
consecutively manufactured guns.” PCAST Report at 110 n.331.
9
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 10

comparisons” in total: among these, there were 1,421 correct elimination conclusions,

735 inconclusive findings, and 22 erroneous identification conclusions. Id. at 16.

The false-positive rate in the Ames Study, excluding inconclusive

determinations, was 1.52%. This may overestimate the error rate for firearm toolmark

examinations that most law-enforcement agencies and laboratories conduct. Notably,

“[a]ll but two of the 22 false identification calls were made by five of the 218

examiners, strongly suggesting that this error probability is not consistent across

examiners.” Id. And “the study specifically asked participants not to use their

laboratory or agency peer review process.” Id. at 5. Therefore, the error rate for

comparisons that are peer-reviewed could be lower than 1.52%.

In 2015 the President’s Council of Advisors on Science and Technology began

investigating “whether there [were] additional steps on the scientific side, beyond

those already taken by the Administration in the aftermath of the highly critical 2009

[NRC Report], that could help ensure the validity of forensic evidence used in the

Nation’s legal system.” PCAST Report at x. Like its predecessor, the PCAST

Report—issued in 2016—said that there is a need for additional studies to verify the

principles and methods underlying firearm toolmark examination; it found “that

firearms analysis currently falls short of the criteria for foundational validity, because

there is only a single appropriately designed study to measure validity and estimate

reliability”—the Ames Study.5 Id. at 112. The PCAST Report also complained that

5
According to PCAST, “Foundational validity for a forensic-science method
requires that it be shown, based on empirical studies, to be repeatable, reproducible,
10
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 11

the standard used by toolmark examiners in making an identification is ill-defined,

asserting that the theory of identification is “circular”: “The ‘theory’ states that an

examiner may conclude that two items have a common origin if their marks are in

‘sufficient agreement,’ where ‘sufficient agreement’ is defined as the examiner being

convinced that the items are extremely unlikely to have a different origin.” Id. at 104.

But the PCAST Report did not call for the immediate, wholesale exclusion of

firearm toolmark evidence from the courts. Instead it took the position that

“[w]hether firearms analysis should be deemed admissible based on current evidence

is a decision that belongs to the courts.” Id. at 112. The PCAST Report’s assessment

of this field stands in stark contrast to its assessment of the state of at least one other

forensic-science method that involves feature comparison—bitemark analysis. See id.

at 87 (“[B]itemark analysis does not meet the scientific standards for foundational

validity, and is far from meeting such standards.”).

With this background in mind, we now turn to Defendant’s motion to exclude

expert testimony regarding firearm toolmark examination. His core argument was

that “the Government cannot show that the field of ballistics identification is

and accurate, at levels that have been measured and are appropriate to the intended
application. . . . It is the scientific concept we mean to correspond to the legal
requirement, in [Federal Rule of Evidence] 702(c), of ‘reliable principles and
methods.’” PCAST Report at 4–5. PCAST recognized that “[s]ome methods that have
not been shown to be foundationally valid may ultimately be found to be reliable,
although significant modifications to the methods may be required to achieve this
goal.” Id. at 14.
11
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 12

scientifically valid.” R., Vol. I at 109.6 After discussing the critiques presented in the

PCAST and NRC Reports, he proceeded to argue that exclusion was warranted under

Federal Rule of Evidence 702, applying the nonexclusive factors articulated in

Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), for assessing

whether an expert’s methodology is sufficiently reliable for the expert’s opinion to be

admissible evidence. The Daubert factors are: “(1) whether the [methodology] can be

tested; (2) whether it is subject to peer review and publication; (3) the known or

potential error rate; (4) the existence and maintenance of standards [to control the

methodology’s application]; and (5) the general acceptance in the relevant scientific

community.” United States v. Foust, 989 F.3d 842, 845 (10th Cir. 2021) (citing

Daubert, 509 U.S. at 593–94). Defendant argued:

On each Daubert factor, ballistics identification evidence falls
short. First, the very premise of firearms analysis as a field—the theory
of uniqueness of firearms, bullets and cartridge cases—rests on an
assumption that has not been properly tested. It comes from a time when
bullets were hand-cut and unique, but the field simply has not reckoned
with standardization and technological developments in production. . . .
Second, “peer review and publication” is limited. There has only
been one black-box study on firearms identification. See PCAST Report
at 11, 111. This study was not published in a scientific journal and was
not subject to peer review or publication. Id.
Third, [Defendant] is aware of only one black-box study
conducted with respect to error rates. See [i]d. at 11. The single study
estimated an error rate, but a single study estimating such an error rate
is insufficient to determine a known rate of error for the field. . . .
Fourth, there are no uniform standards for controlling the
technique’s operation. Rather, an individual makes a subjective

6
Defendant’s motion also argued that the government “has not shown that [its]
firearm expert, Ron Jones, conducted his examination of the ballistics evidence in a
reliable manner.” R., Vol. I at 109. That argument is not renewed on appeal as to
Howard Kong, the expert who testified at trial.
12
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 13

determination about an identification, based on his or her own internal
compass. There is no required number of points of agreement for
making an identification. . . .
...
Although courts have also previously allowed firearms examiners
to testify, the awareness of the limitations of ballistics identification is
recent and growing. It does not appear that courts have yet seriously
considered all aspects of the field’s development, or tested its reliability
since the PCAST report decided it was not foundationally valid from a
scientific perspective. . . . Since firearms identification meets none of
the Daubert criteria, this Court should exclude testimony from the
government’s proposed ballistics identification expert.

R., Vol. I at 118–20. Defendant also contended that if the firearms-identification

evidence was admitted, (a) the expert should not be permitted to overstate the

certainty of his conclusion that the cartridge cases were a match and (b) the jury

(1) should be “made aware that the foundational validity of firearms analysis as a

field has not been established” and (2) should be informed of the error rates

documented in the only “appropriately designed” study. Id. at 124.

The government filed a response to Defendant’s motion in which it identified a

second firearm expert, Howard Kong, who is a firearm and toolmark examiner with

the Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) Forensic Science

Laboratory in California. Kong became involved in the case after images of the two

Blazer cartridge cases—one from each incident—were separately uploaded into the

ATF’s National Integrated Ballistics Information Network (NIBIN), a database of

“three dimensional digital ballistic images of spent shell casings recovered from

crime scenes and from crime gun test-fires” that “can automatically generate a list of

potential matches,” purportedly with a “very high level of accuracy.” Erin Aslan,

13
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 14

Crime Gun Intelligence Centers: Using Technology and Intelligence as a Lead

Generator to Identify Trigger-Pullers and Focus Enforcement and Prevention

Efforts, DOJ J. Fed. L. & Prac., Nov. 2018, at 49, 53. The NIBIN system identified a

potential link between the two Blazer cartridge cases; the images were then reviewed

by a correlation review specialist, a peer reviewer, and an ATF firearm examiner

before a NIBIN lead was generated, advising that further investigation was

warranted. All four spent cartridge cases were then sent to the ATF laboratory, where

Kong—who had been advised only that there was a NIBIN lead—conducted a

microscopic comparison and concluded that “the probability that these cartridge

cases were fired in a different firearm is so small that it is negligible.” R., Vol. I

at 216. A peer reviewer reached the same conclusion.

The district court denied Defendant’s motion to exclude, without a hearing, in

a 17-page order that applied the Daubert factors to analyze the validity of firearm

toolmark examination as a field and considered whether Jones and Kong reliably

applied firearm-examination methods in this case. The court observed that the use of

firearm toolmark examination in court is far from novel and that “no federal court has

deemed such evidence wholly inadmissible.” Hunt, 464 F. Supp. 3d at 1256. It then

analyzed the five Daubert factors. See id. at 1256–60.

First, the district court found that “the theory of firearm toolmark

identification can be and has been tested.” Hunt, 464 F. Supp. 3d at 1257. It cited a

collection of studies compiled by the AFTE and noted that Defendant had presented

no authority to the contrary. See id. at 1256–57.

14
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 15

Second, the court found that Daubert’s peer-review factor also favored

admission, citing the AFTE Journal (a publication for firearm toolmark research), the

PCAST and NRC Reports, and various studies that have been conducted on firearm

toolmark examination. See id. at 1257–58.

Third, the court found that the low error rate favored admissibility, citing the

1.52% false-positive rate reported in the Ames Study7 and “a Miami-Dade Study that

reported a potential error rate of less than 1.2%.” Id. at 1258. The court observed that

“[o]ther federal courts examining the AFTE method’s rate of error have likewise

found it to be low” and noted that Defendant had not “introduce[d] any contradictory

studies.” Id.

On the other hand, the district court found that the “standards that control the

[methodology’s application]” factor weighed against admissibility because the

7
In Defendant’s view the real error rate of the Ames Study was roughly 35%.
His argument is that inconclusive determinations should be considered errors because
“the samples were controlled to ensure that they bore sufficient markings and that
these markings where [sic] not disturbed by environmental factors.” Aplt. Br. at 27.
He cites a portion of the Ames Study that mentions that the fired cartridge cases were
collected in a brass catcher and that cases that fell out of the catcher were discarded.
See Ames Study at 11–12. But this is weak support for his proposition, considering
that the Ames Study did not “prescreen[] the quality of samples provided to the
participants” and actually sought to collect data on how many samples “had marks
that were suitable for comparison.” Id. at 4. In any event, we find persuasive the
government’s argument that the Ames Study’s false-positive rate furnishes the
relevant error rate. That is because “a false positive identification . . . is the type of
error that could lead to a conviction premised on faulty evidence.” United States v.
Harris, 502 F. Supp. 3d 28, 39 (D.D.C. 2020). There is no harm to the defendant if a
toolmark examiner makes an inconclusive finding, and Defendant presents no
evidence to support his speculation that examiners will feel pressured to render
conclusive opinions in the trial setting.
15
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 16

general AFTE method is “subjective in nature.” Id. at 1259 (internal quotation marks

omitted). The district court concluded, however, that “the subjectivity of a

methodology is not fatal under Rule 702 and Daubert.” Id. at 1260 (internal quotation

marks omitted).

Finally, with respect to the general-acceptance factor, the district court found

that “the AFTE method used by the Government’s expert here[] is the field’s

established standard” and that the lack of universal acceptance—citing the NRC and

PCAST Reports—did not preclude admission in court. Id. at 1259–60 (internal

quotation marks omitted).

Despite the court’s weighing of the factors in favor of admissibility, it

restricted how the expert’s opinion could be presented. It required that the testimony

adhere to certain limitations set forth in guidance issued by the Department of

Justice.8 See id. at 1261–62. The experts would need to “refrain from expressing their

8
The following limitations are set forth in the Department of Justice’s
“Uniform Language for Testimony and Reports for the Forensic Firearms/Toolmarks
Discipline – Pattern Match Examination,” R., Vol. I at 295:
 An examiner shall not assert that two toolmarks originated from the
same source to the exclusion of all other sources. This may wrongly
imply that a ‘source identification’ conclusion is based upon a
statistically-derived or verified measurement or an actual comparison
to all other toolmarks in the world, rather than an examiner’s expert
opinion.
 An examiner shall not assert that examinations conducted in the
forensic firearms/toolmarks discipline are infallible or have a zero
error rate.
 An examiner shall not provide a conclusion that includes a statistic
or numerical degree of probability except when based on relevant
and appropriate data.
16
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 17

findings in terms of absolute certainty, and they [would] not state or imply that a

particular bullet or shell casing could only have been discharged from a particular

firearm to the exclusion of all other firearms in the world.” Id. at 1261. The court

permitted the experts to testify only “that their conclusions were reached to a

reasonable degree of ballistic certainty, a reasonable degree of certainty in the field

of firearm toolmark identification, or any other version of that standard.”9 Id. at 1262.

 An examiner shall not cite the number of examinations conducted in
the forensic firearms/toolmarks discipline performed in his or her
career as a direct measure for the accuracy of a proffered conclusion.
An examiner may cite the number of examinations conducted in the
forensic firearms/toolmarks discipline performed in his or her career
for the purpose of establishing, defending, or describing his or her
qualifications or experience.
 An examiner shall not use the expressions ‘reasonable degree of
scientific certainty,’ ‘reasonable scientific certainty,’ or similar
assertions of reasonable certainty in either reports or testimony
unless required to do so by a judge or applicable law.
Id. at 297. Defendant concedes that “Kong was able to follow the letter of the
[district court’s] order” limiting the testimony but complains that his testimony
nonetheless “provided the false sense of certainty that the order was attempting
to avoid.” Aplt. Br. at 33. Defense counsel did not, however, raise any
objections during Kong’s testimony at trial.
9
Federal Rule of Evidence 702(d) currently requires that the proponent of
expert testimony show that “the expert has reliably applied the principles and
methods to the facts of the case.” The Advisory Committee on Evidence Rules has
proposed amending Rule 702(d) to require a showing that “the expert’s opinion
reflects a reliable application of the principles and methods to the facts of the case.”
Committee on Rules of Practice and Procedure, Judicial Conference of the United
States, Proposed Amendments to the Federal Rules of Appellate, Bankruptcy, Civil,
and Criminal Procedure, and the Federal Rules of Evidence 308–09 (2021)
(emphasis added). The committee note explains that “[f]orensic experts should avoid
assertions of absolute or one hundred percent certainty—or to a reasonable degree of
scientific certainty—if the methodology is subjective and thus potentially subject to
error.” Id. at 311.
17
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 18

As relevant here, Defendant proceeded to trial on Counts 8 and 9. Kong

testified at Defendant’s trial, but Jones did not. Kong outlined his training and

experience, which included earning a bachelor’s degree in materials engineering,

completing a one-year course at the ATF Firearms Examiner Academy, and receiving

six to eight months of in-house training. Before he became an official firearms

examiner, he was required to pass a competency test. In 2009, after having acquired

five years of experience, he passed an examination required for certification by

AFTE. AFTE certification lasts for five years but is subject to passing yearly

proficiency tests. He has maintained his certification and was recertified in 2014 and

2019. Kong testified that he had 18 years of experience in firearms examination and

that he had handled an average of 50 cases a year. Also, he has toured the facilities of

over a dozen firearms manufacturers.

Consistent with his written report (which had been submitted to Defendant and

the court in response to the pretrial Daubert motion), Kong opined that the four spent

cartridge cases found at the January and February 2019 crime scenes were fired from

the same gun. The class characteristics of the cartridge cases “were all similar.” R.,

Vol. III at 467. With respect to individual characteristics, he explained that he had

examined the striated firing-pin aperture shear marks that were created when the

“head” of each of the four cartridge cases—that is, “[t]he base of the cartridge case

which contains the primer,” AFTE Glossary at 32—scraped against the breech face of

18
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 19

the firearm upon discharge.10 Using a comparison microscope to look at the

cartridges side by side, he found that the striations in the shear marks were in

“sufficient agreement . . . to identify them as having been fired from the same gun.”

R., Vol. III at 467. He also found that the firing-pin impressions—marks left by the

firing pin upon striking the cartridge cases—were in “excellent agreement.” Id.

at 468.

In discussing the striated firing-pin aperture shear marks, Kong said that there

were “somewhere in the neighborhood of 15 to 20 . . . consecutively matching

stria[e].” Id. Earlier in his testimony he had mentioned a study conducted by Alfred

Biasotti and John Murdock that examined the striated marks left on bullets fired from

different firearms and concluded that the best agreement between two different-

source bullets was “maybe three or four consecutive matching striations,” that is,

lines that are “consecutive and right next to each other.” Id. at 462. According to

Kong, the study’s authors concluded that “if you have six consecutive stria[e], then

that would signify a match.” Id. He thus opined that this was “not a borderline case in

terms of identification.” Id. at 468. He said that he was able to reach his conclusion

to “a reasonable degree of certainty within the firearms examination field.” Id.

10
Kong described the breech face as “the backstop or the area that contacts the
base of the cartridge case. . . . [W]hen the cartridge fires, it pushes the case back
against this breech face and picks up marks.” R., Vol. III at 453. The firing-pin
aperture is “[t]he hole in the breech face of a firearm through which the firing pin
protrudes.” AFTE Glossary at 52. Aperture shear marks are “[s]triated marks caused
by the rough edges of the firing pin aperture scraping the primer metal during
unlocking of the breech.” Id. at 52–53.
19
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 20

Kong also testified that, based on differences in class characteristics, the

recovered cartridge casings could not have been fired by two firearms he was asked

to examine, firearms that belonged to two of the men who accompanied Defendant

during the January 2019 incident. The testimony at trial indicated that Defendant was

the only other person—of the four men who confronted Brison—who could have

carried a gun during that incident. Defense counsel did not conduct any cross-

examination of Kong. Defendant was convicted on all counts that went to trial and

sentenced to 960 months in prison.

II. DISCUSSION

We review for abuse of discretion the district court’s application of Daubert in

denying Defendant’s motion to exclude the firearms-examination evidence. See

Foust, 989 F.3d at 845. “We give the district court substantial deference, reversing

only when its ruling was arbitrary, capricious, whimsical, or manifestly unreasonable

or when it made a clear error of judgment or exceeded the bounds of permissible

choice in the circumstances.” Id. (internal quotation marks omitted).

But “we review de novo the question of whether the district court applied the

proper standard [in admitting an expert’s testimony] and actually performed its

gatekeeper role in the first instance.” Dodge v. Cotter Corp., 328 F.3d 1212, 1223

(10th Cir. 2003). This is because “[w]hile the district court has discretion in the

manner in which it conducts its Daubert analysis, there is no discretion regarding the

actual performance of the gatekeeper function.” Goebel v. Denver & Rio Grande W.

R.R. Co., 215 F.3d 1083, 1087 (10th Cir. 2000) (emphasis in original). Thus, our de

20
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 21

novo review is limited to whether the district court properly followed the Daubert

framework and performed an adequate inquiry into the relevance and reliability of

the expert testimony. See Kumho Tire Co. v. Carmichael, 526 U.S. 137, 147 (1999);

Dodge, 328 F.3d at 1221. “For purposes of appellate review, a natural requirement of

the gatekeeping function is the creation of a sufficiently developed record in order to

allow a determination of whether the district court properly applied the relevant law.”

Adamscheck v. Am. Fam. Mut. Ins. Co., 818 F.3d 576, 586 (10th Cir. 2016) (brackets

and internal quotation marks omitted). And the trial court is required to “reply in

some meaningful way to the Daubert concerns the objector has raised.” StorageCraft

Tech. Co. v. Kirby, 744 F.3d 1183, 1190 (10th Cir. 2014).

A. The District Court’s Gatekeeping Role

Defendant does not dispute that the district court cited “the [Daubert] factors

and looked to them exclusively.” Aplt. Br. at 16. But he contends that the district

court “procedurally erred in its application of Daubert” because it failed to conduct a

diligent assessment of the available empirical evidence before making its ruling. Id.

at 14. He faults the district court for taking “a short-cut around Daubert by relying on

prior judicial decisions instead of conducting a meaningful review of the science.”

Id.11

11
Defendant appears to argue that this critical assessment should have
included a “probing inquiry,” Aplt. Br. at 26, into the methodologies used in the
Miami-Dade and Ames Studies, and a thorough analysis of the peer-review procedure
and publication practices of the ATFE Journal. But Defendant did not raise these
specific concerns in his pretrial brief. As we have explained, the district court’s gate-
keeping function is flexible, “requiring the court to focus its attention on the specific
21
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 22

Defendant finds some support in the beginning of the district court’s analysis:

The use of this type of firearm toolmark identification in criminal trials is
“hardly novel.” United States v. Taylor, 663 F. Supp. 2d 1170, 1175
(D.N.M. 2009). “For decades ... admission of the type of firearm
identification testimony challenged by the defendant[ ] has been semi-
automatic ....” United States v. Monteiro, 407 F. Supp. 2d 351, 364 (D.
Mass. 2006); see also, e.g., United States v. Hicks, 389 F.3d 514 (5th Cir.
2004); United States v. Johnson, 875 F.3d 1265, 1281 (9th Cir. 2017).
Indeed, no federal court has deemed such evidence wholly inadmissible.
See United States v. Romero-Lobato, 379 F. Supp. 3d 1111, 1117 (D. Nev.
2019).

Hunt, 464 F. Supp. 3d at 1256. If this was the court’s only analysis, and the court had

rested its decision solely on other courts admitting firearm toolmark identification

evidence, it may well have failed to perform its gatekeeping duty. But the district

court continued, stating that “because of the seriousness of the criticisms launched

against the methodology underlying firearms identification by Defendant in this case,

the Court will carefully assess the reliability of this methodology, using Daubert as a

guide.” Id. at 1256. The district court then worked through the Daubert factors, while

considering the arguments presented in Defendant’s pretrial brief. See id. at 1256–

60. There is nothing improper in a court adopting the reasoning of a prior court. And

the decision primarily relied upon by the district court to support its findings, United

States v. Romero-Lobato, discussed the PCAST and NRC reports in depth. See 379 F.

Supp. 3d 1111, 1117–18 (D. Nev. 2019), aff’d in relevant part, No. 20-10280, 2022

WL 2387214, at *1 (9th Cir. July 1, 2022) (unpublished). Although Defendant argues

factors implicated by the circumstances at hand.” StorageCraft, 744 F.3d at 1190.
The district court is not required to discuss issues that are not raised by the parties.
22
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 23

that “[t]he district court effectively took an end-run around the PCAST Report by

relying on other decisions that either could not or did not consider the report,” Aplt.

Reply Br. at 4, he ignores the district court’s discussion of the PCAST Report’s

critiques that were raised by Defendant in his pretrial briefing, see Hunt, 464 F.

Supp. 3d at 1257–58. Defendant may disagree with the manner in which the district

court fulfilled its gatekeeping responsibilities, but because the district court analyzed

the Daubert factors and addressed Defendant’s arguments, we conclude that the

district court’s gatekeeping role was performed.

B. Admissibility of the Expert’s Testimony

Defendant argues that the district court abused its discretion in allowing Kong

to testify regarding his firearm toolmark examination. He claims that “firearm

toolmark examination methods are subjective, unproven, and not subject to

meaningful review or acceptance outside the insular community of firearm toolmark

examiners.” Aplt. Br. at 15. Although he challenges the reliability of the AFTE

method, he does not challenge Kong’s credentials or whether he reliably applied the

methodology to the facts of this case.

In reviewing the district court’s ruling, we are not limited to its exposition

supporting the ruling. Even if the exposition may be deficient in some respects, any

shortcoming may be harmless error if the record contains the necessary support. As

stated in StorageCraft, “If . . . it is readily apparent from the record that the expert

testimony was admissible, it would be pointless to require a new trial at which the

very same evidence can and will be presented again.” 744 F.3d at 1191. In particular,

23
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 24

we may evaluate the ruling in light of evidence presented at trial that was not

presented at the Daubert hearing. See Kinser v. Gehl Co., 184 F.3d 1259, 1271 (10th

Cir. 1998) (In reviewing the admissibility of expert testimony, “we do not think it

necessary to confine our review to the materials accompanying the Daubert hearing

request. Rather, we believe we may look at the entire record, including testimony

presented at trial.”), abrogated in part on other grounds by Weisgram v. Marley Co.,

528 U.S. 440 (2000); see also StorageCraft, 744 F.3d at 1191–92 (in response to an

argument that the expert made an assumption unsupported by the evidence, the court

held that any error was harmless because the assumption was supported by party’s

deposition); cf. United States v. Harrison, 296 F.3d 994, 1003 (10th Cir. 2002) (“[T]o

affirm the district court’s ruling, we may decide to consider all the evidence at trial,

including evidence not presented at the hearing on the motion in limine [to admit

evidence].”); United States v. Corral, 970 F.2d 719, 723 (10th Cir. 1992) (“In

evaluating the correctness of the district court’s rulings, the appellate court may

consider the entire record developed from the trial even though such evidence may

not have been presented during the suppression hearing.”).

Our concern is the breadth of the district court’s ruling. It can be read as

upholding AFTE methodology in general. This would hardly be a remarkable ruling.

Both before and after the 2016 publication of the PCAST Report, other circuits have

affirmed decisions to admit expert testimony grounded in the AFTE method. See

United States v. Brown, 973 F.3d 667, 702–04 (7th Cir. 2020); United States v.

Johnson, 875 F.3d 1265, 1281 (9th Cir. 2017); United States v. Williams, 506 F.3d

24
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 25

151, 157–62 (2d Cir. 2007). But in light of the critiques expressed in the PCAST and

NRC Reports, we think courts should be cautious, and our holding should go no

further than necessary. “Our task is not to determine the admissibility or

inadmissibility of [firearm toolmark examination] for all cases but merely to decide

whether, on this record, the district judge in this case made a permissible choice in

exercising [his] discretion to admit the expert testimony.” United States v. Baines,

573 F.3d 979, 989 (10th Cir. 2009).

We therefore restrict our consideration to the specific methodology described

by expert Kong at trial. To determine that all four cartridges were fired by the same

weapon, Kong employed the consecutive-matching-striae (CMS) method to three-

dimensional images of the cartridges. As we explain below, the CMS method has

impressive empirical support that would plainly permit a court to find it reliable. The

rub in this case is that the district court’s Order expressing its ruling on the Daubert

motion does not describe that method or the technical literature supporting it. But

that is hardly surprising. Until Kong testified, nothing in the record would have

informed the court about the specific identification method used by Kong.12 Although

the technical literature referenced by the government in its Daubert brief and by the

court in its Order says a good deal about the CMS method, there was no reason for

12
The government first identified Kong as an expert in its response to
Defendant’s motion to exclude. Defendant filed no reply challenging Kong’s
qualifications or expertise. At trial Hunt did not cross-examine Kong or raise any
objection to his testimony. He thereby forfeited any challenge that might derive from
the failure to disclose this methodology before trial. Nor has Hunt raised such a
challenge on appeal.
25
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 26

the Order to focus on that method as opposed to general AFTE methods. But if the

court had been called on at trial to opine on this specific subset of AFTE

methodology, there can be little doubt that the court, having already declared that the

general AFTE methodology is reliable, would have found that the CMS method, with

its extensive empirical support, is a fortiori reliable. Hence, any error in the court’s

not specifically addressing the CMS method is harmless. We now turn to a discussion

of that method and how it would affect the district court’s Daubert analysis.

In applying the CMS method, the examiner aligns the objects being compared

and counts the number of consecutive striae where the width, morphology, and

relative position match exactly. Kong testified that finding six or more consecutive

striae that match is sufficient to determine that two cartridges were fired from the

same weapon. He also said that each additional consecutive matching stria further

reduces the likelihood that two cartridges were fired from different weapons. Thus,

because the four cartridge cases here had 15 to 20 consecutive matching striae, it was

“not a borderline case in terms of identification.” R., Vol. III at 468.

Kong said that the criterion he used was based on

a study done some time ago by Biasotti and Murdock. And what they
did was they looked at the best agreement and bullets that were fired
from different firearms, and the best they came up with was, like, maybe
three or four consecutive matching striations, and that’s the absolute
best. So what they did was they proposed that if you have six
consecutive strias, then that would signify a match.

Id. at 462. He further testified that if two bullets (or cartridges) fired from different

guns had striae that satisfied the criterion for identifying the bullets as coming from

26
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 27

the same weapon, firearms examiners would learn of the discovery, yet he had never

heard of such a discovery.

The criterion used by Kong is supported by the research of Biasotti and

Murdock that he mentioned in his testimony and by several studies referenced in the

government’s pretrial Daubert brief and in the district court’s Order. Biasotti’s

original study, first published in 1959, examined 244 bullets and analyzed 1,200

known match comparisons and 1,080 known nonmatch comparisons. It found

empirical support for the proposition that a significant number of consecutive

matching striae will appear only on bullets fired from the same gun.13 See Biasotti, A

Statistical Study of the Individual Characteristics, supra, at 35–36; Alfred Biasotti,

John Murdock & Bruce R. Moran, Development of Objective Criteria for

Identification, in 4 Modern Scientific Evidence: The Law and Science of Expert

Testimony § 34:13, at 1013–18 (David L. Faigman et al. eds., 2021). Biasotti and

Murdock continued to research consecutive matching striae in toolmarks, and in 1997

they formulated their “conservative quantitative criteria for identification” (the

Biasotti-Murdock criteria). Jerry Miller, An Examination of the Application of the

Conservative Criteria for Identification of Striated Toolmarks Using Bullets Fired

from Ten Consecutively Rifled Barrels, 33 AFTE J. 125, 126 (2001); see Biasotti,

13
Before this research some experts looked to the total percent of matching
striae (regardless of whether they were consecutive) between the two bullets to
determine a match. But Biasotti concluded from his data that the total percent of
matching striae was an unreliable basis for identification. See Alfred Biasotti, A
Statistical Study of the Individual Characteristics of Fired Bullets, 4 J. Forensic Sci.
34, 37–39 (1959).
27
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 28

Murdock & Moran, Development of Objective Criteria, supra, at 1018–20. Under

their criteria a match is found for three-dimensional striae “when at least two

different groups of at least three consecutive matching striae appear in the same

relative position, or one group of six consecutive matching striae are in agreement.”14

Michael Neel & Major Wells, A Comprehensive Statistical Analysis of Striated Tool

Mark Examinations Part 1: Comparing Known Matches and Known Non-Matches,

39 AFTE J. 176, 177 (2007). But to apply the criteria, the toolmark examiner must

rule out the influence of subclass characteristics.15

14
Biasotti and Murdock also formulated a criterion for examining two-
dimensional toolmarks. Two-dimensional toolmarks, also sometimes called lines,
include “[a]ny impressed or striated toolmark that lacks apparent depth.” Neel &
Wells, A Comprehensive Statistical Analysis of Striated Tool Mark Examinations,
supra, at 177.) Under the two-dimensional criterion, a match occurs “when at least
two groups of at least five consecutive matching striae appear in the same relative
position, or one group of eight consecutive matching striae are in agreement in an
evidence toolmark compared to a test toolmark.” Id. Biasotti’s original research
examined two-dimensional marks.
15
Kong’s notes from his examination of the cartridge cases, which were
submitted to the court, discussed his elimination of subclass characteristics:
Aperture shear marks: The aperture shapes are similar to a
“teardrop[,]” which means there is a ramp at the 6 o’clock position. The
aperture shear marks had excellent correspondence, and the agreement
was sufficient for identification. Aperture shear marks were generated
when the primer of the cartridge case rubbed on the edge of the breech
face at the ramp. The edge is the intersecting of these two differently
machined surfaces and will not have subclass potential. The ramp was
produced by a rotating cutting tool to a tilted breech, therefore
producing toolmarks that are perpendicular to the [cartridge cases’]
movement during firing; so no potential subclass marks are from the
ramp.
Firing pin marks: Excellent correspondence of a series of marks
observed on the firing pin (FP) impression. A line was visible running
from the 6 to 12 o’clock in the center of the FP impression, and
28
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 29

The Biasotti-Murdock criteria, the basis of the CMS method, are supported in

several ways by technical papers referenced in the government’s Daubert brief and

the district court’s Order. One 2007 empirical study looked at “4188 striated

toolmark comparisons from a variety of sources”16 and found no known nonmatches

that met the Biasotti-Murdock criteria. Id. at 179, 180. In particular, the study found

no more than four consecutive matching three-dimensional striae in a known

nonmatch—two less than the six required under the Biasotti-Murdock criterion. The

study pointed out that “[t]here has been over 50 years of research regarding CMS run

counts, beginning with Biasotti’s thesis in 1955, and no documented 2D or 3D

comparisons have been shown to contradict the original criteria set forth by Biasotti

and Murdock.” Id. at 190.

The Biasotti-Murdock criteria are also supported by a theoretical analysis that

calculated the probabilities of consecutive matches of two-dimensional striae

appeared to be a mold line from castings or metal injection molding
(MIM) parts. Mold surfaces can have features carryover from part to
part. However, the firing pin appeared to have a defect that produced the
observed corresponding marks, and such a defect could be unique, but
can have subclass potential. If the mold has a defect, then there is
subclass.
R., Vol. I at 272.
16
The sources for this study included “fired cartridge cases, fired bullets,
sandpaper of various grit sizes (60, 22, 320) used to scratch emulsion based film,
chisels slid across lead foil, photomicrographs of plastic replicas taken from
consecutively rifled barrels, fired bullets from cut sections of a Thompson Contender
barrel, and fired bullets from consecutively rifled Bar-Sto barrels.” Neel & Wells,
A Comprehensive Statistical Analysis of Striated Tool Mark Examinations, supra,
at 179.
29
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 30

occurring by chance. See David Howitt, et al., A Calculation of the Theoretical

Significance of Matched Bullets, 53 J. Forensic Sci. 868 (2008). The researchers

found a “close similarity” between Biasotti’s 1959 experimental observations and

their calculated probabilities. Id. 872–73. Based on their calculations, the probability

of the random creation of eight consecutive matching two-dimensional striae (the

Biasotti-Murdock two-dimensional criterion) would be less than one in a hundred

million.

In addition, the Biasotti-Murdock criteria have been successfully tested in an

automated identification system. See Wei Chu, et al., Automatic Identification of

Bullet Signatures Based on Consecutive Catching Striae (CMS) Criteria, 231

Forensic Sci. Int’l 137 (2013). The system minimized subjective factors by using a

computer program objectively applying the three-dimensional Biasotti-Murdock

criterion. It correctly identified 29 of 30 matching bullet pairs from the “unknown”

test set and found no false positives in 12,960 known nonmatch comparisons.

To be sure, one can find literature criticizing the CMS method. Mr. Hunt’s

reply brief cites a 2005 article asserting that striae counting is inherently subjective

and that “the CMS approach fails to place firearms and toolmark identification on

adequate statistical empirical foundations.” Schwartz, A Systemic Challenge, supra,

at 21. But even that article recognizes advantages of CMS identification as compared

to the general AFTE methodology. Id. at 15 (“CMS differs from and is scientifically

superior to the subjective approach because it is interpretable in a way that is

compatible with the probabilistic nature of identity claims.”). And more importantly,

30
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 31

articles discussed above that were published after 2005 undercut Schwartz’s

concerns. The article by Chu, et al., shows that CMS can be applied objectively; and

the article by Howitt, et al., provides theoretical statistical support for CMS. We also

think it significant (1) that the CMS method is not mentioned in the PCAST report,

which questions only AFTE methods in general and (2) that the 2009 NRC Report

suggests that its criticism of the lack of studies supporting firearms and toolmark

identification in general may not apply to CMS: “Recent research has attempted to

develop a statistical foundation for assessing the likelihood that more than one tool

could have made specific marks by assessing consecutive matching striae, but this

approach is used in a minority of cases.” NRC Report at 154 n.63.

We now turn to an examination of the district court’s Daubert analysis as

supplemented by specific information on the reliability of the CMS method.

Regarding the first Daubert factor, the district court found that “the theory of firearm

toolmark identification can be and has been tested.” Hunt, 464 F. Supp. 3d at 1257.

Discussion of the CMS method would only have reinforced that conclusion. By

providing quantitative criteria for identification, the CMS method is readily testable

and has been tested. As discussed above, the CMS method has been empirically

tested for over 60 years and no nonmatches have been found that meet the Biasotti-

Murdock criteria. See Neel & Wells, A Comprehensive Statistical Analysis of Striated

Tool Mark Examinations, supra, at 190; see also Biasotti, Murdock & Moran,

Development of Objective Criteria, supra, at 1021–23 & n.26 (summarizing the

empirical studies evaluating Biasotti-Murdock criteria and concluding that “[n]o

31
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 32

known non-matching . . . toolmarks were found . . . which exhibited agreement in

excess of the proposed Biasotti-Murdock criteria.”).

The district court also determined that the second Daubert factor—whether the

technique has been subjected to peer review and publication—weighed in favor of

admissibility. See Hunt, 464 F. Supp. 3d at 1257–58. Again, that determination finds

particular support with respect to the CMS method. Biasotti’s original 1959 article

was published in the Journal of Forensic Science, which is peer-reviewed. See

Biasotti, Murdock & Moran, Development of Objective Criteria, supra, at 1013 &

n.5. And the other studies discussed above were each published in the AFTE Journal,

the Journal of Forensic Science, or Forensic Science International. The studies

supporting the CMS methodology were subject to the same peer-review process as

the articles addressed by the district court in its Order. This factor would only be

strengthened in support of reliability.

The third Daubert factor is whether the technique has a known or potential rate

of error. The district court, noting the error rates from the Ames Study (which, as

here, involved cartridge-case comparisons) and the Miami-Dade Study, weighed this

factor in favor of admissibility. See Hunt, 464 F. Supp. 3d at 1258.17 For the Biasotti-

17
We add that at least one “post-PCAST Report study . . . followed the
PCAST recommended black-box model and found that of 1512 possible
identifications tested, firearms examiners correctly identified 1508 casings to the
firearm from which the casing was fired.” Harris, 502 F. Supp. 3d at 38 (citing Mark
A. Keisler et al., Isolated Pairs Research Study, 50 AFTE J. 56, 58 (2018)). Notably,
“[n]o false positive . . . results were reported.” Keisler et al., Isolated Pairs Research
Study, supra, at 57.
32
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 33

Murdock criteria, there is not a known error rate from empirical testing because no

false match has been found. But the theoretical analysis discussed above suggests

that the probability of two random bullets satisfying the Biasotti-Murdock criteria

would be infinitesimal. Again, consideration of the CMS method would only have

increased the support of this factor in favor of reliability.

The district court weighed the fourth factor—whether there are standards that

control the technique’s operation—against admissibility. See Hunt, 464 F. Supp. 3d

at 1259. But even if the court’s assessment is correct with respect to toolmark

examination in general, the Biasotti-Murdock criteria provide specific objective

standards that control the application of the CMS method. See Romero-Lobato,

379 F. Supp. 3d at 1121 (“The CMS method, standing alone, qualifies as an objective

standard under Daubert.”). Although the determination of subclass features and

deciding whether two striae match may involve some subjectivity, the toolmark

examiner’s discretion is constrained: “CMS is defined as striated markings that ‘line

up’ exactly (close doesn’t count) with one another without a break or dissimilarity in

between them.” Biasotti, Murdock & Moran, Development of Objective Criteria,

supra, at 1015–16 n.8.

Finally, the district court found that the general-acceptance factor favored

admission because the AFTE method is widely used by firearms examiners. See

Hunt, 464 F. Supp. 3d at 1259–60; see also Brown, 973 F.3d at 704 (district court

observed that “firearm and toolmark analysis is widely accepted beyond the judicial

system”). Like the other factors, consideration of the CMS method would have

33
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 34

further supported this conclusion. Although the CMS method might not be “routinely

used by firearm examiners across the country,” Romero-Lobato, 379 F. Supp. 3d

at 1122, this may be because “many firearms examiners consider [the Biasotti-

Murdock criteria] to be too stringent,” United States v. Taylor, 663 F. Supp. 2d 1170,

1178 (D.N.M. 2009). This feature of the CMS method—missing some identifications

through the application of stringent criteria that minimize the possibility of a false

positive—enhances the reliability of CMS-derived identifications. There is no reason

to doubt that CMS identifications are generally accepted, at least within the

community of firearms examiners. See Foust, 989 F.3d at 846 (“Although . . .

acceptance by unbiased experts is always better, that does not mean this factor cannot

support admission.”); Baines, 573 F.3d at 991 (similar).18

We conclude that even if the district court’s opinion is inadequate to support

the AFTE methodology for firearm toolmark examination in general, any

shortcoming was harmless because the specific CMS method relied on by Kong has

compelling support in his testimony and the technical literature referenced by the

district court. And Kong’s opinion in this case seems particularly resistant to

criticism given the large number of consecutive matching striae and his expertise and

experience in applying the methodology.

18
We note that one of the most critical judicial assessments of firearm
toolmark identification—repeatedly cited by Defendant—did not consider the CMS
method. See generally United States v. Tibbs, No. 2016 CF1 19431, 2019 WL
4359486 (D.C. Super. Ct. Sept. 5, 2019).
34
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 35

C. Response to the Concurrence

The Concurrence complains that the majority opinion improperly goes outside

the record and violates the principle of party presentation. Both complaints are

unwarranted.

First, the record. The record consists of those documents (broadly construing

the term to include electronically stored matters, such as video or audio recordings)

filed with the court. The scientific and technical papers referenced in the majority

opinion that the concurrence complains were not in the record were summarized on a

website referenced in the district court’s opinion and hyperlinked in the

government’s memorandum filed with the court for the Daubert hearing. The only

relevance of the referenced website was the studies it collected. Under any reasonable

notion of record, those papers were part of the district-court record.19 They would

surely have been part of the record if they were attached as hard copies to the

government’s brief. Is such useless printing to be required by parties in the future?

What purpose is advanced by not considering those papers as part of the record?

The Concurrence suggests that maybe the papers were part of the record but

only for a limited purpose. It says that the reference to the studies by the government

was just to establish that such studies existed, “not their quality or content.”

19
That was certainly the view of defense counsel at oral argument, who
described the website articles as “evidence.” A few seconds into his argument,
counsel noted the citation to the website in the district court’s opinion and suggested
that the opinion did not contain a sufficient analysis of the studies themselves,
arguing that “[t]he court wasn’t conducting a review of the evidence by merely
looking at that website.” Oral Argument at 1:22–1:27.
35
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 36

Concurrence at 1. But surely a court’s role in performing a Daubert analysis is not

limited to looking at titles and counting how many there are. What sort of a

gatekeeper is that? The court must examine contested studies to protect the system

from junk science and self-serving ipse dixits by advocates for particular theories.

The opposing party has every right, and every incentive, to challenge the quality of

the studies proffered by the offering party and the relevance of their content to the

issues before the court. Here, Defendant had retained his own ballistics expert,

although the expert was never called to testify. The fact that the opposing party made

no such challenge hardly means that the quality and content of the studies are not part

of the record. A great deal of the record in every case goes unchallenged. If the

district court had in fact read some of the referenced papers to assist it in ruling on

the Daubert motion (which is not entirely implausible since the district-court opinion

notes that the court visited the site two weeks before filing the opinion, see

464 F. Supp.3d at 1257), would we be required to reverse the court’s decision

because it went “outside the record” in reaching its decision?

Insisting on a cramped notion of what is part of the record is particularly

inappropriate in the present context. The studies at issue do not address adjudicative

facts peculiar to the specific case before us. They concern legislative facts that are

applicable to a great many, and wide range of, cases; the reliability of firearm

toolmark examination must be assessed to determine the admissibility of evidence of

such examinations. When the resolution of a dispute turns on legislative facts, courts

regularly relax the restrictions on judicial inquiry. For example, while appellate

36
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 37

courts can take judicial notice of adjudicative facts only in limited circumstances,

see Fed. R. Evid. 201; Winzler v. Toyota Motor Sales U.S.A., Inc., 681 F.3d 1208,

1212–13 (10th Cir. 2012) (Gorsuch, J.) (appellate court may take judicial notice of

facts “at any stage of the proceeding if the facts are not subject to reasonable dispute”

(internal quotation marks omitted)), there are no such strict limits with respect to

legislative facts, see Fed. R. Evid. 201 advisory committee note (“‘In determining the

content or applicability of a rule of domestic law, the judge is unrestricted in his

investigation and conclusion. He may reject the propositions of either party or both

parties. He may consult the sources of pertinent data to which they refer, or he may

refuse to do so. He may make an independent search for persuasive data or rest

content with what he has or what the parties present.’” (quoting Edmund M. Morgan,

Judicial Notice, 57 Harv. L. Rev. 269, 260–71 (1944))); Edward K. Cheng,

Independent Judicial Research in the Daubert Age, 56 Duke L.J. 1263, 1293 (2007)

[hereinafter Cheng] (“If one takes the Advisory Committee’s adoption of the Morgan

view seriously, this conclusion [that the scientific facts used for Daubert

determinations should be treated as legislative facts] means that the Federal Rules

free judges to do independent research in the Daubert context.”). As noted by Prof.

Frederick Schauer in The Decline of “The Record”: A Comment on Posner, 51 Duq.

L. Rev. 51 (2013) [hereinafter Schauer], courts sometimes rely on nothing more than

the personal intuitions of its members (as when it selected the burden of persuasion

for libel actions based on how it thought the press would react to different standards,

see New York Times v. Sullivan, 376 U.S. 254, 282 (1964); Schauer at 57–58, 63),

37
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 38

and they sometimes rely on their review of sources never presented to the court (as

with the per curiam majority in Bush v. Gore, 531 U.S. 98, 103 (2000), and Justices

Stevens and Breyer on multiple occasions, including for the court in Kumho Tire Co.

v. Carmichael, 526 U.S. 137, 142–43 (1999); see Schauer at 51 n.3, 56). The

propriety of this independent research is somewhat controversial, see, e.g., Schauer at

51–52; Cheng at 1280–84, but when the highest court in the land uses non-record

materials to resolve legislative facts, it is hard to justify barring appellate

consideration of materials that were referenced in the district court and were subject

to prior examination and dispute by all the parties.

The Concurrence also misapplies the party-presentation principal. As we

recently explained, that principle generally restricts the court from raising issues sua

sponte, but it does not preclude a court from resolving a presented issue in a way not

proposed by any party. See United States v. Cortez-Nieto, 43 F.4th 1034, 1052 (10th

Cir. 2022). For example, what if each party in a contract dispute argues that a

provision of a contract is unambiguous but they disagree on what that unambiguous

meaning is? It would be wholly proper for the court to decide that the provision has a

third meaning or that the provision is ambiguous and further factual development is

necessary.

Here, the government argues that AFTE methodology—as a whole—is reliable

(so admissibility should turn only on the expertise of the examiner and the

application of the methodology), while Defendant argues that the methodology is

never reliable enough to be used in court. In reviewing the issue, we are not limited

38
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 39

to choosing between those alternatives. We can, and do, reserve ruling on the

reliability of the AFTE methodology in general, largely because of the prestige of the

government-sponsored reports questioning whether the accuracy of the methodology

has been sufficiently tested. On the other hand, there has been considerable testing,

over an extended period of time, of the accuracy of one subset of AFTE

methodology—the CMS methodology used in this case. We therefore can affirm the

conviction here without going further and addressing the more general question of

reliability of AFTE methodology.

Given that liberty is at stake in this case, and in most cases where expert

firearm toolmark testimony is offered, we are reluctant to go beyond what is

necessary in this case and provide a judicial imprimatur to AFTE methodology in

general. Although the concurrence believes that the majority opinion takes appellate

review beyond its proper sphere, we think our approach is fully consonant with

traditional appellate practice and, in the circumstances, not only prudent but less

adventurous than full endorsement of the district-court ruling. We are particularly

reluctant to adopt the ad hoc approach of the Concurrence, which would admit the

testimony in this case while reserving judgment on admissibility in other cases,

without providing needed guidance to the lower courts.

Of course, appellate judges must be particularly careful in vetting scientific or

technical research that was not debated in district court. But nothing precluded

Defendant, who had retained his own ballistics expert, from challenging the cited

literature. In any event, it is not uncommon for an appellate court to perform an

39
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 40

independent Daubert analysis, as when it is assessing whether a district court’s

failure to perform the analysis was harmless error. See Smith v. Jenkins, 732 F.3d 51,

64–68 (1st Cir. 2013); Sarkees v. E. I. Dupont De Nemours & Co., 15 F.4th 584,

589–93 (2d Cir. 2021); UGI Sunbury LLC v. A Permanent Easement for 1.7575

Acres, 949 F.3d 825, 833–36 (3d Cir. 2020); Sardis v. Overhead Door Corp., 10

F.4th 268, 285–96 (4th Cir. 2021); Anderson v. Raymond Corp., 59 F.4th 279, 283–

85 (7th Cir. 2023); United States v. Ruvalcaba-Garcia, 923 F.3d 1183, 1190–91 (9th

Cir. 2019). This court explicitly approved that practice in StorageCraft Technology

Corp. v. Kirby, 744 F.3d 1183, 1190–92 & n.2 (10th Cir 2014) (Gorsuch, J.).

What the majority opinion does in this case is significantly less

comprehensive. We have merely supplemented the district court’s analysis from the

Daubert hearing with the new information provided at trial. As previously noted, it is

accepted practice for appellate courts to affirm a ruling on a pretrial motion in limine

after considering what happened at trial. See, e.g., Kinser, 184 F.3d at 1271 (In

reviewing the admissibility of expert testimony, “we do not think it necessary to

confine our review to the materials accompanying the Daubert hearing request.

Rather, we believe we may look at the entire record, including testimony presented at

trial.”). At trial, expert witness Kong reported for the first time his use of the CMS

methodology in comparing the cartridge shells and he described the leading study

supporting that methodology. If there were flaws in that methodology or the studies,

Defendant could have raised challenges; but he did not even cross-examine the

expert. The concurrence complains that neither the parties nor the district court

40
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 41

discussed or analyzed the studies relevant to the CMS technique that were described

on the AFTE website referenced by both the government and the court. But when

Defendant raises no challenge to those studies or the methodology, we can properly

assume their validity. Our approach, which the concurrence disapproves, has been

simply to conduct our own examination of the relevant studies to confirm that the

CMS methodology is not subject to the criticisms of firearm toolmark methods in

general which have been made by distinguished panels of forensic experts. We see

that as our duty, not a usurpation of power.

In short, we determine that the support the district court found for the general

AFTE methodology is particularly strong with respect to the specific CMS

methodology used in this case, making it unnecessary to resolve the reliability of the

general methodology.

We recognize that our standard of review under Daubert is abuse of discretion.

But the issues are much too important to be resolved by a simple wave of the hand

and deference to the decision below. Respect for the courts is significantly reduced

when litigation is resolved on the basis of pseudoscience that is rejected by highly

educated and intelligent citizens who begin to think that the overlap between truth

and judicial judgment is too slim. Appellate judges have an obligation to educate

themselves and engage in the sometimes difficult work of assessing scientific and

technical material. At least in this case, the analysis is not particularly difficult

because of the extensive testing of the CMS methodology.

41
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 42

III. CONCLUSION

We AFFIRM Defendant’s convictions.

42
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 43

United States v. Hunt, 21-6046

MORITZ, J., concurring.

I agree that the district court did not abuse its discretion when it deemed Howard

Kong’s toolmark-identification methodology reliable under Daubert v. Merrell Dow

Pharmaceuticals, Inc., 509 U.S. 579 (1993). But unlike the majority, I would reach that

conclusion without supplementing the district court’s analysis or engaging in an

independent review of studies and literature about consecutive matching striae (CMS).

Initially, I can’t join the majority’s reliance on and discussion of the CMS studies

because those studies are not part of the record in this case. Cf. United States v. Kennedy,

225 F.3d 1187, 1191 (10th Cir. 2000) (“This court will not consider material outside the

record before the district court.”). To be sure, citations to most of the studies discussed by

the majority appear on a website that the government cited once in its response to the

motion to exclude and that the district court cited once in its order. This website,

published by the Association of Firearm and Tool Mark Examiners (AFTE), offers

“literature citations for the more important studies that qualify as material principally

concerned with the validity of firearm and toolmark identification,” including “[a] short

summary follow[ing] each citation.” Testability of the Scientific Principle, The

Association of Firearm and Tool Mark Examiners, https://afte.org/resources/swggun-

ark/testability-of-the-scientific-principle (last visited March 23, 2023).

But the government’s and district court’s citations merely supported the existence

of the studies listed on the website, not their quality or content. The government cited the
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 44

website to support its assertion that “[t]he AFTE theory is regularly tested both on an

individual level, by peer review and verification, and on a larger level with numerous

studies.” R. vol. 1, 192. Likewise, the district court cited the website to support the

conclusion that “the theory of firearm toolmark identification can be and has been

tested.” United States v. Hunt, 464 F. Supp. 3d 1252, 1257 (W.D. Okla. 2020). Thus,

contrary to the majority’s characterization—that Kong’s CMS methodology is supported

“by several studies referenced in the government’s pretrial Daubert brief and in the

district court’s [o]rder”—neither the government nor the district court “referenced” any

of the studies the majority relies on. Maj. Op. 27.

Nor are the government’s and district court’s passing citations to the AFTE

website sufficient to incorporate any or all of the listed studies—over 100 studies

grouped into six topics—into the record for purposes of appeal. The majority cites no

authority to support its decision to do so, and I have found none.1 Although a reviewing

court may sometimes affirm the admission of expert testimony despite possibly flawed

reasoning by the district court, such harmlessness must be “readily apparent from the

record.” StorageCraft Tech. Corp. v. Kirby, 744 F.3d 1183, 1190–91 (10th Cir. 2014)

(emphasis added); see also Kinser v. Gehl Co., 184 F.3d 1259, 1271 (10th Cir. 1999).

1
The majority also fails to explain whether it reviewed all the website’s listed
studies or how it chose the handful that it relies on. Nevertheless, because I would not
consider such studies at all, I will not substantively engage with the majority’s
assessment of the literature.
2
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 45

Because the studies analyzed, discussed, and applied by the majority are not part of the

record, I can’t join the majority’s decision to rely upon them.2

Moreover, and perhaps more critically, even if these studies are part of the record,

neither the district court nor the parties ever mentioned such studies in their briefing

below, let alone discussed or analyzed those studies. As a result, the court best positioned

to evaluate the CMS literature and its reliability (the district court) had no opportunity to

consider it. See Goebel v. Denver & Rio Grande W. R.R. Co., 215 F.3d 1083, 1088–89

(10th Cir. 2000) (“Performance of the gatekeeping function on the record [e]nsures that a

judgment in favor of either party factors in the need for reliable and relevant scientific

evidence. It is not an empty exercise; appellate courts are not well-suited to exercising the

discretion reserved to district courts.”). And the same is true on appeal: We lack the

benefit of the parties’ positions on this literature and the reliability of the CMS method.

Thus, the majority’s reasoning—which turns heavily, if not entirely, on its “own

examination” of CMS literature that is included amongst a variety of other toolmark-

identification studies on an AFTE website, Maj. Op. 41—significantly departs from “the

principle of party presentation.” United States v. Sineneng-Smith, 140 S. Ct. 1575, 1578

(2020). This principle “restricts courts from raising new issues,” yet the majority here

does exactly that when it affirms the district court’s admission of Kong’s expert

2
The majority also appears to rely on Kong’s brief mention during his trial
testimony of “a study done some time ago by Biasotti and Murdock.” Maj. Op. 26
(quoting R. vol. 3, 462). But this passing mention doesn’t meaningfully incorporate the
majority’s swath of scientific literature into the appellate record. Additionally, the study
Kong mentioned doesn’t appear to be included on the AFTE website, which lists one
study by Biasotti and two by Murdock, but none by both individuals.
3
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 46

testimony based on a theory of CMS reliability that no party has either “raised” or

“responded to.” United States v. Cortez-Nieto, 43 F.4th 1034, 1052 (10th Cir. 2022)

(emphasis omitted). In so doing, the majority does far more than “supplement[] the

district court’s analysis”; it ignores the district court’s analysis and conducts its own.

Maj. Op. 40.

Because of these concerns, I would follow the parties’ lead and analyze the district

court’s actual application of the Daubert factors. Doing so, I’m not persuaded that the

district court erred; “nothing in the controlling legal authority” requires the “extremely

high degree of intellectual purity” that Dominic Hunt presses on appeal. United States v.

Baines, 573 F.3d 979, 989 (10th Cir. 2009). To be sure, neither this concurrence nor the

majority opinion should be read to endorse the rote admission of toolmark-identification

experts. See United States v. Smith, 756 F.3d 1179, 1193 n.13 (10th Cir. 2014) (“Firearm

toolmark analysis has recently come under attack for depending on subjective judgment,

rarely using control weapons, and risking an observer effect.”). But I see no abuse of

discretion in the district court’s decision in this case. See United States v. Foust, 989 F.3d

842, 847 (10th Cir. 2021) (finding no abuse of discretion in admitting handwriting expert

despite “criticism of handwriting expertise in both the courts and academic literature”). I

accordingly concur.

4