Stupp Co. v. United States

Slip Op. 23-23

UNITED STATES COURT OF INTERNATIONAL TRADE

STUPP CORPORATION ET AL.,

Plaintiffs and Consolidated
Plaintiffs,

and

MAVERICK TUBE CORPORATION,

Plaintiff-Intervenor and
Consolidated Plaintiff-Intervenor,
Before: Claire R. Kelly, Judge
v.
Consol. Court No. 15-00334
UNITED STATES,

Defendant,

and

SEAH STEEL CORPORATION AND
HYUNDAI STEEL COMPANY,

Defendant-Intervenors and
Consolidated Defendant-
Intervenors.

OPINION AND ORDER

[Sustaining the U.S. Department of Commerce’s third remand redetermination in the
less-than-fair-value investigation of welded line pipe from the Republic of Korea.]

Dated: February 24, 2023

Jeffrey M. Winton and Jooyoun Jeong, Winton and Chapman PLLC, of Washington,
D.C., argued for plaintiff SeAH Steel Corporation.
Consol. Court No. 15-00334 Page 2

Robert R. Kiepura, Trial Attorney, Commercial Litigation Branch, Civil Division,
U.S. Department of Justice, of Washington, D.C., argued for defendant. With him on
the brief were Claudia Burke, Assistant Director, Patricia M. McCarthy, Director,
and Brian M. Boynton, Principle Deputy Assistant Attorney General. Of Counsel
was Mykhaylo Gryzlov, Senior Counsel, Office of the Chief Counsel for Trade
Enforcement and Compliance, U.S. Department of Commerce, of Washington, D.C.

Jeffrey D. Gerrish, Schagrin Associates, of Washington, D.C., argued for plaintiff
Welspun Tubular LLC USA. With him on the brief were Roger B. Schagrin and Saad
Y. Chalchal.

Kelly, Judge: Before the court is the U.S. Department of Commerce’s

(“Commerce”) third remand redetermination in its 2015 less-than-fair-value

investigation of welded line pipe imported from the Republic of Korea (“Korea”). See

Final Results of Redetermination Purs. Ct. Remand, April 4, 2022, ECF No. 208

(“Remand Results”); see also Welded Line Pipe From [Korea], 80 Fed. Reg. 61,366

(Dep’t Commerce Oct. 13, 2015) (final determination of sales at less than fair value),

as amended by Welded Line Pipe From [Korea], 80 Fed. Reg. 69,637 (Dep’t Commerce

Nov. 10, 2015) (“Amended Final Determination”) and accompanying Issues &

Decisions Memo, A-580-876, (Oct. 5, 2015), ECF No. 30-3 (“Final Decision Memo”).

In Stupp Corporation v. United States, the Court of Appeals for the Federal Circuit

vacated this court’s opinion, remanding to Commerce to further explain why it is

reasonable to apply the Cohen’s d test as part of its differential pricing analysis if

certain statistical assumptions have not been met. Stupp Corporation v. United

States, 5 F.4th 1341 (Fed. Cir. 2021) (“Stupp III”). For the following reasons, the court

sustains Commerce’s third remand redetermination.
Consol. Court No. 15-00334 Page 3

BACKGROUND

The court presumes familiarity with the facts of this case as set out in this

court’s previous opinions, as well as the Court of Appeals’ decision in Stupp III, and

now recounts only the facts relevant to the court’s review of the Remand Results. On

November 14, 2014, Commerce initiated an antidumping duty investigation of welded

line pipe from Korea. Welded Line Pipe From [Korea], 79 Fed. Reg. 68,213, 68,213

(Dep’t Commerce Nov. 14, 2014) (initiation of less-than-fair-value investigation).

Commerce published its final determination on October 5, 2015 and, finding that

39.72% of SeAH Steel Corporation’s (“SeAH”) U.S. sales passed the Cohen’s d test,

applied the average-to-transaction method to those sales. Final Decision Memo. at 4.

Commerce accordingly calculated a 2.53% dumping margin for SeAH. Amended

Final Determination at 69,638. SeAH appealed, arguing that Commerce’s

differential pricing analysis and application of the Cohen’s d test were contrary to

law and unsupported by substantial evidence. See Stupp Corp. v. United States, 359

F. Supp. 3d 1293, 1302 (Ct. Int’l Tr. 2019) (“Stupp I”), reconsideration denied, 365 F.

Supp. 3d 1373 (Ct. Int’l Tr. 2019). SeAH also argued that Commerce improperly

rejected its case brief, which contained citations to certain academic texts not part of

the administrative record. Id. at 1300–03; Letter from Commerce Rejecting SeAH’s

Sept. 1, 2015 Case Br., 1–2, PD 384, bar code 3302027-01 (Sept. 3, 2015); [SeAH’s]

Case Br., PD 377–79, bar codes 3301610-01–03 (Sept. 1, 2015) (“SeAH’s Rejected

Brief”).
Consol. Court No. 15-00334 Page 4

This court sustained Commerce’s determinations with respect to its use of

differential pricing analysis and rejection of SeAH’s case brief. Stupp I, 359 F. Supp.

3d at 1299–1306. Specifically, the court found that Commerce correctly rejected

SeAH’s brief because the academic authorities cited in the brief constituted new

factual information intended to advance SeAH’s arguments. Id. at 1301. The court

also found that Commerce’s differential pricing analysis was supported by

substantial evidence because, among other reasons, Commerce was not required to

apply the Cohen’s d test in accordance with academic literature. Id. at 1302–06.

The Court of Appeals remanded, instructing Commerce to further explain why

its use of the Cohen’s d test was reasonable in light of “significant concerns” related

to application of the test. Stupp III, 5 F. 4th at 1357. Specifically, the Court of Appeals

questioned the reasonableness of Commerce’s application of Cohen’s d test to data

failing to satisfy the statistical criteria of normality, equal variance, and sufficient

observation size. Id. 1357–60. Citing to academic literature examining the use of

Cohen’s d test to measure effect size, the Court of Appeals expressed concern that

Commerce’s failure to satisfy the statistical criteria assumed by Cohen’s test could

“undermine the usefulness of the interpretive cutoffs,” resulting in artificially

inflated dumping margins. Id. at 1357. The Court of Appeals affirmed the remaining

issues from Stupp I, including this court’s decision to uphold Commerce’s rejection of

SeAH’s case brief. Id. at 1344.
Consol. Court No. 15-00334 Page 5

JURISDICTION AND STANDARD OF REVIEW

The court has jurisdiction pursuant to 28 U.S.C. § 1581(c) (2018), which grants

the court authority to review actions initiated under 19 U.S.C. § 1516a(a)(2)(B)(i)1

contesting the final determination in an antidumping duty order. The court will

uphold Commerce’s determination unless it is “unsupported by substantial evidence

on the record, or otherwise not in accordance with law.” 19 U.S.C. § 1516a(b)(1)(B)(i).

“The results of a redetermination pursuant to court remand are also reviewed ‘for

compliance with the court’s remand order.’” Xinjiamei Furniture Co. v. United

States, 968 F. Supp. 2d 1255, 1259 (Ct. Int’l Tr. 2014).

DISCUSSION

On remand, SeAH challenges Commerce’s application of the Cohen’s d test on

the grounds that (1) assumptions underlying the test have not been met, (2) the large

cutoff prescribed by the test is arbitrary, and (3) random variables such as exchange

rates can cause “false positives.” See Cmts. of [SeAH] on Final Determ. on Remand,

5–36, June 14, 2022, ECF No. 216 (“SeAH’s Cmts.”). Defendant and Welspun Tubular

LLC (“Welspun”) counter that (1) the assumptions are inapplicable, (2) Commerce’s

application of Cohen’s d test leads to reasonable results, (3) the cutoff is supported by

statistical literature, (4) SeAH cannot introduce non-record documents for the first

time on remand, and (5) SeAH failed to exhaust administrative remedies for its

1 Further citations to the Tariff Act of 1930, as amended, are to the relevant
provisions of Title 19 of the U.S. Code, 2018 edition.
Consol. Court No. 15-00334 Page 6

exchange rate-related arguments. See Def.’s Corr. Resp. to Cmts., 9–34, Sept. 22,

2022, ECF No. 230 (“Def.’s Reply”); [Welspun’s] Reply [SeAH’s] Cmts. on Remand

Redeterm., 18–32, Aug. 15, 2022, ECF No. 218 (“Welspun’s Reply”). For the following

reasons, the court sustains the results of Commerce’s remand redetermination.

I. SeAH’s Non-Record Documents

SeAH’s comments to the Remand Results reference several pieces of academic

literature which were not included in the administrative record. See SeAH’s Cmts.

at 6–36. Welspun and Defendant argue that the court should disregard these

materials, as judicial review is limited to the agency record. Welspun’s Reply at 19–

20; Def.’s Reply at 10–12. SeAH argues that the court may take judicial notice, or

otherwise consider, these materials to better understand the statistical principles

behind Cohen’s d test. Reply of [SeAH] to Responses by Def. and [Welspun], 10–11,

Sept. 28, 2022, ECF No. 236 (“SeAH’s Reply”). For the following reasons, the court

need not take judicial notice of SeAH’s non-record documents to understand the

statistical principles they illustrate.

Judicial review is generally limited to the administrative record before the

agency at the time it rendered its decision. See Camp v. Pitts, 411 U.S. 138, 142

(1973). “The purpose of limiting review to the record actually before the agency is to

guard against courts using new evidence to ‘convert the “arbitrary and capricious”

standard into effectively de novo review.’” Axiom Res. Mgmt, Inc. v. United States,
Consol. Court No. 15-00334 Page 7

564 F.3d 1374, 1380 (Fed. Cir. 2009) (quoting Murakami v. United States, 46 Fed. Cl.

731, 735 (2000), aff’d, 398 F.3d 1342 (Fed. Cir. 2005)).

This court previously upheld Commerce’s decision to reject SeAH’s non-record

documents, on the grounds that the submissions constituted new factual information

not on the administrative record. Stupp I, 359 F. Supp. 3d at 1299–1306. SeAH’s

submissions primarily cited academic articles relating to application of the Cohen’s d

test under certain conditions. SeAH’s Rejected Brief at 26–33. On appeal, the Court

of Appeals affirmed this court’s decision rejecting the non-record information,

concluding that SeAH’s materials were not introduced to correct inaccuracies in

Commerce’s reporting, but to support its argument challenging Commerce’s use of

Cohen’s d test. Stupp III, 5 F.4th at 1350. In Stupp III, the Court of Appeals

nevertheless referenced and quoted from several of the non-record texts introduced

by SeAH.2 Id. at 1357–59. On remand, Commerce asked SeAH to place the

2 The Federal Circuit cited the following five works: Grissom, Robert and Kim, John,
Effect Sizes for Research: Univariate and Multivariate Applications (2nd ed. 2012), A-
580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Grissom & Kim”); Coe,
Robert, It’s the Effect Size Stupid: What Effect Size Is and Why It Is Important, paper
presented at the Annual Conference of the British Educational Research Association
(September 2002), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Coe”);
Lane, David, et al., Introduction to Statistics, Online Edition, A-580-876, PRRD 8,
bar code 4181776-01 (Nov. 12, 2021) (“Lane”); Algina, James, Keselman, H.J., and
Penfield, Randall, An Alternative to Cohen’s Standardized Mean Difference Effect
Size: A Robust Parameter and Confidence Interval in the Two Independent Groups
Case, 10 Psychological Methods (2005), A-580-876, PRRD 8, bar code 4181776-01
(Nov. 12, 2021) (“Algina”); Li, Johnson Ching-Hong, Effect Size Measures in a Two-
Independent-Samples Case With Nonnormal and Nonhomogenous Data, Behavioral
Research (2015), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Li”).
Consol. Court No. 15-00334 Page 8

previously-rejected materials on the record, which SeAH did. See Letter from

[Commerce] to Interested Parties, A-580-876, PRRD 1, bar code 4176823-01 (Oct. 29,

2021); SeAH Submission of Publications Requested, A-580-876, PRRD 8, bar code

4181776-01 (Nov. 12, 2021). In its comments on the remand redetermination, SeAH

again cites new academic sources not on the record, arguing that the court may

consider the underlying statistical principles which the texts discuss.3 See SeAH’s

Cmts. at 6–25. SeAH claims that the Court of Appeals considered SeAH’s previous

academic sources in Stupp III, despite upholding Commerce’s rejection of SeAH’s

brief which contained these materials. SeAH’s Reply at 10–11. Although SeAH

states that the Court of Appeals took judicial notice of the texts, it later clarified that

the court may consider the statistical principles regardless of whether the texts

themselves are on the record. Response of [SeAH] to Def’s Sur-Reply, 2–3, Nov. 14,

2022, ECF No. 247 (“SeAH’s Sur-Reply”).4

3 SeAH cites to the following six non-record sources in its comments: Todd D. Little,
Oxford Handbook of Quantitative Methods in Psychology (2013); Ricca and Blaine,
Notes on a Nonparametric Estimate of Effect Size, 90:1 Journal of Experimental
Education 249 (2022); Hedges and Olkin, Overlap Between Treatment and Control
Distributions as an Effect Size Measure in Experiments, 21:1 Psychological Methods
61 (2016); Huberty and Lohman, Group Overlap as a Basis for Effect Size, 60:4
Educational and Psychological Measurement 543 (2000); J. Cohen, A Power Primer,
112:1 Psychological Bulletin 155 (1992); F. Alvarez, A. Atkeson, and P. Kehoe, If
Exchange Rates Are Random Walks, Then Almost Everything We Say about
Monetary Policy is Wrong, Federal Reserve Bank Of Minneapolis Research
Department Staff Report 388 (2007).
4 SeAH argues that the Court of Appeals’ decision “stands for the proposition that,

when an agency purports to be using a statistical test in accordance with widely-
adopted statistical practice, the courts may consider non-record academic materials
to evaluate that claim.” SeAH’s Sur-Reply at 2.
Consol. Court No. 15-00334 Page 9

Consistent with the approach of the Court of Appeals, the court may recognize

the basic statistical principles discussed in these texts. The idea, for example, that a

skewed statistical sample may yield inaccurate results is inductive reasoning—not

an assertion of fact. The Court of Appeals’ references to academia do not render its

reasoning dependent on academic sources. Thus, the court considers Commerce’s

Cohen’s d methodology in the same way it would review any other methodology, and

may make logical inferences without taking judicial notice of SeAH’s literature.

II. Administrative Exhaustion

SeAH argues that random fluctuations of exchange rates can affect the Cohen’s

d test, and lead to inaccurate results. SeAH’s Cmts. at 24–28. Welspun and

Defendant argue SeAH failed to properly exhaust this argument. Welspun’s Reply

at 20–21; Def.’s Reply. at 27–28. For the following reasons, the court concludes that

SeAH has exhausted this argument.

Pursuant to 28 U.S.C. § 2637(d), the court “shall, where appropriate, require

the exhaustion of administrative remedies,” including at the preliminary

determination stage before the agency. 28 U.S.C. § 2637(d); 19 C.F.R § 351.309(c)(2).

Section 2637(d) grants the court “discretion to identify circumstances where

exhaustion of administrative remedies does not apply.” ABB, Inc. v. United States,

920 F.3d 811, 818 (Fed. Cir. 2019) (quoting Consol. Bearings Co. v. United States,

348 F.3d 997, 1003 (Fed. Cir. 2003)). The court may also excuse exhaustion in certain
Consol. Court No. 15-00334 Page 10

circumstances, such as when a party is raising a “pure question of law.” Agro Dutch

Indus. Ltd. v. United States, 508 F.3d 1024, 1029 (Fed. Cir. 2007).

In its comments on the draft redetermination, SeAH argued that random

factors, such as exchange rates, could cause the standard deviation of test populations

to vary significantly. SeAH’s Cmts. on Draft Redetermination, 17–20, A-580-876,

PRRD 30, bar code 4224356-02 (March 21, 2022). SeAH did not provide an exchange

rate table, or assert that its actual sales during the period of review were affected by

these factors. See id. Subsequently, in its comments on the final remand results,

SeAH again argues that Cohen’s d could be significantly affected by random factors

where the population of data is not normally distributed. SeAH’s Cmts. at 24–28.

SeAH adds that its sales were, in fact, affected by fluctuations in the exchange rate

between the U.S. dollar and Korean won, because its inland freight expenses were

denominated in won. Id. at 24. Welspun counters that SeAH failed to raise its

exchange rate argument and supporting factual information during the draft

redetermination. Welspun’s Reply at 26.

SeAH has exhausted its exchange rate argument. SeAH’s exchange rate

examples provide an illustration of how it believes random factors can render the

Cohen’s d test inaccurate when values are not normally distributed. Normal

distributions is one of the three assumptions that the Court of Appeals remanded to

Commerce to explain. See Stupp III, 5 F. 4th at 1360. Therefore, Welspun’s argument

that Commerce had no opportunity to address SeAH’s exchange rate calculations
Consol. Court No. 15-00334 Page 11

misses the point; these calculations are not a new argument, but an illustration of

the same scenario Commerce was directed to explain.

SeAH separately argues Commerce must “ignore” fluctuations in exchange

rates pursuant to 19 U.S.C. 1677b-1(a). SeAH’s Br. at 8. SeAH concedes that it did

not raise this argument in its comments to the draft remand results; nevertheless, it

argues that this argument may be considered as a “pure question of law.” Oral

Argument at 0:27:39–0:27:53. Defendant argues that Commerce had no opportunity

to consider this argument on remand, and Welspun characterizes the argument as a

mixed question of law and fact. Oral Argument at 0:24:15–0:26:33, 0:27:56–0:28:39.

Whether § 1677b-1(a) is pertinent to Commerce’s differential pricing analysis is a

matter of statutory interpretation, not subject to exhaustion requirements. See Agro

Dutch Indus., 508 F.3d at 1029. However, SeAH’s argument that § 1677b-1(a) directs

Commerce to compensate for exchange rate variations is inapposite. In its full

context the statute directs Commerce to use the exchange rate “in effect on the date

of sale” for valuation of merchandise, and to ignore fluctuations on that particular

date. 19 U.S.C. 1677b-1(a). The plain language does not mandate that Commerce

compensate for a respondent’s decision to report expenses in a foreign currency, as

SeAH suggests.

III. Differential Pricing Analysis

In Stupp III, the Court of Appeals remanded for further explanation of

Commerce’s application of the Cohen’s d test as part of its differential pricing
Consol. Court No. 15-00334 Page 12

analysis. Stupp III, 5 F.4th at 1360. On remand, SeAH renews its argument that

Commerce’s application of Cohen’s d test is flawed because it fails to take into account

assumptions of sample size, distribution, and variance underlying the test, and

further argues Commerce’s choice of Cohen’s large cutoff is arbitrary. SeAH’s Cmts.

at 6–24. SeAH also claims random fluctuations in exchange rates can affect the d

coefficient, causing even test groups with identical prices to pass. Id. at 24–28.

Commerce counters that its Cohen’s d analysis does not operate in a vacuum, and

must be considered with the ratio test and meaningful difference test. See Remand

Results at 26, 28, 30–31, 41–42, 54–60. Commerce also argues the cutoffs are tied

to real-world criteria, that small fluctuations in price will not lead to “false positives”

in Cohen’s test, and that use of the 0.8 threshold results in reasonably infrequent

application of alternative methodologies. Remand Results at 16–19, 32, 54–60. For

the following reasons, Commerce has adequately addressed Court of Appeals’

concerns.

When investigating whether subject merchandise is being sold at less than fair

value, Commerce typically compares “the weighted average of the normal values to

the weighted average of the export (and constructed export prices) for comparable

merchandise” unless it determines another method is appropriate. 19 U.S.C. § 1677f-

1(d)(1)(A)(i); 19 C.F.R. § 351.414(c)(1). This average-to-average (“A-to-A”) method

compares the weighted average of a respondent’s home country sales prices during

the investigation period to the weighted average of the respondent’s sales prices in
Consol. Court No. 15-00334 Page 13

the United States during the same period. See 19 C.F.R. § 351.414(b)(1). One concern

with the A-to-A method is that it may allow a foreign producer or exporter to engage

in “targeted dumping,” which occurs when an exporter sells at a dumped price to

particular customers or regions, while selling at higher prices to other customers or

regions. See Apex Frozen Foods Priv. Ltd. v. United States, 862 F.3d 1337, 1341 (Fed.

Cir. 2017) (“Apex II”). As a result, higher-priced products can mask dumped products

when Commerce averages the sales using the A-to-A method.

Congress addressed concerns over targeted dumping with the passage of 19

U.S.C. § 1677f-1(d)(1)(B). See Apex II, 862 F.3d at 1342. Section 1677f-1(d)(1)(B)

allows Commerce to compare “the weighted average of the normal values to export

prices . . . of individual transactions for comparable merchandise if (i) there is a

pattern of export prices . . . for comparable merchandise that differ significantly

among purchasers, regions or periods of time, and (ii) [Commerce] explains why such

differences cannot be taken into account using [the A-to-A method or transaction-to-

transaction method5].” 19 U.S.C. § 1677f-1(d)(1)(B)(i)–(ii). Targeted dumping is more

likely when export prices fit a pricing model that differs significantly across different

market segments. Apex II, 862 F.3d at 1341–42. Congress has not provided a method

for Commerce to use to determine whether a pattern of significantly different prices

5Commerce’s regulations provide that the transaction-to-transaction method, which
compares prices of individual transactions, will be employed only in rare cases, “such
as when there are very few sales of subject merchandise and the merchandise sold in
each market is identical or very similar or is custom-made.” 19 C.F.R. § 351.414(c)(2).
Consol. Court No. 15-00334 Page 14

exists. However, the Statement of Administrative Action (“SAA”) of the Uruguay

Round Agreements Act explains that Commerce should proceed “on a case-by-case

basis, because small differences may be significant for one industry or one type of

product, but not for another.” Uruguay Round Agreements Act, Statement of

Administrative Action, H.R. Doc. No. 103-316, vol. 1, at 842–43 (1994), reprinted in

1994 U.S.C.C.A.N. 4040, 4178.6

To determine whether the criteria set forth in § 1677f-1(d)(1)(B) are met,

Commerce conducts a “differential pricing analysis” of a respondent’s sales. See

Differential Pricing Analysis; Request for Comments, 79 Fed. Reg. 26,720, 26,722

(Dep’t of Commerce May 9, 2014). This analysis contains three tests. First,

Commerce applies to respondent’s sales what it refers to as the “Cohen’s d test,”

described in more detail below, which measures the degree of price disparity between

groups of sales. Id. Commerce then counts the percentage of sales by value which

“pass” the Cohen’s d test, and applies its “ratio test.” Id. at 26,722–23. If 33% of

respondent’s sales or less pass, Commerce uses the A-to-A method, and if 66% or

more pass, Commerce uses the A-to-T method. Id. If the total percentage of passing

sales is between 33% and 66%, Commerce takes a hybrid approach, applying the A-

to-T method to those sales passing the test, and the A-to-A method to the remainder.

Id. Finally, if Commerce has not selected the A-to-A method for all sales, it applies

6 The SAA is an “authoritative expression by the United States concerning the
interpretation and application” of the Uruguay Rounds Agreement Act. 19 U.S.C.
§ 3512(d).
Consol. Court No. 15-00334 Page 15

the “meaningful difference” test to determine whether the A-to-A method could

nevertheless account for the disparate pricing. Id. at 26,723. Commerce applies the

test by comparing a respondent’s dumping margin using both A-to-A and the selected

method. Id. If the A-to-A margin is below the de minimis threshold and the margin

from the selected method is not, or if both margins are above the threshold and differ

by 25% or more, Commerce continues to use the selected method; otherwise,

Commerce applies the A-to-A method for all sales. Id.

As applied by Commerce, the Cohen’s d test involves comparing the product-

specific prices of “test groups” of a respondent’s sales to a “comparison group” by

region, purchaser, and time period. Stupp III, 5 F.4th at 1346. For each category,

Commerce segregates sales into subsets, with one subset becoming the test group,

and the remaining subsets being combined as the comparison group. Id. Commerce

then calculates the means and standard deviations of the test and comparison groups.

Id. Commerce then calculates a Cohen’s d coefficient by dividing the difference in the

groups’ means by the groups’ standard deviation.7 Id. Each subset is thus tested

against the remaining subsets across each category, and assigned a d coefficient by

7 Thus, d = |mean of test group – mean of control group| ÷ standard deviation.
Commerce uses a modified version of this formula, substituting the square root of the
simple average of the groups’ variances for standard deviation. The Cohen’s d test
solves for a coefficient representing “effect size.” See generally Cohen, Jacob,
Statistical Power Analysis for the Behavioral Sciences, (2nd ed. 1988), A-580-876,
PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Cohen”). “Effect size quantifies the
size of the difference between two groups, and may therefore be said to be a true
measure of the significance of the difference.” Coe at 7.
Consol. Court No. 15-00334 Page 16

solving Cohen’s ratio. If the d value of a test group is equal to or greater than the

“large threshold,” or 0.8, the observations within that group are said to have “passed”

the Cohen’s d test. Id. at 1347.

In Stupp III, the Court of Appeals expressed concern that Commerce’s

application of Cohen’s d under certain circumstances could undermine the usefulness

of the test in less-than-fair-value determinations. Specifically, the Court of Appeals

identified three potential scenarios in which use of Cohen’s d could be problematic:

first, when the distribution of a respondent’s sales data is not normal, second, when

the test groups have few data points, and third, when there is minimal variance in a

respondent’s sales. Stupp III, 5 F.4th at 1357–59. The assumption of normality is

satisfied when a fixed percentage of the population falls within each standard

deviation from the mean—in other words, that a population density graph generally

shows a symmetrical, bell-shaped curve. See Starnes, Yates, and Moore, Statistics

through Applications, 116 (2005), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12,

2021). The assumption of size is satisfied when the population is sufficiently large.

See Cohen at 21. The assumption of homogeneous variances is satisfied when the

standard deviations of test and comparison groups are similar. See Grissom at 68–

69. Commerce argued in Stupp III, as it does now, that the three assumptions are

only relevant as a matter of statistical significance, and do not apply when analyzing

a whole population; the Court of Appeals concluded that this explanation did not fully

address its concerns. Stupp III, 5 F.4th at 1360.
Consol. Court No. 15-00334 Page 17

The Court of Appeals illustrated the problems it identified with the Cohen’s d

test through two hypotheticals. First, the Court of Appeals considered a situation in

which Commerce analyzed a group of only eight export sales across four groups, such

that each test group would consist of only two sales. Stupp III, 5 F. 4th at 1358–59.

With groups of such small numbers, the Court of Appeals pointed out that there

would be some upward bias in effect size, such that the test would produce more

“passing” results, and potentially exaggerate dumping margins. Id. at 1359. The

Court of Appeals also observed that a group of only two sales would lack normality.

Id. Second, the Court of Appeals described a test group of five sales of about $100

each, which differed from one-another by up to two cents. Id. Because the standard

deviation of such a group would be so small, the Court of Appeals pointed out that

the denominator in Cohen’s ratio would be drastically reduced, again causing an

increase in effect size, and inflating the resulting dumping margin. Id. The Court of

Appeals noted that an objective examiner looking at these similar sales prices “would

be unlikely to conclude that they embody a ‘pattern’ of prices which ‘differ

significantly.’” Id. (citing 19 U.S.C. § 1677f-1(d)(1)(B)(i)).

Commerce reasonably explains that Cohen’s d test does not operate in a

vacuum, but as part of the differential pricing analysis as a whole.8 Turning first to

8 The parties devote a significant part of their briefings to discussion of (1) the
permissibility of using Cohen’s d test on full populations, and (2) questions of

(footnote continued)
Consol. Court No. 15-00334 Page 18

the assumptions of population size and normalcy, the Court of Appeals questioned

whether small sample sizes without normal distributions could “exaggerate”

dumping margins by introducing an “upward bias” to effect size. Stupp III, 5 F.4th at

1359.9 Addressing the Court of Appeals’ concerns about population size, Commerce

statistical significance versus practical significance. See Remand Results at 11–16,
43–51; SeAH’s Cmts. at 7–10; Def.’s Br. at 12–20; Welspun’s Br. at 21–23; SeAH’s
Reply at 17–27; Def’s Sur-Reply at 12–29, 34–35; SeAH’s Sur-Reply at 16–24. Neither
question is determinative of whether Commerce’s methodology is reasonable. Both
arguments have already been raised before the Court of Appeals, which concluded
that they did not resolve its concern over whether the absence of certain assumptions
forecloses Commerce’s use of Cohen’s d test. Stupp III, 5 F.4th at 1360.
Commerce correctly asserts that a “t-test” for statistical significance is used
with sampled data, and that Dr. Cohen considered normal distribution and equal
variance as necessary assumptions in a t-test. See Cohen at 19; Remand Results at
12–16. However, Commerce improperly reasons that because there is no need for a
t-test, there is no basis for the assumptions. Remand Results at 14. Commerce also
asserts that SeAH’s assumptions are only relevant as a matter of statistical
significance, and that they do not apply because Cohen’s d test determines practical
significance. Remand Results at 14, 43–45. That these assumptions are required
for questions of statistical significance does not answer the question of whether they
are also needed to determine practical significance, as the Court of Appeals suggests.
See Stupp III, 5 F.4th at 1360.
Although SeAH claims that academic sources do not support Commerce’s use
of Cohen’s d in its differential pricing analysis, this argument is inapposite. SeAH’s
decision to substantially advance its arguments using labels taken from statistical
literature does not alter the court’s obligation on review. See Soc Trang Seafood Joint
Stock Co. v. United States, 321 F. Supp. 3d 1329, 1339 n.13 (2018) (“the fact that
Commerce has adopted a methodology based upon a statistical tool known as Cohen’s
d, and chooses to refer to this methodology as Cohen’s d, does not diminish the
discretion granted to Commerce”); see also Mid Continent Steel & Wire, Inc. v. United
States, 31 F.4th 1367 (Fed. Cir. 2022) (“Commerce’s job is not to follow a statistical
test as explained in published literature for its own sake, but to implement the
statutory mandate to determine when prices of certain groups “differ significantly”).
9 Although the parties dispute whether such results are really “false positives,” it is

undisputed that in at least some instances, groups with as few as two sales have

(footnote continued)
Consol. Court No. 15-00334 Page 19

explains that its Cohen’s d analysis does not stand alone, and operates together with

the ratio test and meaningful difference test. See Remand Results at 26, 28, 30–31,

41–42, 54–60. Thus, even if the Cohen’s d values of small test groups were less

accurate than for large test groups, this difference does not by itself render

Commerce’s use of Cohen’s test unreasonable, because the ratio test and meaningful

difference test compensate for inaccuracies. See id. Commerce’s differential pricing

analysis looks at the frequency and impact of effect size to detect targeted dumping—

not the effect size alone. See Cohen at 8; Remand Results at 26–28. As Commerce

points out in its remand redetermination, the “sole purpose of the Cohen’s d test” is

to determine whether prices “differ significantly” across region, time period, or

customer. Remand Results at 41. The “pattern” of export prices which Commerce

must find under 19 U.S.C. § 1677f-1(d)(1)(B)(i) is then determined by the ratio test.

Id. at 42. The ratio test has already been approved by the Court of Appeals, which

found that Commerce’s choice of the 33% and 66% thresholds was a “reasonable

choice.” Stupp III, 5 F.4th at 1355. SeAH’s attacks on Cohen’s d test presuppose that

what SeAH claims are “false positives” automatically affect the accuracy of

passed Cohen’s test. See Stupp III, 5 F.4th at 1357; SeAH’s Cmts. at 17–19; see
Remand Results at 55, 58–59. Identifying results as “false” positives begs the
question of what is a false positive. See Remand Results at 59 (“To label this result
a ‘false-positive’ does not render the variances inaccurate or erroneous”). SeAH
illustrates this situation using data from its own sales, showing how a group of only
two sales to a single customer passed Cohen’s test, despite SeAH’s observation that
a visual comparison of the groups on a graph showed those sales to be near the
average price. SeAH’s Cmts. at 18. Commerce counters that a visual inspection may
be inadequate in situations involving complex calculations. Remand Results at 59.
Consol. Court No. 15-00334 Page 20

Commerce’s differential pricing analysis, when in fact Commerce has allowed for 33%

positives before there is any potential effect on a respondent’s dumping margins.

Commerce also addresses the Court of Appeals’ concern whether samples

without normal distributions will produce an inappropriate number of passes. SeAH

points to numerous academic sources which it claims confirm the usefulness of

Cohen’s test is compromised when comparing data sets with non-normal

distributions. See SeAH’s Cmts. at 7, n.19 (citing Cohen at 13); Id. at 12 (citing Ellis

at 41); Id. at 13 (citing Starnes, Yates, and Moore at 135). The Court of Appeals has

acknowledged some of these sources. See Stupp III, 5 F.4th at 1357–59 (citing Cohen

at 21, Grissom & Kim at 66, Coe at 13, Lane at 645, Algina at 318, and Li at 1571).

The court need not opine on the relevance of these academic observations;10 however,

it logically follows that a relatively large-tailed distribution (i.e., with large standard

deviation) in a test group would tend to decrease Cohen’s d coefficient, while the

opposite would result in an increase. See Remand Results at 29 (“in other words, the

fat-tailed distribution may undervalue the significance of effect”) (emphasis in

10 The task of the court is not to interpret the meaning of literature treating with
correct application of Cohen’s d. Rather, the court must determine whether
Commerce’s methodology is reasonable in light of considerations that run counter to
its decision. See Motor Vehicle Mfrs. Ass’n of U.S. v. State Farm Mut. Auto. Ins. Co.,
463 U.S. 29, 43 (1983); Ceramica Regiomontana, S.A. v. United States, 636 F. Supp.
961, 966 (Ct. Int’l Tr. 1986), aff’d, 810 F.2d 1137, 1139 (Fed. Cir. 1987) (“As long as
the agency’s methodology and procedures are reasonable means of effectuating the
statutory purpose, and there is substantial evidence in the record supporting the
agency’s conclusions, the court will not impose its own views as to the sufficiency of
the agency’s investigation or question the agency’s methodology”).
Consol. Court No. 15-00334 Page 21

original). SeAH focuses on the second of these two scenarios, arguing that even

inputting random data, such as exchange rates, can cause test groups to frequently

pass Cohen’s d test. SeAH’s Cmts. at 25–28. SeAH further argues that the ratio test

does not account for such random fluctuations.11 Oral Argument at 0:42:57–0:43:43.

Commerce addresses these arguments by explaining that even if Cohen’s test can

produce positive results under unusual circumstances, this possibility does not mean

its use of Cohen’s d is unreasonable when combined with the ratio test and

meaningful difference test. See Remand Results at 26, 28, 30–31, 41–42, 54–60.

The Court of Appeals also specifically asked Commerce to explain why it can

use the 0.8 threshold identified by Dr. Cohen as a measure of a significant price

difference, when Commerce evaluates data which fails to meet statistical

assumptions of normality, size and variance.12 Stupp III, 5 F.4th at 1360. Although

Commerce reiterates those assumptions are irrelevant, see Remand Results at 11–

11 Commerce explains that, even if exchange rate fluctuations do affect prices, this
effect is not “random” because a respondent can control in which currency it
denominates its prices. Remand Results at 45; Oral Argument at 0:49:11–0:50:05.
12 Although the Court of Appeals approved the 0.8 cutoff in Mid Continent Steel &

Wire, Inc. v. United States, it explained in Stupp III that it had yet to consider the
reasonableness of the 0.8 cutoff value when the assumptions in question have not
been met. Stupp III, 5 F.4th at 1356–57(“We held that . . . it is reasonable to adopt
that [0.8] measure where there is no better objective measure of effect size. We did
not, however, address SeAH’s second argument [on assumptions] in Mid Continent”)
(citation omitted) discussing Mid Continent Steel & Wire, Inc. v. United States, 940
F.3d 662, 673 (Fed. Cir. 2019); see also Oral Argument at 1:39:45–1:40:30. More
specifically, SeAH argues that it is unreasonable to compare its prices, which are not
normally distributed, using a subjective benchmark that was derived from a
normally-distributed population. SeAH’s Cmts. at 10–12.
Consol. Court No. 15-00334 Page 22

16, it also explains its choice of the 0.8 threshold as a function of its differential

pricing analysis. First, Commerce explains that it employs the 0.8 threshold to

identify where prices “differ significantly” pursuant to 19 U.S.C. § 1677f-1(d)(1)(B)(i).

Remand Results at 11. Second, Commerce states the 0.8 measurement “represents a

difference which is ‘grossly perceptible.’” Remand Results at 52. Reasonably

discernible from this statement is that Commerce considers a significant difference

to be grossly perceptible in the same way that Dr. Cohen identified a large threshold

as one that is “grossly perceptible.” See Cohen at 27. The SAA to the Uruguay Round

Agreements Act directs Commerce to proceed “on a case-by-case basis, because small

differences may be significant for one industry or one type of product, but not for

another,” SAA at 842–43; thus, Commerce’s choice of a measurement that is a

function of standard deviation as a uniform approach to identify differences as

significant is reasonable, even if the absolute difference in means is small.

Commerce’s approach tailors the question of what is a significant difference in price

to the pricing parameters of different products. Third, Commerce adequately

explains its adoption of Cohen’s widely-recognized choice of 0.8 as a large threshold

as significant. Remand Results at 18. It explains that it chose the 0.8 standard

because it was “a conservative standard to determine that the observed price

differences are significant.” Id. Commerce summarizes its reasoning by explaining

that “[u]sing Dr. Cohen’s thresholds is a reasonable approach to interpret whether

the difference in the prices is significant and the further interpretation of the
Consol. Court No. 15-00334 Page 23

difference in the prices in the context of the calculation of dumping margins ensures

the reasonable and limited application of the alternative comparison methodology.”

Id. at 33. Thus, Commerce chose a threshold it predicted would result in limited

application of the alternative methodology.

Although Commerce adopted this yardstick from Dr. Cohen, and did so because

it was widely acknowledged in the statistical literature, Commerce does not rely on

the prominence of this yardstick alone. Commerce elaborates that its “actual

application of the Cohen’s d test in the context of the differential pricing analysis

resulted in the application of an alternative comparison methodology to a relatively

small number of respondents.” Remand Results at 32. Discernible from Commerce’s

explanation is that the 0.8 cutoff produces reasonable passing rates once the ratio

and meaningful difference tests are applied. SeAH challenges Commerce’s reliance

on the 0.8 threshold as large, arguing that Commerce’s only basis for using the

threshold is that it is widely accepted. SeAH’s Cmts. at 10–11. However, in addition

to relying on a widely-accepted standard for “grossly perceptible” to determine what

is significant, Commerce defines “significant” with reference to the impact a price

difference has on a respondent’s dumping margins. Remand Results at 32. Finding

that the 0.8 threshold leads to relatively few determinations of targeted dumping,

Commerce concludes that its choice is reasonable. Id.

Congress delegated to Commerce the authority to determine where a price

difference is significant. 19 U.S.C. § 1677f-1(d)(1)(B)(i). Congress also made clear
Consol. Court No. 15-00334 Page 24

that the definition of a “significant price difference” would depend on the product at

issue. See SAA at 842–43. Thus, Congress entrusted Commerce to use its expertise

and knowledge of pricing to gauge price distinctions. Cf. Fujitsu General Ltd. v.

United States, 88 F.3d 1034, 1039 (Fed. Cir. 1996) (granting Commerce significant

deference in determinations “involv[ing] complex economic and accounting decisions

of a technical nature”). Commerce’s decision to adopt Cohen’s 0.8 (“large”) threshold

as a measure of significance because it is widely accepted in the statistical literature

does not undermine the reasonableness of that choice, if it is based on Commerce’s

expertise and Commerce demonstrates the reasonableness of that choice with

reference to the impact it has on the differential pricing analysis. Thus, Commerce’s

reference to Cohen’s work does not circumscribe its discretion to choose the same

values in a new context, because that choice is itself reasonable.

Commerce addresses the Court of Appeals’ concern that prices with small

variances, which hover around the same value, will produce inaccurate results on

Cohen’s test. As an initial matter, Commerce explains that results which pass

Cohen’s test under these circumstances are not “false positives,” as small differences

in average prices will mean that variances, too, will be small. Remand Results at 59;

Oral Argument at 1:06:32–1:07:02. Thus, it is discernable that a small variance

means a small difference in price will be more significant, and a passing result under

these circumstances is not necessarily “erroneous.” Remand Results at 59.

Nevertheless, the Court of Appeals observed that an objective examiner considering
Consol. Court No. 15-00334 Page 25

a group of sales where prices differed by only a few cents would be unlikely to conclude

that they show a “pattern” of prices that “differ significantly” under the statute.

Stupp III, 5 F.4th at 1359. Commerce responds to this issue by pointing out that an

examiner would indeed conclude that there was no pattern—because Commerce does

not look for a pattern at this stage of its differential pricing analysis. Remand Results

at 41. Again, Commerce explains that the ratio test determines whether a pattern

exists, while Cohen’s d test only shows whether there are significant price differences.

Id. at 41–42. Thus, Cohen’s test would need to generate enough “false positives” to

overcome the 33% threshold, at minimum, and there is no evidence on the record

suggesting that price patterns, such as that proposed by the Court of Appeals, occur

with frequency in SeAH’s sales.

Additionally, to specifically address the hypothetical proposed by the Court of

Appeals, Commerce explains that, in addition to the ratio test, the meaningful

difference test would prevent low-variance sales which pass Cohen’s d test from

impacting a respondent’s dumping margins. See Remand Results at 30–31. Adopting

the Court of Appeals’ example in which all of a respondent’s prices hovered around

$100 and passed Cohen’s test, Commerce explains that even in this extreme scenario,

the respondent would still be assessed under the A-to-A method. Id. Choosing a

normal value for comparison equal to the highest sales price, and thus maximizing

the respondent’s theoretical dumping margin, Commerce observes the margin would

still be under the 2% de minimis threshold. Id.; see 19 U.S.C § 1673d(a)(4),
Consol. Court No. 15-00334 Page 26

1673b(b)(3). SeAH argues that Commerce’s reliance on the meaningful difference

test is misplaced, because even changes of less than 2% in a respondent’s dumping

margin can cross the de minimis threshold and result in a “meaningful difference”

finding. SeAH’s Sur-Reply at 15. Specifically, SeAH argues that when the Cohen’s

d results from small-variance data sets of different products are cumulated,

Commerce may find that a respondent’s sales pass the thresholds for both ratio test

and the meaningful difference test, even if price differences are negligible. Oral

Argument at 1:13:37–1:16:06.13 This argument overstates Commerce’s position. It is

reasonably discernable that Commerce does not rely on the meaningful difference

test to prevent all “inappropriate” passes from affecting a respondent’s dumping

margins. Commerce has explained the meaningful difference test compensates for a

specific concern with low-variance sales which the Court of Appeals identified. See

Stupp III, 5 F.4th at 1359; Remand Results at 30–31. Moreover, SeAH’s argument is

misplaced, because the question before the court is not whether it is possible to

construct an unusual scenario where Cohen’s d test can result in an alternative

comparison method. Rather, the question is whether Commerce’s use of Cohen’s test,

when applied as a component of its differential pricing analysis, is reasonable. See

Ceramica Regiomontana, S.A. v. United States, 636 F. Supp. 961, 966 (Ct. Int’l Tr.

13SeAH does not argue that it received an alternative method because its own
combined sales inappropriately passed the Cohen’s d test. Rather it offers a
hypothetical to challenge the reasonableness of Commerce’s methodology more
generally. SeAH’s Cmts. at 8, 14–21.
Consol. Court No. 15-00334 Page 27

1986). Thus, for the forgoing reasons, Commerce has adequately explained how its

methodology is reasonable.

CONCLUSION

For the foregoing reasons, Commerce’s remand results are supported by

substantial evidence and comply with the court’s Order, Oct. 8, 2021, ECF No. 192,

in conformity with the Court of Appeals’ Mandate, Oct. 8, 2021, ECF No. 191, and are

therefore sustained. Judgment will enter accordingly.

/s/ Claire R. Kelly
Claire R. Kelly, Judge

Dated: February 24, 2023
New York, New York