IN RE: ACCUTANE LITIGATION (MCL NO. 271, ATLANTIC COUNTY AND STATEWIDE) (CONSOLIDATED)

NOT FOR PUBLICATION WITHOUT THE APPROVAL
OF THE APPELLATE DIVISION

SUPERIOR COURT OF NEW JERSEY
APPELLATE DIVISION
DOCKET NO. A-4698-14T1
A-0910-16T1
APPROVED FOR PUBLICATION

IN RE: ACCUTANE LITIGATION July 28, 2017

APPELLATE DIVISION

Argued March 7, 2017 – Decided July 28, 2017

Before Judges Reisner, Koblitz and Sumners.

On appeal from the Superior Court of New
Jersey, Law Division, Atlantic County, Case
No. 271 (MCL).

Bruce D. Greenberg and David R. Buchanan
argued the cause for appellants (Seeger Weiss,
LLP, Lite, DePalma, Greenberg, LLC, and Weitz
& Luxenberg, PC, attorneys; Mr. Buchanan and
Peter Samberg, of counsel; Mr. Buchanan, on
the briefs).

Paul W. Schmidt (Covington & Burling, LLP) of
the District of Columbia bar, admitted pro hac
vice, argued the cause for respondents Hoffman
LaRoche, Inc. and Roche Laboratories, Inc.
(Gibbons PC, Dughi Hewit & Domalewski, PC and
Mr. Schmidt, attorneys; Michelle M. Bufano,
Natalie H. Mantell, Russell L. Hewit, Mr.
Schmidt and Michael X. Imbroscio (Covington &
Burling, LLP) of the District of Columbia bar,
admitted pro hac vice, of counsel; Ms. Bufano,
on the brief).

Hollingsworth LLP, attorneys for amicus curiae
Pharmaceutical Research and Manufacturers of
America (Gregory S. Chernack, of counsel and
on the brief).
 The parties have not filed briefs in A-0910-
16.

The opinion of the court was delivered by

REISNER, P.J.A.D.

Plaintiffs, in these 2076 multicounty litigation (MCL)

products liability cases, alleged that they developed Crohn's
1
disease as a result of taking Accutane (isotretinoin), a

prescription acne drug manufactured by defendants Hoffman-La Roche

Inc. and Roche Laboratories Inc. (collectively Roche or

defendants). After a Kemp2 hearing, the trial court issued a

February 20, 2015 order granting defendants' omnibus motion to bar

plaintiffs' experts - Dr. David Madigan, a statistician, and Dr.

Arthur Kornbluth, a gastroenterologist - from testifying, among

other things, that the epidemiology studies on which the defense

relied were flawed and unreliable, and that Accutane can cause

Crohn's disease. The trial court also directed the parties to

prepare an order listing the lawsuits affected by the ruling, and

subsequently issued a May 8, 2015 order dismissing 2076 MCL claims

1
Crohn's disease is a form of inflammatory bowel disease (IBD).
2
Kemp ex rel. Wright v. State, 174 N.J. 412, 417 (2002).

2 A-4698-14T1
with prejudice. Plaintiffs appeal from those orders.3

On this appeal, plaintiffs primarily contend that the trial

court misapplied its discretion in finding that the methodologies

Madigan and Kornbluth used were scientifically unreliable and

inadmissible. After reviewing the record, we reverse the orders

on appeal and remand this case to the trial court.

We agree with plaintiffs that the trial court went beyond its

gatekeeping function, as set forth in Rubanick v. Witco Chemical

Corp., 125 N.J. 421, 449 (1991), Landrigan v. Celotex Corp., 127

N.J. 404 (1992), and Kemp, supra, 174 N.J. at 412.4 The trial

court took too narrow a view in determining whether the experts

were using accepted scientific methodologies to analyze the

evidence, and improperly determined the weight and credibility of

the experts' testimony. Among other things, the judge

3
In a second appeal (A-0910-16), ninety-eight plaintiffs appeal
from a September 19, 2016 order dismissing their complaints on the
same basis. By order dated December 7, 2016, we granted an
unopposed motion to consolidate A-0910-16 with the current appeal,
A-4698-14 (the first appeal); however, we excused the parties in
the second appeal from filing briefs or appendices, based on their
agreement to be bound by the outcome of the first appeal.
4
Plaintiffs also argue that the trial court erred as a matter of
law in applying the strict, scientific certainty admissibility
standard, instead of the relaxed standard set forth in Rubanick.
That argument is without sufficient merit to warrant discussion.
R. 2:11-3(e)(1)(E).

3 A-4698-14T1
inappropriately condemned the experts for relying on relevant

scientific evidence other than epidemiological studies, despite

their plausible explanations for doing do. 5 Consequently, we

conclude that the trial court mistakenly exercised discretion in

barring the experts' testimony.

In reaching our conclusion, we emphasize that we are not

placing this court's imprimatur on plaintiffs' experts or on their

opinions. The experts on both sides are highly reputable

scientists, who view the evidence differently. We find no basis

to describe plaintiffs' experts pejoratively as "hired guns," any

more than the defense experts are "hired guns." Their testimony

should not have been barred because their analyses emphasized

different evidence and produced different conclusions than those

reached by the defense experts. The fact that plaintiffs' experts

found certain evidence to be critically important did not

constitute improper "cherry picking," because they provided

plausible scientific explanations for their choices. See State

v. Dreher, 302 N.J. Super. 408, 464 (App. Div. 1997) ("Expert

testimony should not be excluded merely because it fails to account

5
Those same types of evidence were held admissible by a prior
judge, who had handled the Accutane MCL litigation for a decade.

4 A-4698-14T1
for some condition or fact that the opposing party considers

relevant.").

We are not predicting whether a jury will find plaintiffs'

experts - or defendants' experts - credible or persuasive. That

is not our role, as it was not the trial court's role in the Kemp

hearing. See Hisenaj v. Kuehner, 194 N.J. 6, 24 (2008) (N.J.R.E.

104 hearings "are intended to determine admissibility, not

credibility."). We only hold that, on the record created in the

Kemp hearing in this case, the plaintiffs' experts provided well-

explained scientific reasons for analyzing the available evidence

differently from the defense experts, and for relying more heavily

on different evidence than the defense experts relied on.

Accordingly, plaintiffs are entitled to present the experts'

testimony at trial.

I

This case cannot be viewed in a vacuum. It is one in a long

series of mass tort litigations concerning Accutane.6 We need not

6
McCarrell v. Hoffman-La Roche, Inc. (McCarrell I), No. A-3280-
07 (App. Div. Mar. 12, 2009), certif. denied, 199 N.J. 518 (2009);
Kendall v. Hoffman-La Roche, Inc. (Kendall I), No. A-2633-08 (App.
Div. Aug. 5, 2010), aff'd, 209 N.J. 173 (2012); Sager v. Hoffman-
La Roche, Inc., No. A-3427-09 (App. Div. Aug. 7, 2012), certif.
denied, 213 N.J. 568 (2013); Gaghan v. Hoffman-La Roche, Nos. A-
2717-11, A-3211-11, A-3217-11 (App. Div. Aug. 4, 2014); McCarrell

5 A-4698-14T1
review the history in detail, as it is set forth in a series of

previous unpublished opinions issued by different panels of this

court. We summarize only what is important to this case.

For more than a decade, the same trial judge had handled the

Accutane cases. To some extent, that judge's familiarity with the

prior litigation, and with the multiplicity of scientific issues

involved, may have shaped the way the parties and their experts

prepared for the current litigation.7 The first judge's rulings

no doubt also shaped the parties' litigation strategies.

In particular, during the course of the litigation, the first

judge determined that the opinions of plaintiffs' experts, based

on the same types of evidence relied on by plaintiff's experts in

this case, would be admissible as scientifically reliable. We

v. Hoffman-La Roche, Inc. (McCarrell II), No. A-4481-12 (Aug. 11,
2015), rev’d and remanded, 227 N.J. 569 (2017); Kendall v. Hoffman-
La Roche, Inc. (Kendall II), No. A-0301-14 (June 16, 2016);
and Rossitto v. Hoffman-La Roche, Nos. A-1236-13, A-1237-13 (July
22, 2016), certif. denied, 228 N.J. 419 (2016).
7
Both of the parties' epidemiology experts (Dr. Madigan and Dr.
Steven N. Goodman) who had testified previously, expressed their
belief that, to some extent, their current reports and testimony
would be viewed in light of their testimony in previous Accutane
trials.

6 A-4698-14T1
affirmed that determination in McCarrell I, supra, A-3280-07,8

finding that animal studies, case reports, analogous medications,

and other evidence relied on by plaintiffs' experts, were types

of evidence accepted in the scientific community.9

In the present case, defendants contend that the existence

of epidemiological studies now precludes reliance on the other

types of evidence on which plaintiff's experts had previously

relied. However, the studies on which defendants rely are not the

controlled clinical trials that the Federal Judicial Center's

Research Manual on Scientific Evidence calls "the gold standard"

of scientific evidence. Rather they are observational studies that

8
Unpublished opinions are not to be cited as legal precedent, and
we do not do so here. R. 1:36-3. However, it is appropriate to
consider an unpublished opinion of this court where, as here, it
forms a part of the history of the case on appeal. See Mountain
Hill, L.L.C. v. Twp. Comm. of Twp. of Middletown, 403 N.J. Super.
146, 155 n.3 (App. Div. 2008), certif. denied, 199 N.J. 129 (2009).
Moreover, an unpublished opinion of this court is binding on the
trial court in the same case. Ibid. The parties have not briefed
and, hence, we do not decide, whether an unpublished opinion of
this court is binding on the trial court in the same MCL docket,
albeit in a different case within that docket.
9
In a 2014 oral opinion, the first trial judge made a detailed
analysis of similar testimony by Dr. David Sachar and Dr. Madigan
concerning the connection between Accutane and ulcerative colitis.
The first judge concluded that the expert testimony, which relied
on very similar types of evidence as that used in this case, was
admissible. That decision was appealed, but was settled before
we decided the appeal. See Kendall II, supra, A-0301-14.

7 A-4698-14T1
depend on the collection of information from databases or from

patient questionnaires. Plaintiffs' experts testified that the

studies are biased and otherwise flawed. We conclude that

plaintiffs should be entitled to present that testimony at trial,

along with their affirmative evidence in support of their case.

II

We begin with some background as to Accutane, the

epidemiological studies of the drug, and relevant scientific

principles of epidemiology.

A. Accutane

In 1982, the Food and Drug Administration (FDA) approved

defendants' application to market Accutane, the brand name for

isotretinoin, "to treat recalcitrant nodular acne that has not

responded to other regimens." Kendall I, supra, 209 N.J. at 180.

The drug is a retinoid, derived from vitamin A, and is very

effective in treating severe acne. Ibid. It is well established

that Accutane "has a number of known side effects, including dry

lips, skin and eyes; conjunctivitis; decreased night vision;

muscle and joint aches; elevated triglycerides; and a high risk

of birth defects if a woman ingests the drug while pregnant."

Ibid. There is also some evidence that Accutane, which was

8 A-4698-14T1
originally studied for use in treating cancer, has an effect on

the gastrointestinal tract.

The MCL cases concern the alleged propensity of Accutane to

cause IBD, a chronic disease which primarily manifests as one of

two diseases: Crohn's disease or ulcerative colitis. Id. at 180-

81. Although both ulcerative colitis and Crohn's disease share

the same core symptoms, including abdominal pain, frequent and

often bloody bowel movements, and rectal bleeding, there are

differences in the clinical presentation of the disease and the

triggers statistically associated for developing it, which include

family history, infections, frequent use of some antibiotics,

smoking, and possibly the use of oral contraceptives and

nonsteroidal anti-inflammatory drugs. Id. at 181.

The peak onset of the disease occurs during adolescence—the

same period that individuals are likely to have been prescribed

Accutane. Ibid. For both diseases there may be a significant

latency effect (the time from the exposure to the trigger for IBD

to the first symptom of the disease) and a prodromal period (the

time from the first symptom of the disease to diagnosis).

B. Epidemiological studies

For the first six years of this MCL litigation, from 2003 to

9 A-4698-14T1
2009, there were no epidemiological studies regarding Accutane and

IBD. In previous trials, the plaintiffs were permitted to present

expert testimony that relied on animal studies, human clinical

studies, case reports, class effects, published scientific

literature, causality assessments, and biological plausibility.

McCarrell I, supra, slip op. at 86; Kendall I, supra, slip op. at

85-86; Sager, supra, slip op. at 20.

The first two epidemiological studies (Crockett and

Bernstein), 10 were published in 2009 and in 2010, finding no

statistically significant increased risk of developing Crohn's

disease from the use of Accutane, although the Crockett study

found ulcerative colitis is associated with exposure to the drug.

The Crockett and Bernstein studies were addressed in expert

testimony in Gaghan, McCarrell II, and Rossitto. In Kendall II,

the expert witnesses addressed four new epidemiological studies

10
Seth D. Crockett et al., Isotretinoin Use and the Risk of
Inflammatory Bowel Disease: A Case-Control Study, 105 Am. J.
Gastroenterol. 1986 (Sept. 2010) (ulcerative colitis but not
Crohn's disease is associated with isotretinoin use); Charles N.
Bernstein et al., Isotretinoin is not Associated with Inflammatory
Bowel Disease: A Population-Based Case-Control Study, 104 Am. J.
Gastroenterol. 2774 (Nov. 2009) (unlikely that isotretinoin use
is associated with development of IBD).

10 A-4698-14T1
(Etminan, Alhusayen, Fenerty, and Racine).11 After the trial in

Kendall II, two additional studies were published (Rashtak and

Sivaraman).12 The epidemiological studies vary in whether they

show that Accutane increases or decreases the risk of developing

Crohn's disease. However, with one exception, none of them

demonstrates a statistically significant increased risk of

developing Crohn's disease from exposure to Accutane. One small

study (Sivaraman) did find a statistically significant increased

risk. However, when the study authors adjusted the study results

for antibiotic use, the results were no longer statistically

significant. Plaintiffs' experts questioned the appropriateness

of that adjustment.

C. Epidemiology

11
Mahyar Etminan et al., Isotretinoin and Risk for Inflammatory
Bowel Disease, 149 JAMA Dermatol. 216 (Feb. 2013) (no increased
risk for IBD); Raed O. Alhusayen et al., Isotretinoin Use and the
Risk of Inflammatory Bowel Disease: A Population-Based Cohort
Study, 133 J. Invest. Dermatol. 907 (2013); Sarah Fenerty et al.,
Impact of Acne Treatment on Inflammatory Bowel Disease, 68 J. Am.
Acad. Dermatol. 6751 (Apr. 2013); Antoine Racine et al.,
Isotretinoin and Risk of Inflammatory Bowel Disease: A French
Nationwide Study, 109 Am. J. Gastroenterol. 563 (Apr. 2014).
12
Shadi Rashtak et al., Isotretinoin Exposure and Risk of
Inflammatory Bowel Disease, 150 JAMA Dermatol. 1322 (Dec. 2014);
Susil Silverman et al., Risk of Inflammatory Bowel Disease from
Isotretinoin: A Case Control Study (Oct. 2014).

11 A-4698-14T1
 In understanding the epidemiological studies, it is first

helpful to define the methodology used in conducting such studies

and the relevant terms, as testified by the experts at the hearing

and as set forth in the Federal Judicial Center, Reference Manual

on Scientific Evidence 549, 555 (3d. ed. 2011) (Reference Manual

or Manual).13 "Epidemiology is the field of public health and

medicine that studies the incidence, distribution, and etiology

of disease in human populations." Id. at 551. "Epidemiology

assumes that disease is not distributed randomly in a group of

individuals and the identifiable subgroups, including those

exposed to certain agents [such as prescription drugs], are at

increased risk of contracting particular diseases." Ibid.

Epidemiological studies identify agents that are associated with

an increased risk of a disease in groups of individuals, but "is

not equivalent to causation." Id. at 552.

There are two types of epidemiological studies: experimental

and observational. Id. at 555. Experimental studies, or double-

blind randomized control trials, in which one group is exposed to

13
Available at https://www.fjc.gov/sites/default/files/2015/
SciMan3D01.pdf. The epidemiological section was written by
Michael D. Green, among others, and is entitled Reference Guide
on Epidemiology.

12 A-4698-14T1
an agent and the other is not, are "considered the gold standard

for determining the relationship of an agent to a health outcome

or adverse side effect." Ibid. There are, however, no Accutane

experimental studies because although such studies have the

potential to provide higher quality evidence, they cannot

ethically be conducted if researchers suspect that a drug's side-

effects are harmful. Id. at 555-56.

Instead, all of the Accutane epidemiological studies to date

are less rigorous observational studies, which are considered to

be the next best available evidence. Id. at 556. There are two

types of observational studies: 1) a case-control study, which

measures and compares the frequency of exposure in the group with

the disease (cases) and a similar group without the disease

(controls); and 2) a cohort study, which compares a group of

exposed and unexposed individuals over a period of time. Id. at

557-59. In these studies, researchers "observe" individuals who

have already been exposed to the drug and compare them to a group

of individuals who have not. Id. at 555-56.

Unlike experimental studies in which risk factors can be

controlled, observational studies generally focus on individuals

living in a community, "for whom characteristics other than the

13 A-4698-14T1
one of interest, such as diet, exercise, exposure to other

environmental agents, and genetic background, may distort a

study's results." Id. at 556. "[T]he Achilles' heel of

observational studies is the possibility of differences in the two

populations being studied with regard to risk factors other than

exposure to the agent." Ibid.

Epidemiological studies commonly express the strength of

association between exposure to a drug and a disease in numerical

terms as: 1) "relative risk" (RR), the ratio of the incidence

rate of a disease in exposed individuals to the risk among the

unexposed; or 2) "odds ratio" (OR), the ratio of the odds that an

individual with the disease was exposed to the drug to the odds

that an individual without the disease was exposed. Id. at 566-

69.14 An RR of 1.0 means that the relative risk is equal to the

"null hypothesis," that is, that the risk in individuals exposed

to Accutane is the same as the risk in unexposed individuals, or

that Accutane use is not associated with an increased risk of

14
The Manual explains that an OR is "a convenient way to estimate
the relative risk in a case-control study when the disease under
investigation is rare." Id. at 568. Most of the studies at issue
in this case are case-control studies. An OR is calculated
somewhat differently in a cohort study but the difference is not
pertinent here.

14 A-4698-14T1
developing Crohn's disease. Id. at 567. If the RR is greater

than 1.0, the risk in exposed individuals is greater than the risk

in unexposed individuals. Ibid. For example, an RR of 1.5 means

that an exposed individual has a 50% greater chance of contracting

Crohn's disease. If the RR is less than 1.0, the exposed group

has a decreased risk of contracting the disease. Ibid. Thus,

an RR of .32 represents a 68% reduction in risk, which might mean

that the drug had a protective effect on developing the disease.

The OR or RR is, however, only an estimate of the true value.

Determining whether an association identified in an

epidemiological study is causal "requires an understanding of the

strengths and weaknesses of the study's design and implementation,

as well as a judgment about how the study findings fit with other

scientific knowledge." Id. at 553. An assessment must be made

of the power of the study to detect associations, the role of

chance, and what sources of error might have produced a false

result, including sampling variability, bias, and confounding

variables (extraneous variables that may affect result). Id. at

566-97.

Therefore, a showing of an increased relative risk for Crohn's

disease does not automatically prove that Accutane use creates a

15 A-4698-14T1
higher risk of developing the disease because the discrepancy

between the exposed and unexposed groups could be the product of

chance as a result of random sampling error. Id. at 553. In

determining whether a relative risk greater than 1.0 is a true

association or the result of random error, researchers consider

whether the association is statistically significant. Id. at 628.

In making that assessment, researchers calculate a p-value, which

"represents the probability that an observed positive association

could result from random error even if no association were in fact

present." Id. at 576. The p-value quantifies the statistical

significance of a relationship; the smaller the p-value the greater

the likelihood that associations determined in a study do not

result from chance. Id. at 626. The most commonly used p-value

is .05, that is for example, that there is a 5% chance that the

relative risk could have occurred by random error. Id. at 576-

77.

A more sophisticated approach, which was used in the studies

at issue in this case, involves calculating a confidence interval

(CI):

A confidence interval is a range of possible
values calculated from the results of a study.
If a 95% confidence interval is specified, the
range encompasses the results we would expect

16 A-4698-14T1
 95% of the time if samples for new studies
were repeatedly drawn from the same
population. . . . The advantage of a
confidence interval is that it displays more
information than significance testing.
"Statistically significant" does not convey
the magnitude of the association found in the
study or indicate how statistically stable
that association is. A confidence interval
shows the boundaries of the relative risk
based on selected levels of . . . statistical
significance. . . . [T]he confidence interval
reveals the likely range of risk estimates
consistent with random error.

[Id. at 580.]

If, for example, a study reveals a RR of 1.5, which represents

an elevated risk of developing Crohn's disease, that result might

or might not be considered statistically significant, depending

on the boundaries of the confidence interval. If the CI includes

1.0 (the null hypothesis, meaning that taking Accutane neither

increases nor decreases the risk of developing Crohn's disease),

then the 1.5 result is said not to be statistically significant.

However, if the CI is entirely above 1.0, for example if it ranges

from 1.2 to 3.2, then the 1.5 RR would be considered statistically

significant. Id. at 580-81.

In assessing whether the failure of a study to find a

statistically significant association was exonerative of the drug

or simply inconclusive, scientists consider the "power" of a study,

17 A-4698-14T1
or "the probability of finding a statistically significant

association of a given magnitude (if it exists) in light of the

sample sizes used in the study." Id. at 582. "The power of a

study depends on several factors: the sample size; the level of

statistical significance specified; the background incidence of

disease; and the specified relative risk that the researcher would

like to detect." Ibid. The higher the power of the study the

less likely it will show a false negative. Ibid. For example, a

study with a likelihood of .25 of failing to detect a true RR of

2.0, has a power of .75, meaning the study has a 75% chance of

detecting a true RR of 2.0. Ibid. On the other hand, a study

with low power has a greater likelihood of failing to detect a

significant relative risk, even though such a risk exists. "With

large numbers [of individuals included in the study group], the

outcome of the test is less likely to be influenced by random

error, and the researcher would have greater confidence in the

inferences drawn from the data." Id. at 576.

Under the proper circumstances, researchers can increase the

power of a series of studies by conducting a meta-analysis, which

involves pooling the results of different studies, some of which

are small and lack statistical power, to arrive at a single figure

18 A-4698-14T1
to represent the totality of the studies. Id. at 608. The Manual

indicates, however, that a meta-analysis may produce an unreliable

result.

The appeal of a meta-analysis is that it
generates a single estimate of risk (along
with an associated confidence interval), but
this strength can also be a weakness, and may
lead to a false sense of security regarding
the certainty of the estimate. A key issue
is the matter of heterogeneity of results
among the studies being summarized. If there
is more variance among study results than one
would expect by chance, this creates further
uncertainty about the summary measure from the
meta-analysis. Such differences can arise
from variations in study quality, or in study
populations or in study designs. Such
differences in results make it harder to trust
a single estimate of effect; the reasons for
such differences need at least to be
acknowledged and, if possible, explained.
People often tend to have an inordinate belief
in the validity of the findings when a single
number is attached to them, and many of the
difficulties that may arise in conducting a
meta-analysis, especially of observational
studies such as epidemiologic ones, may
consequently be overlooked.

[Ibid.]

III

We next address the parties' conflicting testimony on the

subjects of gastroenterology and epidemiology. As background, the

19 A-4698-14T1
following chart15 summarizes the epidemiological studies at issue

in this case:

STUDY DATABASE NO. OF RR for CD ACCUTANE STUDY
AND SUBJECTS at CI 95% EXPOSURE RESULTS
SUBJECTS PRIOR TO
DIAGNOSIS
Bernstein Canadian 21,500 1.15 2.6 years Positive
2009 Health (total) (0.61-2.02) association
manuscript Ins. 1118 (increased risk),
(case- (CD) but not SS
control)
Crockett US Health 29,000 0.68 1 year Negative
2010 Ins. (55 (total) (0.28-1.68) association
manuscript million) 3664 (decreased risk)
(case- (CD)
control) 0.89 2 year Negative
(0.32-2.52) association
(decreased risk)
Etminan US Health 45,000 1.05 1 year Positive
2013 Ins. (women (total) (0.5501.98) unadjusted
manuscript who had 1103 association
(case- taken oral (CD)
control) contracep- 0.91 Negative adjusted
tives) (0.37-2.25) association
(meta- (decreased risk)
analysis)
0.75 Negative
(0.46-1.24) association (meta-
analysis)
Racine French 44,000 0.45 1 to 2 SS protective
2014 Health (total) (0.24-0.85) years association
manuscript Ins. (47 2829 (reduced risk)
(case- million) (CD)
control)
Alhusayen Canadian 46,922 1.17 1 year Positive
2013 Database (treated (0.90-1.52) association
manuscript (4.5 with (increased risk),
(cohort) million) Accutane) but not SS

15
For purposes of the chart, we abbreviate Crohn's disease as "CD"
and statistical significance as "SS."

20 A-4698-14T1
STUDY DATABASE NO. OF RR for CD ACCUTANE STUDY
AND SUBJECTS at CI 95% EXPOSURE RESULTS
SUBJECTS PRIOR TO
DIAGNOSIS
Sivaraman US Patient 509 5.6 unknown Positive
2014 Question- (total) (1.1-28.0) unadjusted
abstract/ naire association and
poster from three SS for CD
(case- clinics
control) 4.8 Positive adjusted
(0.3-70) association, but
not SS for CD
Fenerty Marketscan 176,889 For IBD Negative
2013 Medicaid (total) 0.57 association for
abstract/ Database 324 (0.28-1.16) IBD (decreased
power-point (CD) risk)
(case- Not reported
control) for CD
Rashtak Mayo Clinic 1078 For IBD Negative
2014 patients (total) 0.28 association for
manuscript (0.10-0.79) IBD
(cohort) (decreased risk)
Not reported
for CD

A. Gastroenterology experts

1. Dr. Asher Kornbluth

Plaintiffs' expert, Dr. Kornbluth, was a highly qualified

expert who was board-certified in internal medicine and

gastroenterology and was a professor of medicine at Mount Sinai,

the preeminent hospital for IBD. He had specialized in Crohn's

disease for twenty-seven years, conducted research on IBD,

conducted clinical trials on several drugs intended for use in the

management of IBD, treated between 5,000 and 10,000 patients with

Crohn's disease, been retained as a consultant to pharmaceutical

21 A-4698-14T1
companies, and published more than 100 articles on IBD in peer-

reviewed scientific journals, textbooks, and other publications.

Kornbluth opined that Accutane can cause Crohn's disease in

humans. In reaching that conclusion, Kornbluth relied on his

personal experience in treating thousands of patients with the

disease. Additionally, he relied on some of the same evidence

that Dr. David Sachar, a previous plaintiffs' expert, had relied

on in seven previous Accutane trials in this MCL docket, 16

including animal studies, case reports, class effects of Vesanoid,

biological plausibility, scientific articles, internal studies,

causality assessments, and epidemiological studies. However, as

more fully discussed infra, both Dr. Kornbluth and Dr. Madigan

testified that most of the epidemiological studies done to date

were fundamentally flawed, thus warranting greater reliance on

other forms of scientific evidence.

Evidence of an Association

In accord with what he testified was the established

scientific methodology, Kornbluth first considered whether there

was an association between Accutane and Crohn's disease. In making

16
McCarrell I and II, Kendall I and II, Sager, Gaghan, and
Rossitto.

22 A-4698-14T1
that determination, he reviewed scientific articles, MedWatch

reports, epidemiological studies, and causality assessments, which

he found reflected a strong association between Accutane and

Crohn's disease.

a. Scientific articles

Kornbluth reviewed articles published in peer-reviewed

scientific journals, many of which analyzed a single anecdotal

case report, which he found supported a finding that there was an

association between Accutane and Crohn's disease.17 For example,

an article by Reddy and several colleagues reported that of the

17
P. Martin et al., Isotretinoin-Associated Proctosigmoiditis, 93
Gastroenterology 606 (1987) (case report); Philippe Deplaix et al.,
Acute Hemorrhagic Colitis Probably Due to Isotretinoin with
Recurrence Following Reintroduction of Treatment, 20
Gastroenterol. Clin. Biol. 113 (1996) (case report); Marianne Melki
et al., Granulomatous Colitis Probably Due to Isotretinoin, 25
Gastroenterol. Clin. Biol. 433 (2001) (case report); J.L.M. Passier
et al., Isotretinoin-Induced Inflammatory Bowel Disease, 64 Neth.
J. Med. 52 (Feb. 2006) (case reports); Deepa Reddy, M.D. et al.,
Possible Association Between Isotretinoin and Inflammatory Bowel
Disease, 101 Am. J. Gastroenterol. 1569 (July 2006) (adverse event
reports); Cristiano Spada, M.D. et al., Isotretinoin-associated
Pan-enteritis, 42 J. Clin. Gastroenerol. 923 (Sept. 2008) (case
report); Matthew Shale et al., Isotretinoin and Intestinal
Inflammation: What Gastroenterologists Need To Know, 58 Gut 737,
739 (June 2009) (study of adverse event reports concluding
isotretinoin may act as trigger for IBD in susceptible patients);
and Michael B. Brodin, M.D., Inflammatory Bowel Disease and
Isotretinoin, 14 J. Am. Acad. Dermatol. 843 (1986) (letter to
editor).

23 A-4698-14T1
approximately four or five million people that took Accutane

between 1997 and 2002, the FDA received eighty-five reports of

IBD. Using the Naranjo ADR probability scale, the authors found

that "4 cases (5%) scored in the 'highly probable' range for

isotretinoin as the cause of IBD, 58 cases (68%) were 'probable,'

23 cases (27%) were 'possible,' and no cases were doubtful."

Reddy, supra note 17, at 1571. The authors concluded that

"isotretinoin appears to be a potential precipitant of IBD." Id.

at 1572.

b. MedWatch reports

Kornbluth next reviewed a series of MedWatch reports, reports

which are made by physicians, patients and others to the FDA

listing among other information, a description of the adverse

event and whether it abated after the patient stopped using

Accutane and returned after reintroduction (referred to as

challenge/dechallenge/rechallenge). He found that if corrected

for underreporting, the number of MedWatch reports suggested a

strong association between Accutane and Crohn's disease.

c. Epidemiological studies

Kornbluth also reviewed six observational epidemiological

studies (Bernstein, Crockett, Alhusayen, Etminan, Racine, and

24 A-4698-14T1
Sivaraman), only one of which (Sivaraman) found a statistically

significant positive association (before adjustment for antibiotic

use) between Accutane and Crohn's disease, one found a

statistically significant negative association (Racine) and no

study concluded that Accutane use presents an increased risk for

developing Crohn's disease. 18 Kornbluth opined that despite

"significant flaws" in five of those studies that distorted the

results, the studies nonetheless provided some evidence of an

association between Accutane and Crohn's disease.

Kornbluth relied on the unadjusted results of the Sivaraman

study, which was summarized in a published abstract (not manuscript

form), and selected by the American College of Gastroenterology

to be presented as a poster at their annual conference to enable

peers and colleagues to discuss the findings with the researchers.

In that small study (509 patients), the authors initially found a

statistically significant association between Accutane and Crohn's

disease: that the risk of developing the disease was more than

five times higher in the group exposed to Accutane. The authors

18
The studies yielded different results for ulcerative colitis:
Crockett found a statistically significant association; Bernstein,
Etminan, Racine, and Alhusayen found a positive association, but
not a statistically significant association; and Fenerty and
Rashtak found a negative association.

25 A-4698-14T1
collected data using a questionnaire, which included information

about use of antibiotics, family history of IBD, and smoking,

thereby accounting for confounding variables. The authors then

"adjusted" the analysis to remove subjects who had taken

antibiotics, which Kornbluth said still yielded a "very striking

increased risk" of developing Crohn's disease from Accutane use,

although the adjusted sample size was too small to demonstrate

statistical significance. The authors concluded that

"isotretinoin exposure does not appear to confer risk for Crohn's

disease independent of antibiotic exposure." Kornbluth and

Madigan both questioned the basis for the adjustment the authors

made.19

Kornbluth opined that the results of the other five studies

(Bernstein, Crockett, Alhusayen, Etminan, and Racine), were

inconclusive because they: failed to account for the prodrome

(see section 1 below); were insufficiently "powered"; or contained

design flaws that biased or distorted the results to show a reduced

risk of developing Crohn's disease. Nonetheless, Kornbluth found

19
The authors noted that taking antibiotics alone did not appear
to affect the risk of developing Crohn's disease. Nonetheless,
they removed the subjects who had taken antibiotics when they
recalculated the study results.

26 A-4698-14T1
that the studies were informative in determining causation, noting

that four of the studies (Bernstein, Etminan (unadjusted),

Alhusayen, and Sivaraman) found a positive association between the

drug and the disease.

1. Prodrome

Kornbluth opined that four of the studies (Crockett,

Alhusayen, Etminan, and Racine) had not followed patients for long

enough to detect an effect from Accutane exposure and thus had

failed to account for the prodrome of Crohn's disease, that is,

the delay between the time of the first or early symptoms and the

diagnosis. He opined, based on his decades of experience as a

treating gastroenterologist, that the average prodrome for Crohn's

disease was from two to four years. He found support for that

opinion in several studies, including: 1) the Pimentel study,20 a

referral-based study (45 of the 66 subjects had Crohn's disease),

in which the authors found the mean prodrome for Crohn's disease

was 6.9 years; and 2) the Barratt study,21 in which the authors

20
Mark Pimentel, M.D. et al., Identification of a Prodromal Period
in Crohn's Disease but Not Ulcerative Colitis, 95 Am. J.
Gastroenterol. 3458 (Dec. 2000).
21
S.M. Barratt et al., Prodromal Irritable Bowel Syndrome May Be
Responsible For Delays In Diagnosis In Patients Presenting With

27 A-4698-14T1
found the mean prodrome was four years. He distinguished the

findings of a larger study by Chouraki22 in which the authors found

a three-month prodrome, based upon the selection of the patients

for the study and the use of patient charts as opposed to patient

questionnaires, as used in the Barratt and Pimentel studies.

Kornbluth explained that because the prodrome for Crohn's

disease is two to four years, a study that looks back only one-

year from diagnosis would not capture patients who developed

Crohn's disease from Accutane exposure 366 days to four years

after taking the drug. He noted, for example, that in the Crockett

study the odds ratio increased from 0.68 (one-year analysis) to

0.89 (two-year analysis), which he said was likely a result of

capturing more patients who had developed the disease. Similarly,

the Bernstein study, which looked back approximately 2.6 years,

found a positive association between Accutane and Crohn's disease,

which Kornbluth opined may also have resulted from capturing more

Unrecognized Crohn's Disease And Celiac Disease, But Not
Ulcerative Colitis, 56 Dig. Dis. Sci. 3270 (Nov. 2011).
22
V. Chouraki et al., The changing pattern of Crohn's disease
incidence in northern France: a continuing increase in the 10- to
19-year-old age bracket (1988-2007), 33 Aliment. Pharmacol. Ther.
1133 (2011). Chouraki was not a study of the prodrome for Crohn's
disease. It was a study of the increased incidence of the disease
in northern France.

28 A-4698-14T1
Crohn's disease patients than the other shorter studies. He found

that these four studies were not designed to accurately account

for all of the Accutane patients who had developed Crohn's disease,

thereby distorting the results. He opined that if the studies had

been designed to account for the long prodrome, the results would

have shown a greater increased risk of developing Crohn's

disease.23

2. Power

Next, Kornbluth opined that three of the studies (Bernstein,

Crockett, and Etminan), were insufficiently powered to detect a

statistically significant association; in other words, that the

sample size was not large enough to make a definitive conclusion

as to whether there was a statistically significant risk. For

example, in the Bernstein study, a large case-control study using

a Canadian database, the study population only comprised 1118

Crohn's disease cases out of a control population of 19,419. He

calculated that the "power" of the Bernstein study was low (about

25% to 30%), that is, there was only a 25% to 30% chance of

23
Kornbluth's view about the length of the prodrome was hardly
unique to him or scientifically unorthodox. It was supported by
the Pimentel and Barratt studies, references in Passier, supra
note 17, at 52, and admissions made by defendant's expert, Dr.
Oliva-Hemker.

29 A-4698-14T1
detecting a statistically significant association (greater than

2%) between Accutane and Crohn's disease. He explained that 80%

power was appropriate for a study. He opined that if these studies

had not been underpowered, the results would have been

statistically significant for an increased risk of developing the

disease.

However, Kornbluth admitted that where individual studies are

underpowered to detect outcomes, studies can be pooled using a

meta-analysis, to increase the power to detect a risk. One such

study was done by Etminan, which combined the Bernstein, Crockett,

Etminan (case-control study) and Racine studies, and found a pooled

RR of .75 with a CI of 0.46 to 1.24, indicating no increased risk

of developing Crohn's disease. Another study was done by Goodman,

defendants' expert, as discussed more fully infra, who found a

pooled RR of 0.87 with a CI of 0.59 to 1.28, or, again, no

statistically significant increased risk of developing Crohn's

disease. However, Kornbluth, like Madigan, rejected the results

of these meta-analyses, explaining that a meta-analysis using

underpowered and flawed studies (as he said existed in this case),

which did not account for the prodrome, are not informative and

should not be relied upon by scientists in determining causation.

30 A-4698-14T1
 3. Design flaws

Kornbluth also found the results of the five studies were

inconclusive because they had design flaws, including differences

in the populations and the failure to account for confounding

variables. For example, he opined that the Bernstein study, a

Canadian study, was flawed because of the differences in

recommended doses between the United States (higher dose) and

Canada (lower dose). He expected that given that difference there

would be far fewer cases of Crohn's disease in Canada thereby

decreasing the relative risk ratio. Similarly, there were

differences in recommended doses between the United States

(higher) and France (lower), which Kornbluth testified could

account for the Racine study's finding of a protective effect.

Next, Kornbluth found that most of the studies had failed to

account for smoking and family history confounders, that is,

alternative causes of Crohn's disease unrelated to Accutane

exposure that can bias the study by making the association appear

higher or lower than it actually is.24 For example, the Alhusayen

24
Many of the studies had accounted for other confounders,
including gender, oral contraceptive use, use of NSAIDs, and
antibiotics. And the Etminan study adjusted, in part, for patients
who had made a claim for tobacco cessation counseling.

31 A-4698-14T1
study, which comprised 46,922 patients treated with Accutane,

reported an unadjusted RR for Crohn's disease of 1.40, and an

adjusted (for antibiotic use) RR of 1.17. He testified that the

study should have adjusted for more relevant confounders,

including family history and smoking, which presumably would have

yielded a higher risk ratio of developing Crohn's disease from

Accutane. In other words, removing all of the individuals who had

a family history of Crohn's disease from the sample would yield a

better measure of whether Accutane use is associated with Crohn's

disease.

Lastly, Kornbluth did not consider the results of two

additional studies (Rashtak and Fenerty), because those studies

were designed to only report general IBD results, and did not

calculate the relative risk of developing Crohn's disease, which

has different triggers than ulcerative colitis and therefore did

not provide reliable information on the risk of developing Crohn's

disease.

d. Causality assessments

Next, Kornbluth considered defendants' internal causality

assessments of adverse drug experience (ADE) reports of Crohn's

disease in patients taking Accutane, in which defendants had

32 A-4698-14T1
concluded that there was an association between Accutane and the

disease. For example, in an internal causality assessment dated

December 17, 2002, defendants reported that there were 159 reports

of adverse events from exposure to Accutane received from worldwide

sources; of those patients, sixty-four had Crohn's disease, of

which Roche assessed causality as "related" in twenty-seven cases,

with the remainder designated either as unrelated or unknown.

Additionally, defendants concluded in an internal report dated

November 16, 2000, that "[i]sotretinoin has been found to be

causally associated with inflammatory bowel disease, including

colitis."

Similarly, in its "general data memo," a document that

reflected the company's scientific and medical opinion about

Accutane, defendants provided that IBD "is a possible side effect

of ROACCUTANE in very rare cases, possibly in patients predisposed

to inflammatory gastro-intestinal diseases," and that although the

side-effect "does not seem to represent a serious problem in

practice," it is "reasonable to conclude" that the drug is

"basically contraindicated" for patients "in the active phase" of

IBD. Kornbluth testified that those "strong statement[s]" by

defendants, who had a great deal of pharmacovigilance experience

33 A-4698-14T1
in assessing ADE reports, were significant in assessing whether

Accutane use was associated with Crohn's disease. In its core

data sheet, which is a compilation of information set forth in

labels or in reports sent to regulatory boards, defendants also

stated that IBD had been reported in Accutane users and that

adverse events are dose-related.

Existence of a Causal Relationship

After determining that there was an association between

Accutane and Crohn's disease based on the literature, MedWatch

reports, epidemiological studies, and causality assessments,

Kornbluth then considered whether that association reflected a

causal relationship.

a. Bradford Hill

As set forth above, epidemiology cannot prove causation.

Reference Manual, supra, at 598. In making the causation

determination, Kornbluth considered, among other factors, the

widely recognized criteria identified by Sir Austin Bradford Hill

(Bradford Hill criteria): (1) strength of the association; (2)

temporal relationship; (3) consistency of relationship; (4)

biological plausibility; (5) consideration of alternative

explanations; (6) specificity; (7) dose-response relationship; (8)

34 A-4698-14T1
replication; and (9) cessation of exposure. Sir Arthur Bradford

Hill, The Environment and Disease: Association or Causation, 58

Proc. Royal Soc'y of Med. 295, 299 (1965). In assessing these

criteria

[t]here is no formula or algorithm that can
be used to assess whether a causal inference
is appropriate based on these guidelines. One
or more factors may be absent even when a true
causal relationship exists. Similarly, the
existence of some factors does not ensure that
a causal relationship exists. Drawing causal
inferences after finding an association and
considering these factors requires judgment
and searching analysis, based on biology, of
why a factor or factors may be absent despite
a causal relationship, and vice versa.
Although the drawing of causal inferences is
informed by scientific expertise, it is not a
determination that is made by using an
objective or algorithmic methodology.

[Reference Manual, supra, at 600.]

1. Strength of association

Kornbluth opined that the association evidence (scientific

literature, MedWatch reports, epidemiological studies and

causality assessments) reflected a strong association between

Crohn's disease and Accutane, even though no medical organization

or epidemiological study had concluded that Accutane causes

Crohn's disease. He found that although the epidemiological

studies had "some major shortcomings," most of the studies

35 A-4698-14T1
nonetheless showed a "fairly substantial increased risk of

Accutane causing Crohn's disease."

2. Temporal relationship

Kornbluth found that the medical literature, MedWatch

reports, and animal studies supported a finding that there was a

temporal relationship between Accutane exposure and Crohn's

disease, that is, the timing of the exposure to the drug and the

onset of the disease was consistent with the lengthy latency and

prodromal period for the disease.

3. Biological plausibility

Although the precise mechanism by which Accutane could cause

Crohn's disease is unknown, Kornbluth opined that there was a

biologically plausible mechanism by which Accutane could cause

Crohn's disease, namely, that Accutane may cause the migration of

inflammatory T cells to the intestinal tract. Kornbluth explained

that retinoic acid, which is a metabolite of Accutane, causes and

perpetuates Crohn's disease by directing inflammatory T-cells,

using "antenna" known as "alpha 4 beta 7," to the intestines and

allowing the T-cells to bind to "receptors" (or "mucosal addressing

cell adhesion molecules" ("MAdCAMs")), which then spurs invasion

of inflammatory cells into the lining of the intestines. Without

36 A-4698-14T1
retinoic acid, the antenna (alpha 4 beta 7), does not imprint on

the T-cells and are not guided back to the intestines. He said

that studies have shown that blocking retinoic acid prevents

intestinal inflammation, which is characteristic of Crohn's

disease. In other words, without alpha 4 beta 7, one cannot "get

Crohn's disease, because the T cells that are the driver of the

inflammation have no way of getting into the small intestine."25

In fact, he noted that two new drugs (Vedolizumab and

Natalizumab), which Kornbluth said had been very effective in

treating Crohn's disease, operated to block the retinoic acid

antenna (alpha 4 beta 7), thereby preventing the T-cells from

binding to the MAdCAMs and entering the intestine where they cause

damage. The success of these drugs indicated to Kornbluth "that

retinoic acid is a damaging toxic pathway for patients with Crohn's

disease," because inhibition of the harmful molecule caused the

patient to get better.

Moreover, he noted that a Canadian case-control

epidemiological study on children supported his opinion on

biological plausibility because it reported that children

25
Kornbluth's explanation as to biological plausibility had some
support in scientific literature. See, e.g., Spada, supra note
17, at 24; Shale, supra note 17, at 737-39.

37 A-4698-14T1
ingesting dietary supplements of retinol (vitamin A), a compound

related to Accutane, which also breaks down into retinoic acid,

had a statistically significant (two-fold) increased risk of

developing Crohn's disease.26 There was a dose effect, in that

only the children taking higher than normal doses of retinol showed

an increased risk of developing Crohn's disease.

4. Dose relationship

Next, Kornbluth testified that there was a dose-related

relationship between Accutane and gastrointestinal injury—higher

doses cause greater injury—as set forth in the Core Data sheet,

dog studies, MedWatch reports, and epidemiological studies. He

explained that "a dose-response curve" is "scientific evidence"

of causation.

5. Consistency, coherence, and specificity

Lastly, Kornbluth testified that he had observed consistency

across different lines of evidence supporting a causal

relationship, including the MedWatch reports, dog studies, and the

epidemiological studies, which except for the Racine study,

26
Devendra K. Amre et al., Imbalances in Dietary Consumption of
Fatty Acids, Vegetables, and Fruits Are Associated with Risk for
Crohn's Disease in Children, 102 Am. J. Gastroenterol. 2016 (Sept.
2007).

38 A-4698-14T1
reported an increased risk of developing the disease.

He also found that the evidence was coherent with the

scientific understanding of the cause and presentation of the

disease. For example, evidence from the dog studies was consistent

with the knowledge of the pathogenesis of the disease in that a

breakdown of the epithelium (as observed in some of the dogs) can

serve as a trigger for Crohn's disease. The Vedolizumab and

Natalizumab studies demonstrated that blocking the effect of

retinoic acid (an Accutane metabolite) vastly improved a patient's

Crohn's disease.

Kornbluth did not, however, find any specificity for Crohn's

disease because the disease is not caused solely by Accutane use,

and it is not the only side-effect of taking the drug.

b. Other evidence of a causal relationship

1. Animal studies

In reaching his conclusion on causation, Kornbluth relied on

studies in which dogs were given high doses of Accutane (the dogs

achieved similar levels of the active metabolite as humans because

a dogs' metabolism is different). The studies reported

gastrointestinal upset, diarrhea, bloody mucoid stools, intestinal

adhesions, thickening of the mucosa, and epithelial damage, with

39 A-4698-14T1
crypt abscess formation as seen in Crohn's patients, in the treated

dogs. He explained that even though dogs cannot develop IBD, the

studies showed that Accutane can cause "significant obvious

symptomatic damage to the gastrointestinal tract" because a dog's

intestine "is quite analogous to the human intestinal tract."

2. Challenge/dechallenge/rechallenge reports

Kornbluth opined that the challenge/dechallenge/rechallenge

reports contained in the medical literature (Martin, Deplaix, and

Melki) and in the MedWatch reports were "very compelling" evidence

of causation. He explained that the reports of positive

rechallenges were significant because they were essentially a non-

deliberate human experiment, in that a potentially toxic substance

was reintroduced to a patient resulting in further injury.

3. Class effects

Kornbluth also reviewed side effects reported from use of

Vesanoid, a chemically similar retinoid manufactured by Roche used

to treat acute promyelocytic leukemia (APL). Chemically, Vesanoid

is tretinoin, an all-trans retinoic acid. Accutane, or

isotretinoin, another retinoid, metabolizes into tretinoin and 4-

oxo-isotretinoin. The Vesanoid package insert indicates that

gastrointestinal disorders, including gastrointestinal

40 A-4698-14T1
hemorrhage, were reported in thirty-four percent of clinical trial

patients. These results were significant because such a high

percentage of gastrointestinal disorders would not be expected,

even in APL patients, who have a "tremendous tendency to bleed."

Thus, he testified that this evidence supported his opinion that

Accutane can cause gastrointestinal injuries.

2. Dr. Maria Oliva-Hemker

Like Dr. Kornbluth, defendant's gastroenterology expert, Dr.

Oliva-Hemker, was highly qualified. She was board certified in

gastroenterology and was a professor of medicine at Johns Hopkins

University. She had treated hundreds of children who suffered

from IBD, published more than seventy peer-reviewed scientific

articles on IBD, and chaired various gastroenterology committees.

She opined that the available scientific evidence did not

support a finding that Accutane can cause Crohn's disease. She

testified that the scientific evidence supported a finding that

retinoic acid had an anti-inflammatory or protective effect on the

gastrointestinal tract. Moreover, she testified that all of the

epidemiological data--the best available evidence--reported no

increased risk of Crohn's disease associated with Accutane, which

was consistent with the biological evidence of a protective effect.

41 A-4698-14T1
She testified that Kornbluth had failed to follow well-recognized

principles of medical evidence hierarchy by relying on lower-level

data (such as case reports and animal studies), instead of higher-

level epidemiological evidence.

However, on cross-examination, Oliva-Hemker admitted that

Crohn's disease often has a lengthy prodrome. In fact, she

admitted that information was reflected in her own professional

writings.

B. Biostatistical and epidemiological experts

1. Dr. David Madigan

Plaintiff's expert, Dr. Madigan, had a Ph.D. in statistics,

taught statistics at Columbia University, had published more than

150 papers on biostatistics and pharmacovigilance, and served as

an investigator on an FDA pilot program to monitor the safety of

FDA-regulated medical products. He did not testify as to

causation, but rather explained the process of conducting

epidemiological studies, and examined the six epidemiological

studies on Accutane and Crohn's disease (Crockett, Bernstein,

Alhusayen, Etminan, Racine, and Sivaraman).

Madigan was critical of the design of the epidemiological

studies. He found that the studies were biased toward a finding

42 A-4698-14T1
of decreased risk as a result of: 1) power (Crockett, Alhusayen,

and Etminan); 2) prodrome (Crockett, Etminan, Racine, and

Alhusayen); and 3) unmeasured confounders (Bernstein (dose and

duration), Crockett (exposure and outcome), Alhusayen (allowing

reentry of patients after 12-month period), Etminan (confined to

contraceptive users), Racine (dose), and Sivaraman (dose and

duration)).

For example, he found that the power of the studies, or "the

power to detect a true effect of a particular size," was low. In

reaching that determination, he employed standard statistical

techniques and determined that the Accutane studies were not

sufficiently powered to detect even a 50% increased risk -- a

"meaningful" measure of risk. He calculated the nominal power of

the four studies (that did not find a statistically significant

risk to detect a 50% increased risk of Crohn's disease) as follows:

Bernstein (37.8%); Crockett (18.2%); Alhusayen (89.4%); and

Etminan (22.6%). In other words, the Bernstein study had only a

37.8% chance of finding a statistically significant increase when

there is a 50% increased risk, or a 62.2% chance of finding a

false negative.

He also opined that because Crohn's disease has a long

43 A-4698-14T1
variable prodrome (ranging from a few months to several years),

these studies, which focused on a short observation window to

measure exposure, failed to account for all of the patients who

developed Crohn's disease as a result of ingesting Accutane.

Madigan explained that failing to account for just a few patients

will "introduce bias into the study" toward a showing of no or

decreased risk and will decrease the power of the study.

Accounting for a 6.9-year prodrome (which he derived from the

Pimentel study), Madigan calculated that the power of some of the

studies further decreased to: Crockett (5.12%); Alhusayen

(36.2%); and Etminan (4.5%). He found that only Bernstein (with

its 2.6-year study) and Sivaraman (with its questionnaire format)

had addressed prodrome through their study designs.

Moreover, Madigan testified, that it was not scientifically

"appropriate to conduct a meta-analysis in this context, because

of concerns with the individual studies." He explained that the

purpose of a meta-analysis is to combine the results of

epidemiological studies "to make a combined estimate," however,

he noted that you cannot "make the bias" in a study "disappear"

by combining several biased studies. Madigan, like Kornbluth,

found that all of the studies in this case were "biased towards

44 A-4698-14T1
the null," that is, "systematically biased to lower the estimated

effect" and thus combining the studies would not yield an accurate

result.

Instead, Madigan conducted a statistical disproportionality

analysis of the spontaneous ADE reports for Accutane and Crohn's

disease contained in the FDA's ADE reporting system database. He

explained that spontaneous ADE reports "serve as a primary data

source with which we, as a society, study drug safety concerns,"

and is an important component of drug safety investigation even

though the data has limitations due to its reliance on voluntary

reporting. He said that a disproportionality analysis is

"standard" in analyzing ADE reports and is routinely used by the

FDA and the pharmaceutical industry in assessing emerging safety

concerns.

In his disproportionality analysis, which he conducted using

the same methods as he used in conducting an analysis for the

pharmaceutical industry, Madigan compared the observed rate of

reporting for Accutane and Crohn's disease with the rate at which

Crohn's disease was reported for other drugs in that database. He

found that from 1997 to the present, there was a "striking signal

of disproportionality" or a "strong association" between Accutane

45 A-4698-14T1
use and Crohn's disease. Further, when Madigan removed the ADE

reports generated through litigation by lawyer reporting, or

approximately 88% of the reports, the results still showed a

moderate increased risk of developing the disease.

He testified that a similar disproportionality analysis had

been conducted, by researchers affiliated with the World Health

Organization (WHO), of the WHO's drug safety database (Uppsala

Monitoring Centre) in which the researchers compared the observed

rate of ADE reports of Crohn's disease for Accutane to the observed

rate of reports of Crohn's disease for other drugs in the WHO

database.27 They found a statistically significant increased risk

(nineteen times greater) for developing Crohn's disease from

Accutane use.

2. Dr. Steven N. Goodman

The defense expert, Dr. Goodman, had an M.D. degree, as well

as a Ph.D. in epidemiology, was a professor and associate dean for

clinical research at Stanford University, and had published

numerous peer-reviewed scientific articles. He opined that the

epidemiological evidence supported a finding that there was a

27
The findings are cited in an article on case reports, Passier,
supra note 17, at 52.

46 A-4698-14T1
"strongly negative" association between Accutane and Crohn's

disease, and that there was no biologic evidence or scientifically

accepted causal mechanism that outweighed the results of the

epidemiological studies. He opined that Kornbluth's and Madigan's

methodology was not scientifically valid because they placed "very

little weight" on the epidemiological studies, which were the

highest tier of evidence in this case, and placed much more weight

on "lesser forms of evidence," including case reports, animal

studies, and causality assessments.

In forming his opinion, Goodman reviewed nine epidemiological

studies (Bernstein, Crockett, Etminan (case-control and meta-

analysis), Alhusayen, Racine, Rashtak, Sivaraman, and Fenerty).

He considered the overall results of these studies for IBD as well

as the results for Crohn's disease, because ulcerative colitis and

Crohn's disease share a variety of risk factors and because it is

difficult to distinguish between the diseases in the early stages.

He opined that although the Sivaraman study showed an unadjusted

statistically significant association between the drug and the

disease, the study was too small to have any significance. He

also opined that, viewed collectively, the epidemiological studies

were consistent with Accutane having "either [a] potentially

47 A-4698-14T1
protective effect" or "no effect."

To increase the power of the epidemiological studies, Goodman

conducted a meta-analysis of Accutane and IBD (both ulcerative

colitis and Crohn's disease), in which the larger more precise

studies were given more weight. He found an RR of developing IBD

of 0.87 (0.65-1.17), a negative association. He also conducted a

meta-analysis of Accutane and Crohn's disease, and found a similar

RR of 0.87 (0.59-1.28), another negative association.

Further, Goodman found that the epidemiological studies

properly accounted for the prodrome of Crohn's disease. In

determining the prodrome, Goodman reviewed several epidemiological

studies, including the Chouraki study, and concluded that the

average prodrome for Crohn's disease was nine months or less. He

criticized Madigan's reliance on what Goodman characterized as the

outlier non-population-based Pimentel study (6.9-years prodrome),

which Goodman said was too small to provide any valid information.

He concluded that the nine epidemiological studies, which applied

a prodrome from one to two years, were properly designed and

powered, and strongly supported a finding of no association between

Accutane and Crohn's disease.

48 A-4698-14T1
 IV

Before we address plaintiffs' appellate arguments, it is

helpful to review the legal principles applicable to the

admissibility of expert testimony in toxic tort and similar cases.

To establish liability, plaintiffs must prove through expert

testimony that ingestion of Accutane can cause Crohn's disease in

humans (general causation). In addition, each individual

plaintiff must prove specific causation, i.e., that Accutane was

the cause of his or her disease. See DeLuca v. Merrell Dow Pharm.,

Inc., 911 F.2d 941, 958 (3d Cir. 1990). See also Perry v. Novartis

Pharm. Corp., 564 F. Supp. 2d 452, 463 (E.D. Pa. 2008) ("Courts

in toxic tort cases often separate the causation inquiry into

general causation -- whether the substance is capable of causing

the observed harm in general -- and specific causation -- whether

the substance actually caused the harm a particular individual

suffered."). The Kemp hearing at issue here concerned general,

not specific, causation.

The admissibility of scientific evidence is governed by

N.J.R.E. 702, which provides that "[i]f scientific, technical, or

other specialized knowledge will assist the trier of fact to

understand the evidence or to determine a fact in issue, a witness

49 A-4698-14T1
qualified as an expert by knowledge, skill, experience, training,

or education may testify thereto in the form of an opinion or

otherwise." The Rule imposes three requirements:

(1) the intended testimony must concern a
subject matter that is beyond the ken of the
average juror; (2) the field testified to must
be at a state of the art such that an expert's
testimony could be sufficiently reliable; and
(3) the witness must have sufficient expertise
to offer the intended testimony.

[Hisenaj, supra, 194 N.J. at 15.]

The second requirement is at issue here, that is, whether

Kornbluth's causation testimony and Madigan's statistical analysis

testimony was sufficiently reliable in the field of scientific

research to be admitted. Ibid.

In most cases, the proponent of scientific evidence must

demonstrate that the opinions are "generally accepted, within the

relevant scientific community" (the Frye standard). State v.

Chun, 194 N.J. 54, 91, cert. denied, 555 U.S. 825, 129 S. Ct. 158,

172 L. Ed. 2d 41 (2008); State v. Harvey, 151 N.J. 117, 169-70

(1997) (citing Frye v. United States, 293 F. 1013, 1014 (D.C. Cir.

1923)), cert. denied, 528 U.S. 1085, 120 S. Ct. 811, 145 L. Ed.

50 A-4698-14T1
2d 683 (2000).28 See also Hisenaj, supra, 194 N.J. at 17. "That

acceptance entails the strict application of the scientific

method, which requires an extraordinarily high level of proof

based on prolonged, controlled, consistent, and validated

experience." Rubanick, supra, 125 N.J. at 436.

However, our Supreme Court has relaxed the "general

acceptance" standard in tort cases involving injuries caused by

toxic substances or medications, involving new or developing

theories of causation. Kemp, supra, 174 N.J. at 430-31; Landrigan,

supra, 127 N.J. at 414; Rubanick, supra, 125 N.J. at 449. Under

the relaxed standard, "a scientific theory of causation that has

not yet reached general acceptance may be found to be sufficiently

reliable if it is based on a sound, adequately-founded scientific

28
In 1993, the United States Supreme Court, construing the Federal
Rules of Evidence, held that Federal Rule of Evidence 702
superseded Frye and mandated that the federal courts apply a more
relaxed scientific reliability standard. Daubert v. Merrell Dow
Pharm., Inc., 509 U.S. 579, 113 S. Ct. 2786, 125 L. Ed. 2d 469
(1993). Daubert was a pharmacological tort case involving the
drug Benedictin. Id. at 582, 113 S. Ct. at 2791, 125 L. Ed. 2d
at 476. In criminal cases, New Jersey courts have not followed
Daubert, but continue to strictly apply the Frye test, i.e.,
whether the scientific community generally accepts the reliability
of the proffered evidence. See Harvey, supra, 151 N.J. at 168.
As noted, in civil cases involving toxic torts, our courts use the
relaxed standard set forth in Rubanick. See Kemp, supra, 174 N.J.
at 430-31.

51 A-4698-14T1
methodology involving data and information of the type reasonably

relied on by experts in the scientific field." Rubanick, supra,

125 N.J. at 449. Thus, the Court "changed the focus of the inquiry

from the scientific community's acceptance of the substance of the

opinion to its acceptance of the methodology and reasoning

underlying it." Clark v. Safety-Kleen Corp., 179 N.J. 318, 337

(2004). The Rubanick standard does not require the

"extraordinarily high level of proof[,]" ordinarily required

before a scientific theory will attain general acceptance in the

scientific community. Rubanick, supra, 125 N.J. at 436.

The rationale behind relaxation of the standard was "the

extraordinary and unique burdens facing plaintiffs who seek to

prove causation” in such cases. Id. at 433.29 The task of proving

causation in toxic tort cases "is invariably made more complex

because of the long latency period of illnesses caused by

carcinogens or other toxic chemicals." Ayers v. Jackson, 106 N.J.

557, 585 (1987). And, in some drug cases, causation may not have

29
Rubanick relied heavily on persuasive federal court opinions,
in rejecting the general acceptance test. Rubanick, supra, 125
N.J. at 445 (citing United States v. Downing, 753 F.2d 1224, 1237
(3d Cir. 1985), and DeLuca, supra, 911 F.2d at 941). See also
Ferebee v. Chevron Chem. Co., 736 F.2d 1529 (D.C. Cir.), cert.
denied, 469 U.S. 1062, 105 S. Ct. 545, 83 L. Ed. 2d 432 (1984)
(cited in Rubanick, supra, 125 N.J. at 440).

52 A-4698-14T1
been "confirmed by the scientific community but compelling

evidence nevertheless suggests that such a relationship exists."

Kemp, supra, 174 N.J. at 430.

V

Against that legal backdrop, we consider plaintiffs'

contention that the trial court erred in barring their experts'

testimony. Plaintiffs assert that: 1) their experts relied on

methodologies and data of the type relied on by comparable experts;

2) the judge substituted his judgment on the epidemiological

studies for that of the expert scientists; 3) the experts' reliance

on the studies involved a methodology generally followed by

comparable experts; 4) Kornbluth appropriately considered the

epidemiological studies in assessing the relationship between

Accutane and Crohn's disease; 5) the judge mischaracterized

Madigan's testimony; 6) the judge impermissibly weighed evidence

of ADE reports and animal studies; and 7) the judge abused his

discretion in assessing the credibility of plaintiffs' experts.

We agree that the judge erred in excluding the experts' testimony.

Under the relaxed standard, as applicable here, the trial

court assesses "the soundness of the proffered methodology and the

qualifications of the expert." Kemp, supra, 174 N.J. at 426

53 A-4698-14T1
(quoting Rubanick, supra, 125 N.J. at 454). The focus of the

trial court's inquiry must be "solely on principles and

methodology, not on the conclusions that they generate." Ibid.

(quoting Daubert, supra, 509 U.S. at 594-95, 113 S. Ct. at 2797,

125 L. Ed. 2d at 484). In evaluating the methodology, "courts

should consider whether others in the field use similar

methodologies." Rubanick, supra, 125 N.J. at 449.

In making that determination the court should not "directly

and independently determine as a matter of law that a controversial

and complex scientific methodology is sound." Id. at 451. "The

critical determination is whether comparable experts accept the

soundness of the methodology, including the reasonableness of

relying on this type of underlying data and information. Great

difficulties can arise when judges, assuming the role of scientist,

attempt to assess the validity of a complex scientific

methodology." Ibid. Nor is it appropriate for the trial judge

to second-guess an expert's interpretation of the underlying data.

Ibid. For example, in Rubanick the Court found that:

the trial court . . . "independently reviewed"
each of the thirteen studies on which Dr.
Balis relied, and decided that they "do not
say what plaintiff's expert concludes." In
engaging in such an analysis, the court
substituted its own assessment of the studies

54 A-4698-14T1
 for that of an acknowledged expert. . . .
"[t]he interpretation of the data . . . is the
function of the qualified expert . . . .
[C]ourts should be loath to determine whether
the particular expert has properly relied upon
data which experts in the field generally rely
on." Thus, the inquiry is not the reliability
of the expert's ultimate opinion nor is it
whether the expert thought his or her own
reliance on the underlying data was
reasonable, nor whether the court thinks that
the expert's reliance was reasonable[.] The
proper inquiry is whether comparable "experts
in the field [would] actually rely" on that
information.

[Id. at 451-52 (quoting Ryan v. KDI Sylvan
Pools, Inc., 121 N.J. 276, 289 (1990)
(additional citations omitted).]

The qualifications of the expert must also "be factored into the

determination of the soundness of the methodology used." Id. at

452.

"If epidemiological studies are to provide the basis for an

expert's opinion, they must have been 'soundly and reliably

generated' and be 'of a type reasonably relied on by comparable

experts in the particular field.'" Landrigan, supra, 127 N.J. at

419-20 (quoting Rubanick, supra, 125 N.J. at 447). When an expert

relies on epidemiological studies, the "court should review the

studies, as well as other information proffered by the parties,

to determine if they are of a kind on which such experts ordinarily

55 A-4698-14T1
rely." Id. at 417. Significantly, the court must "examine the

manner in which experts reason from the studies and other

information to a conclusion[,]" which "must derive from a sound

methodology that is supported by some consensus of experts in the

field." Id. at 420. See In re Zoloft Prods. Liab. Litig., 26 F.

Supp. 3d 449, 460 (E.D. Pa. 2014) (excluded expert whose "opinion

regarding class effects is not evidence based, and is directly

contrary to the findings of her own peer-reviewed, published

research"); In re Rezulin Prods. Liab. Litig., 369 F. Supp. 2d

398, 425 (S.D.N.Y. 2005) (court excluded expert testimony where

expert selectively chose his support from scientific literature

and failed to "acknowledge or account for" evidence that tended

to refute his theory).

A court's assessment of scientific expert evidence should

include an evaluation of the studies upon which the experts rely,

but the court must not substitute "its own assessment of the

studies for that of an acknowledged expert." Rubanick, supra, 125

N.J. at 451. "Although trial courts are expected to act as

gatekeepers to the proper admission of expert testimony," courts

are not expected "to investigate sua sponte the extent to which

the scientific community holds in esteem the particular analytical

56 A-4698-14T1
writings or research that a proponent of testimony advances as

foundational to an expert opinion." Hisenaj, supra, 194 N.J. at

16. "The court's function is to distinguish scientifically sound

reasoning from that of the self-validating expert, who uses

scientific terminology to present unsubstantiated personal

beliefs." Landrigan, supra, 127 N.J. at 414. The plaintiff bears

the burden of proof in establishing admissibility. Kemp, supra,

174 N.J. at 429.

Ordinarily, the admission or exclusion of expert testimony

is "committed to the sound discretion of the trial court[,]"

Townsend v. Pierre, 221 N.J. 36, 52 (2015), and we review the

decision for abuse of discretion. Hisenaj, supra, 194 N.J. at 12.

However, we owe somewhat less deference to a trial court's

determination in a case of this type. See State v. Torres, 183

N.J. 554, 567 (2005). "Although 'the trial court is in a better

position to shape the record and make credibility determinations,'

an 'appellate court need not be as deferential to the trial court's

ruling on the admissibility of expert scientific evidence as it

should be with the admissibility of other forms of evidence.'"

State v. J.R., 227 N.J. 393, 410 (2017) (quoting Torres, supra,

183 N.J. at 567). In determining whether the trial court

57 A-4698-14T1
misapplied discretion, we consider whether the court's analysis

of the evidence was faithful to the principles set forth in

Rubanick, or whether the court misapplied the standards. See

State v. Darby, 174 N.J. 509, 518 (2002) (no deference is to be

accorded to the trial court's decision to admit other-crime

evidence, nor is that decision entitled to be reviewed under an

abuse of discretion standard, where the trial judge failed to

apply applicable law); Konop v. Rosen, 425 N.J. Super. 391, 401

(App. Div. 2012) (appellate review is de novo when the trial court

fails to apply the proper test in analyzing the admissibility of

proffered evidence). See also Pressler & Verniero, Current N.J.

Court Rules, comment 4.7 on R. 2:10-2 (2017) ("When the trial

court fails to apply the proper test in analyzing the admissibility

of proffered evidence, the de novo standard of review . . .

applies").

Here, the court found that although both of plaintiffs'

experts were "eminently qualified, their reasoning and methodology

is slanted away from objective science and in the direction of

advocacy." The court found that Kornbluth's methodology was not

supported by the scientific community because he interpreted the

Sivaraman study differently than its authors did. Similarly, the

58 A-4698-14T1
court found that Madigan placed undue weight on the Sivaraman

study, and "ignored" the other studies. The judge also criticized

Madigan for failing to perform a meta-analysis of all of the

studies.

The court further reasoned that plaintiffs' experts had failed

to follow valid scientific methodology in relying on the Sivaraman

and Pimentel studies to the exclusion of the larger population-

based epidemiological studies, concluding that the "scientific

literature does not support reliance upon such insignificant

studies to arrive at conclusions." The court concluded that

Kornbluth's "contrived reasoning is not supported by the

scientific community as a reliable basis for making causal

determinations," and Madigan's opinions were not methodology

based, but rather conclusion-driven.

Additionally, the court found that Kornbluth's testimony was

"replete with what can be described as convenient assumptions.

When he needs to bridge an analytical gap in his methodology he

assumes facts, events and conclusions as he wants them to be in

support of his hypothesis." For example,

in response to counsel's questioning regarding
the results of various studies, Dr. Kornbluth
assumed: (a) that all the patients in the two
studies upon which he relied filled out their

59 A-4698-14T1
 questionnaires correctly; (b) despite the fact
that the authors of the Sivaraman study got
it wrong as to their adjustment for
antibiotics, he assumed they got everything
else correct; (c) he assumed that in the
Rashtak Study, the patients with Accutane
exposure were followed for less time than the
control group; and (d) he assumed the size of
the doses of Accutane given to the subjects
in various studies.

With regard to biological plausibility, the court found

Kornbluth's discussion of his hypothesis for
the biological mechanism of the development
of CD [Crohn's disease] as caused by
Isotretinoin falls far short of being
"compelling." His basis for the discussion
are the medications Natalizumab and
Vedolizumab. He attempts to extrapolate
causation of CD by Isotretinoin by discussing
treatment of CD by these other medications.
Dr. Oliva-Hempker [sic] explained the inherent
weakness of trying to rely upon the data on
Natalizumab and Vedolizumab as being probative
of causation. In essence, treating a
"pathway" that develops once a disease occurs,
does not mean that . . . a particular treatment
mechanism informs as to the original cause of
the disease. She also pointed out that this
hypothesis is contrary to a significant body
of scientific literature showing that Retinoic
acid is actually anti-inflammatory . . .

The court described Madigan as "an expert on a mission," and

criticized Kornbluth's approach as being less convincing than

Oliva-Hemker's analysis as to causation. He was also critical of

plaintiffs' experts' reliance on lines of evidence other than

60 A-4698-14T1
epidemiological studies:

[C]oursing through Plaintiffs' presentation
is a refrain that is a ruse. Repeatedly,
counsel for the Plaintiffs and their witnesses
spoke of "lines of evidence," emphasizing that
their experts examined the same "lines of
evidence" as did the experts for the Defense.
Counsels' sophistry is belied by the fact that
the examination of the "lines of evidence" by
Plaintiffs' experts was highly selective,
looking no further than they wanted to --
cherry picking the evidence -- in order to
find support for their conclusion-driven
testimony in support of a hypothesis made of
disparate pieces, all at the bottom of the
medical evidence hierarchy. This crafty
stratagem cannot bridge the analytical gaps
inherent in Plaintiffs' hypothesis.

Plaintiffs contend that, whether or not the trial judge found

their experts opinions persuasive in substance, the experts relied

on methodologies and data of the type reasonably relied upon by

comparable experts. We agree.

A. Data and Information

Whether or not it persuades a jury, it is clear to us that

in forming their conclusions Kornbluth and Madigan relied on the

types of data and information reasonably relied on by comparable

experts in the scientific field, and by the experts in previous

Accutane cases in this docket. That evidence includes

epidemiological studies, scientific articles, case studies,

61 A-4698-14T1
clinical studies, animal studies, and causality assessments.

It is well-established that epidemiological studies,

published in peer-reviewed scientific journals, as were considered

in this case, are the type of data reasonably relied on by the

scientific community to determine whether exposure to a drug is

associated with a disease. Landrigan, supra, 127 N.J. at 419-20.

Properly conducted epidemiological studies are a significant

factor in establishing causation in toxic tort cases. See Reference

Manual, supra, at 551 n.2. "[E]pidemiology is a well-established

branch of science and medicine, and epidemiological evidence has

been accepted in numerous cases." DeLuca, supra, 911 F.2d at 954.

See Magistrini v. One Hour Martinizing Dry Cleaning, 180 F. Supp.

2d 584, 591 (D.N.J. 2002) (epidemiological studies), aff'd o.b.,

68 Fed. App'x 356 (3d Cir. 2003). Notably, although

epidemiological evidence is not required to prove causation, if

it exists, an expert cannot ignore it. Perry, supra, 564 F. Supp.

2d at 465. However, the existence of inconclusive epidemiological

studies does not preclude an expert from relying on alternative

data, such as animal studies. Id. at 466.

Moreover, although the Sivaraman epidemiological study was

published as an abstract and not a full article, and was presented

62 A-4698-14T1
at a conference, our courts recognize that "[s]upport for an

expert's methodology may be found in professional journals, texts,

conferences, symposia, or judicial opinions accepting the

methodology." Kemp, supra, 174 N.J. at 427. In accord with

accepted scientific methodology, Kornbluth also considered other

forms of evidence in determining causation, including animal

studies, case reports, challenge/dechallenge/rechallenge reports,

causality assessments, class effects, and published scientific

literature. Although case reports and causality assessments

should be interpreted with caution, there was nothing so inherently

unreliable about the materials Kornbluth cited as to preclude

their consideration as part of a scientific expert's methodology

under N.J.R.E. 702. Further, the experts did not elevate this

evidence over the epidemiological studies, but rather considered

this evidence in forming their opinions.

B. Methodology and Reasoning

We also conclude that the methodology used by Kornbluth and

Madigan to reach their conclusions was consistent with sound

scientific principles and methodologies accepted in the medical

and scientific community. Rubanick, supra, 125 N.J. at 449; Kemp,

supra, 174 N.J. at 431. In making that determination the court

63 A-4698-14T1
must "examine the manner in which experts reason from the studies

and other information to a conclusion." Landrigan, supra, 127

N.J. at 420. The experts' conclusions "must derive from a sound

methodology that is supported by some consensus of experts in the

field." Ibid. The experts must identify the factual bases for

their conclusions, explain their methodology, and demonstrate that

both the factual bases and the methodology are reliable. Kemp,

supra, 174 N.J. at 427; Rubanick, supra, 125 N.J. at 449-50.

The primary focus in this appeal is upon Kornbluth's and

Madigan's analysis of the epidemiological studies. Those studies

indicated that the relationship between Accutane and Crohn's

disease is more tenuous than the relationship between the drug and

ulcerative colitis. For example, Crockett (a large study) found

a statistically significant association between Accutane and

ulcerative colitis, and four studies (Bernstein, Etminan, Racine

and Alhusayen) found a positive association between the drug and

the disease.

In contrast, only one small study (Sivaraman) found a

statistically significant positive association (before adjustment

for antibiotic use) between Accutane and Crohn's disease, two

found a positive association (Bernstein and Alhusayen), three

64 A-4698-14T1
found a negative association (Crockett, Etminan, and Racine), and

one (Racine) found a statistically significant association for a

protective effect. The degree to which this contrary opinion

dominates the epidemiological studies is relevant to the

reliability inquiry. DeLuca, supra, 911 F.2d at 955. However,

demonstrable flaws in the studies may undercut their significance.

Ibid.

In other words, does the relevant scientific community accept

the process by which Kornbluth and Madigan reasoned to a conclusion

that the epidemiological studies (despite the lack of a

statistically significant association) and the other relevant

evidence supported a finding of a causal relationship between

Accutane and Crohn's disease? In some cases, a court may conclude

that there is simply too great an analytical gap between the data

and the expert's opinion. See Gen. Elec. Co. v. Joiner, 522 U.S.

136, 146, 118 S. Ct. 512, 519, 139 L. Ed. 2d 508, 519 (1997).

However, in this case, we conclude that the data was sufficient

to permit the experts to testify, and any weaknesses in their

opinions can be explored through cross-examination.

It is well-established that "[t]he usefulness of an

epidemiological study depends on the quality of the underlying

65 A-4698-14T1
data, the reliability of the methodology, and the validity of the

interpretations." Landrigan, supra, 127 N.J. at 420 (quoting

Michael Dore, A Commentary on the Use of Epidemiological Evidence

in Demonstrating Cause-in-Fact, 7 Harv. Envtl. L. Rev. 429, 432

(1983)). An expert should, under sound scientific methodology,

evaluate the study in assessing its validity. See ibid. We infer

from the Manual that an expert should consider a study's possible

flaws and weaknesses before deciding whether to rely on it.

Reference Manual, supra, at 553. Further, an expert can rely on

the data generated from a study even if he or she disagrees with

the author's conclusion, and need not subject his or her own

analysis to peer review. DeLuca, supra, 911 F.2d at 954.

Here, in contrast to the court's finding, Kornbluth and

Madigan, in accordance with established scientific methodology,

evaluated all of the epidemiological and prodrome studies, not

just Sivaraman and Pimentel. They testified at length as to the

design flaws and limitations of the epidemiological studies,

including the failure to account for the prodrome, insufficient

power, and design flaws, which are all recognized in the scientific

community as capable of producing an erroneous association in an

epidemiological study. See Reference Manual, supra, at 583. For

66 A-4698-14T1
example, a poorly conceived or conducted study that disproves the

null hypothesis at a high level of significance may be far less

reliable than a well-conceived and conducted study that is

significant at a lower level. DeLuca, supra, 911 F.2d at 955.30

Moreover, the fact that defendants' experts interpret the

epidemiological studies differently does not, standing alone,

indicate that Kornbluth and Madigan failed to rely upon a sound

methodology. "Indeed, 'in most cases, objections to the

inadequacies of a study are more appropriately considered an

objection going to the weight of the evidence rather than its

admissibility.'" Rosenfeld v. Oceania Cruises, Inc., 654 F.3d

1190, 1193 (11th Cir. 2011) (quoting Hemmings v. Tidyman's Inc.,

285 F.3d 1174, 1188 (9th Cir. 2002), cert. denied, 537 U.S. 1110,

123 S. Ct. 854, 154 L. Ed. 2d 781 (2003)).

That said, certainly, as the trial judge correctly observed,

30
DeLuca was "a diversity action brought under New Jersey law"
against the manufacturer of Benedictin. Id. at 942-43.
Anticipating Daubert, the Third Circuit Court of Appeals rejected
the Frye standard, and reversed a trial court decision barring
testimony from an expert whose approach was remarkably similar to
that of plaintiffs' experts in this case. Id. at 955. Although
DeLuca is not binding on us, we find it persuasive. As previously
noted, DeLuca was also one of the seminal federal cases cited with
approval in Rubanick as being "compatible with our own rules of
evidence." Rubanick, supra, 125 N.J. at 445, 447.

67 A-4698-14T1
larger studies enable researchers to form a more accurate

conclusion and reduce the chance of random error in their results.

Reference Manual, supra, at 576. However, Kornbluth could, in

applying accepted scientific methodology, properly consider one

small well-designed study over larger seriously flawed studies as

a basis for drawing an inference about the studied subject. 31

Further, Kornbluth did not ignore the results of the larger studies

in favor of the smaller Sivaraman study, but considered the

relative risk and the bounds of the 95% confidence interval in

reviewing the conclusions. He also found that although the results

of the studies were inconclusive, they were informative on his

theory of causation. For example, he noted that four of the studies

(Bernstein, Etminan (unadjusted), Alhusayen, and Sivaraman) found

a positive association between Accutane use and Crohn's disease,

and that one of the studies (Bernstein) showed an increased

association when accounting for a two-year but not a one-year

prodrome.

31
Scientific acceptance of small studies is not unknown. A
treatise on pharmacoepidemiology, which is included in the
parties' appendices, notes that "using case control studies, one
can study rare diseases with markedly smaller sample sizes. . . .
For example, the classic study of diethylstilbestrol and clear
cell adenocarcinoma required only 8 cases and 40 controls." (Brian
L. Strom, Pharmacoepidemiology 23 (4th ed. 2006)).

68 A-4698-14T1
 Similarly, Kornbluth testified at length as to the results

of both the Pimentel (6.9-year prodrome) and Barratt (four-year

prodrome) prodrome studies and opined that the results of those

studies were in agreement with his decades of experience treating

thousands of Crohn's disease patients. Kornbluth and Madigan

rejected, but did not ignore, the findings of the Chouraki study

based on the selection of patients and the use of patient charts.

Further, Madigan testified that the Pimentel study, which utilized

questionnaires, contained the most useful and relevant data,

including raw data from which he could compute age-specific

estimated prodromes.32

Kornbluth's methodology in analyzing the epidemiological

studies was also bolstered by Madigan, who presented detailed

testimony as to the insufficient power of the epidemiological

studies. Further, Madigan testified that the decision whether to

conduct a meta-analysis is a scientific judgment, and explained

32
Our reading of the Pimentel article supports Kornbluth's
description of the authors' very meticulous approach to gathering
and verifying information about the subjects. We cannot agree
with the trial judge's view that Kornbluth was "cherry picking"
in relying on Pimentel. Kornbluth cogently explained why he
believed that the authors' methodology was reliable, and
consistent with his own medical practice in diagnosing Crohn's
disease patients.

69 A-4698-14T1
that such an analysis would not yield reliable results in this

case. Instead, he conducted and reviewed a disproportionality

analysis, which while not without limitations, is a validated

method in drug safety research and surveillance.

Both Kornbluth and Madigan explained in considerable detail

why most of the studies were biased toward "the null" or no effect,

and were otherwise inadequate to reliably demonstrate whether or

not there was a statistically significant connection between

Accutane and Crohn's disease. They also explained why the

statistically significant initial results of the Sivaraman study

were more reliable than the adjusted results.

The reliability of Kornbluth's opinion on causation was also

strengthened by his consideration of other evidence (including

case reports, animal studies, and causality assessments), and most

notably, because he presented a biologically plausible mechanism

for how Accutane causes Crohn's disease. See Reference Manual,

supra, at 604 ("When biological plausibility exists, it lends

credence to an inference of causality"). Once Kornbluth found

that there was an association between Accutane and Crohn's disease

based on his reading of the epidemiological studies, in addition

to the scientific articles, MedWatch reports, and causality

70 A-4698-14T1
assessments, he then considered whether that association was

causal, utilizing the Bradford Hill criteria. Under that analysis,

he considered, among other criteria, whether there was a

biologically plausible mechanism by which Accutane could cause

Crohn's disease—an important factor for determining a causal

relationship.

Kornbluth, like plaintiffs' causation expert in previous

Accutane trials, presented a biologically plausible mechanism

supported by scientifically authoritative sources. He opined,

based on his experience as a board-certified gastroenterologist

and in conducting clinical trials on several drugs intended for

use in the management of IBD, that retinoic acid, a metabolite of

Accutane, "is a damaging pathway for patients with Crohn's

disease." He found support for that opinion in the fact that two

new drugs (Vedolizumab and Natalizumab) were effective in treating

Crohn's disease and in a Canadian case-control epidemiological

study that reported an increased risk of Crohn's disease from

retinoic acid. Olivia-Hemker disagreed with that opinion and

cited to other studies that supported a finding that retinoic acid

had an anti-inflammatory or protective effect on the intestines,

but that dispute goes to the weight, not the admissibility of the

71 A-4698-14T1
testimony. See Harvey, supra, 151 N.J. at 178.

In conclusion, Kornbluth and Madigan, who are indisputably

extremely well-qualified experts, considered all of the relevant

data and information, applied appropriate methodology in analyzing

the epidemiological studies, and expressed valid reasons for

rejecting the conclusions of some of the epidemiological studies

and in accepting other studies as supportive of their opinion.

Although the relationship between the epidemiological scientific

evidence and the experts' opinions is more tenuous than the

evidence as to ulcerative colitis, the studies do not render the

experts' testimony inadmissible. The manner in which plaintiffs'

experts reasoned from the results of the epidemiological studies

and other data is sufficiently sound to be reliable. Landrigan,

supra, 127 N.J. at 420. Further, the experts did not ignore the

findings of the larger epidemiological studies but explained the

scientific bases for their criticism of the studies. Defendants'

criticisms of the experts' choices as to the evidence on which

they relied, can be addressed during cross-examination at trial.

Hisenaj, supra, 194 N.J. at 24.

We also cannot agree with the trial court's view that because

Kornbluth had not submitted his "current hypothesis" to a peer-

72 A-4698-14T1
reviewed publication, he must have generated his opinion solely

as a result of litigation and was a mere "hired gun." An expert

is not required to submit her own analysis to peer review in order

for a court to consider it. See DeLuca, supra, 911 F.2d at 954.

We also do not subscribe to the trial court's characterization of

Madigan as a hired gun whose testimony "was needed to clear the

way for Dr. Kornbluth's hypothesis and that was the role he played,

without regard to whether or not his efforts led the discussion

any closer to scientific truth." Madigan carefully explained his

methodology and his testimony should not have been discounted

because defendants heavily contested his conclusions.

Given that our evidence rules embody a strong preference for

admissibility, we conclude that the court mistakenly applied its

discretion in excluding the expert scientific testimony. See

N.J.R.E. 702; N.J.R.E. 401; State v. Jenewicz, 193 N.J. 440, 454

(2008) (noting "Rule 702's tilt in favor of the admissibility of

expert testimony"); State v. Granskie, 433 N.J. Super. 44, 47-48

(App. Div. 2013); Kuehn v. Pub Zone, 364 N.J. Super. 301, 320

(App. Div. 2003) (expert's testimony was relevant to issues under

consideration), certif. denied, 178 N.J. 454 (2004).

73 A-4698-14T1
 VI

In concluding, we emphasize the following observations. The

trial court's decision, and our decision of this appeal, must be

viewed in the context of this particular MCL litigation. It

presents a close question concerning the survival of plaintiffs'

cause of action in the face of new scientific information about

Accutane and IBD.

In deciding this appeal, we bear in mind that science is

constantly evolving, and that under our State's legal precedents,

legal decision making in toxic tort and similar cases may vary

from scientific decision making. The opportunity of thousands of

plaintiffs, claiming injury from Accutane, to have their day in

court may rest on that difference and must be decided now.

In general . . . clinical, regulatory,
commercial, and legal decisions need to be
made based on the best evidence available at
the time of the decision. To quote Sir Austin
Bradford Hill:

All scientific work is incomplete -
whether it be observational or
experimental. All scientific work
is liable to be upset or modified
by advancing knowledge. That does
not confer upon us a freedom to
ignore the knowledge we already
have, or to postpone the action that
it appears to demand at a given
time.

74 A-4698-14T1
 Who knows, asked Robert
Browning, but the world may end
tonight? True, but on available
evidence, most of us make ready to
commute on the 8:30 next day.

[Pharmacoepidemiology, supra, at 26-27
(quoting Hill, supra, at 295-300).]

The parties in this case differ sharply on the question of

what constitutes "the best evidence available at the time of the

decision." Id. at 26. In particular, the case presents the

question whether, in the face of several epidemiological studies

that do not demonstrate a statistically significant relationship

between taking Accutane and developing Crohn's disease, plaintiffs

can continue to rely on other types of evidence - which in this

same MCL docket they were previously permitted to use - to prove

general causation. We also consider whether they can rely in part

on information from some of the epidemiological studies that show

a positive correlation, albeit not reaching the level of

statistical significance.

We conclude that in this case, the epidemiology studies are

not a conclusive bar to plaintiffs' case, and that their experts

should be allowed to testify. Although epidemiological studies

are considered as high on the tier of evidence bearing on the

75 A-4698-14T1
question of causation, like any other form of scientific evidence,

any particular study is only valuable if it is conducted in a

scientifically reliable manner. Any party - plaintiff or defendant

- has the right to challenge the methodology and, hence the

results, of an epidemiological study.

In fact, the Reference Manual on Scientific Evidence cautions

that

[A]ll [epidemiological] studies have "flaws"
in the sense of limitations that add
uncertainty about the proper interpretation of
the results. Some flaws are inevitable given
the limits of technology, resources, the
ability and willingness of persons to
participate in a study, and ethical
constraints. In evaluating epidemiologic
evidence, the key questions, then, are the
extent to which a study's limitations
compromise its findings and permit inferences
about causation.

[Reference Manual, supra, at 553.]

After explaining some of the most common biases that may affect

observational epidemiological studies, the Manual states that

["t]here are dozens of other potential biases that can occur in

observational studies, which is an important reason why clinical

studies (when ethical) are often preferable." Id. at 590. Thus

it can be expected that, as in this case, the methodology and

limitations of epidemiological studies - particularly

76 A-4698-14T1
observational studies - will be fertile ground for disagreement

among experts.

Moreover, the Manual supports plaintiffs' continued reliance

on other types of evidence to prove their case, particularly given

their well-explained opinions that most of the epidemiological

studies are fundamentally flawed. Contrary to the trial judge's

view, the Manual does not discount the value of live animal

studies. While noting some potential weaknesses of the studies,

the Manual states that toxicological studies, of which animal

studies are one type, are "often . . . the only or best available

evidence of toxicity." Id. at 564. The Manual also cautions that

"[w]here both animal toxicologic and epidemiologic studies are

available, no universal rules exist for how to interpret or

reconcile them. Careful assessment of the methodological validity

and power of the epidemiologic evidence must be undertaken, and

the quality of the toxicologic studies and the questions of

interspecies extrapolation and dose-response relationship must be

considered." Id. at 564-65.

The judge, and defendants, relied heavily on a section of the

Manual captioned "Hierarchy of medical evidence" (the medical

hierarchy section). Id. at 723. However, that section does not

77 A-4698-14T1
appear in the Reference Guide on Epidemiology. Rather, the section

appears in the Reference Guide on Medical Testimony, as part of a

chapter on medical decision-making. That chapter describes how

doctors make decisions about diagnosing and treating patients and

discusses the difficulties they face in making those decisions.

Id. at 704. We do not construe the medical hierarchy section of

the Manual as prescribing a rigid hierarchy for the acceptance or

rejection of evidence in a legal setting. See Matrixx Initiatives,

Inc. v. Siracusano, 563 U.S. 27, 40-42, 131 S. Ct. 1309, 1318-20,

179 L. Ed. 2d 398, 410-12 (2011); DeLuca, supra, 911 F.2d at 957.

In fact, the preface to the Manual cautions judges as to "the

proper use of the reference guides. They are not intended to

instruct judges concerning what evidence should be admissible or

to establish minimum standards for acceptable scientific

testimony." Reference Manual, supra, at xv. As significantly,

nothing in the Manual suggests that once epidemiological studies

have been done, they are beyond scientific criticism, and no

countervailing evidence should be considered.

We cannot agree with the trial judge's observation that

plaintiff's experts "ignored" the epidemiological studies in favor

of less reliable evidence. The experts did not ignore the studies.

78 A-4698-14T1
Rather, in extensive and detailed testimony, they opined that most

of the studies were unreliable, and they explained in considerable

detail the reasons for those opinions.

In their testimony, both of plaintiffs' experts raised

fundamental objections to the way the studies were conducted -

particularly the length of time for which the studies followed the

subjects. Based both on a prodrome study he found reliable and

on the decades he has spent treating thousands of Crohn's patients,

Dr. Kornbluth testified that the prodrome for Crohn's disease is

much longer than the one-year time frame covered by most of the

studies. Defendant's biostatistical expert, Dr. Goodman, admitted

that if Dr. Kornbluth was correct about the prodrome, then all of

the epidemiological studies on which the defense relied would be

flawed.

Kornbluth also explained in detail other weaknesses of

several of the studies. For example, the Alhusayan study treated

subjects exposed to Accutane as being non-exposed after a one-year

period following treatment. Thus, if those subjects developed IBD

after a year and a day, the study reported them as though they had

79 A-4698-14T1
never taken Accutane.33 Kornbluth also explained that studies from

other countries would not necessarily reflect the experience of

United States subjects, because the standard dose of Accutane

given to patients in those other countries is half that given to

patients in the United States.

Kornbluth's view on the prodrome issue was actually bolstered

by some of the defense testimony. During her cross-examination,

Dr. Oliva-Hemker was confronted with her own book, which answered

the question "How long have I had my IBD" by advising that: "Some

people have years of symptoms before the diagnosis [of IBD] is

made, while in others, these symptoms appear suddenly. Both groups

may have had intestinal inflammation for days, months, or years,

even though they didn't experience any symptoms at all for most

of that time." She then clarified that "we traditionally apply

that more to Crohn's patients rather than ulcerative colitis

patients in terms of [it taking] years" to diagnose the disease.

She was also confronted with a book written by a recognized expert

who referred to "the four-year average delay of diagnosis of

33
Although the study authors downplayed the results, the Alhusayan
study also discovered what the authors characterized as a "weak"
but statistically significant connection between Accutane and the
development of IBD in teenagers, ages twelve to nineteen.

80 A-4698-14T1
Crohn's disease."

Oliva-Hemker also confirmed that IBD affects about one

percent of the population in the United States; Crohn's disease

is a small subset of IBD, so the proportion of persons with Crohn's

is much smaller than one percent. Those admissions support the

view of plaintiff's experts that a study's failure to detect even

a small number of "exposed cases," i.e., persons with Crohn's

disease who had taken Accutane, could produce skewed results.

Further, when it suits their litigation strategy, defendants

do not treat epidemiological studies as the last word in scientific

proof. During the cross-examination of Oliva-Hemker, she admitted

that in earlier Accutane litigation, when four epidemiological

studies - concerning the lack of connection between antibiotics

and ulcerative colitis - did not support her opinion that the

plaintiff's UC was caused by antibiotics rather than Accutane, she

relied on evidence of biological plausibility instead. It took

almost four pages of repetitive questioning before Oliva-Hemker

finally admitted that the methodology she used was valid. Yet,

in this litigation, she criticized Kornbluth for relying on

evidence of biological plausibility and placing less weight on the

epidemiological studies.

81 A-4698-14T1
 Additionally, during the cross-examination of defendant's

epidemiology expert, Dr. Goodman, he admitted that some of the

epidemiological studies in this case had biases and weaknesses.

He admitted, for example, that none of the studies controlled for

family history, even though that is recognized as a strong factor

in a person's potentially developing Crohn's disease. He contended

that scientific judgment was required to evaluate how important

those biases or weaknesses were. Goodman was also confronted with

some of his own writings, in which he stated that, "If bias is

present in each or some of the individual studies, meta-analysis

will simply compound the errors and produce a wrong result that

may be interpreted as having more credibility." That same point

was made by plaintiff's epidemiology expert, Dr. Madigan, and it

finds support in the Manual.

We appreciate that the trial judge had the opportunity, which

we did not, to see the witnesses testify firsthand. However, his

extreme negative reaction to plaintiffs' witnesses is not

supported by the trial record. See J.R., supra, 227 N.J. at 410;

Torres, supra, 183 N.J. at 567. In reviewing Madigan's testimony,

we cannot agree with the judge that Madigan was a biased expert

"on a mission." His testimony was coherent and consistent, and the

82 A-4698-14T1
attorney who cross-examined him made little headway in

discrediting his direct testimony.

The judge's disapproval of plaintiffs' experts' reliance on

"lines of evidence" seems misplaced, because the defense used

the same terminology and considered the same evidence. Dr. Oliva-

Hemker agreed that she and Dr. Kornbluth looked at the same lines

of evidence, although they reached different conclusions from the

evidence. Further, the defense experts generally agreed with the

proposition that, in looking at the issue of causation, it is

appropriate to consider all of the pertinent evidence and not just

the epidemiological studies. The judge also criticized

plaintiffs' experts for their skepticism about the use of meta-

analysis. However, the Manual cautions that "when meta-analysis

is applied to observational studies - either case-control or cohort

- it becomes more controversial" due to the "methodological

differences among studies." Reference Manual, supra, at 607.

In summary, the purpose of a Kemp hearing is to weed out

"junk science," not to shield jurors from hearing expert testimony

that is scientifically-based but unpersuasive to the trial judge.

Landrigan, supra, 127 N.J. at 417; Kemp, supra, 174 N.J. at 427.

"[R]egardless of a trial judge's view of the weight a party's

83 A-4698-14T1
evidence deserves, the judge should trust the jury to evaluate

witness credibility and decide what weight to give each side's

evidence." State v. Stubblefield, __ N.J. Super. __, __ n.6 (App.

Div. 2017) (slip op. at 21 n.16). It is the jury's core function

to weigh the credibility of expert witnesses, and the trial court

should not use a Kemp hearing as a vehicle to dismiss a case the

court perceives as weak." Vigorous cross-examination,

presentation of contrary evidence, and careful instruction on the

burden of proof are the traditional and appropriate means of

attacking shaky but admissible evidence." Daubert, supra, 509

U.S. at 596, 113 S. Ct. at 2798, 125 L. Ed. 2d at 484.

We conclude that the trial court misapplied its discretion

in barring Dr. Kornbluth and Dr. Madigan from testifying.

Accordingly, the orders entered in A-4698-14 and A-0910-16,

barring their testimony and dismissing the complaints on summary

judgment, are reversed and these cases are remanded to the trial

court for further proceedings. We do not retain jurisdiction.

Reversed and remanded.

84 A-4698-14T1