Allen v. Alabama State Board of Education

976 F.Supp. 1410 (1997)

Margaret T. ALLEN, et al., Plaintiffs, Board of Trustees for Alabama State University, et al., Plaintiff-Intervenors,
v.
THE ALABAMA STATE BOARD OF EDUCATION, et al., Defendants.

Civ. A. No. 81-697-N.

United States District Court, M.D. Alabama, Northern Division.

September 8, 1997.

James U. Blacksher, Birmingham, AL, Terry G. Davis, Terry G. Davis, P.C., Solomon S. Seay, Jr., Donald V. Watkins, Donald V. Watkins, P.C., Mark Englehart, Kenneth Lamar Thomas, Thomas, Means & Gillis, P.C., Larry T. Menefee, A. Wesley Pitters, Montgomery, AL, Gregory Brent Stein, Mobile, AL, for Plaintiffs.

Denise Boone Azar, Ashley H. Hamlett, Dept. of Educ. Office of Gen. Counsel, Montgomery, AL, Susan McKinney, J. Callen Sparrow, Office of the Atty. Gen., Montgomery, AL, Jim R. Ippolito, Jr., Alabama Dept. of Transp., Montgomery, AL, Richard N. Meadows, State Personnel Dept., Montgomery, AL, for Defendants.

Michael A. Rebell, Rebell & Katzive, New York City, for National Evaluation System, Inc., amicus.

MEMORANDUM OPINION

MYRON H. THOMPSON, Chief Judge.

This litigation arises from a lawsuit filed in 1981 alleging that the State of Alabama's teacher certification test impermissibly discriminated against African-American persons seeking teacher certification. On May 14, 1987, the court approved and entered a consent decree providing for the development of a new test. On August 28, 1995, defendants filed a motion to modify the consent decree, and on September 25, 1995, defendants filed an alternative motion to vacate the consent decree. A trial was held on these motions on February 23, 1996. For the reasons that follow, the court holds that defendants have not made a good-faith effort to develop a test that both meets the requirements of the consent decree and is psycho-metrically sound or even to find out whether such a test can be developed, and that, unless and until they do, their motions are essentially premature. The motions to vacate or modify the consent decree will, therefore, be denied, albeit without prejudice to the right of the defendants to return to court.

I. BACKGROUND.

A. Overview of teacher testing litigation in Alabama

The procedural history of this case is complicated, and therefore it is necessary to provide an overview of teacher testing litigation in Alabama. In 1979, amidst a groundswell in favor of teacher competency testing, the Alabama State Board of Education placed development of a uniform certification test at the head of its agenda.^[1] In January 1980, it awarded a contract on a non-competitive basis to a private test developer who had previously developed a test for the State of Georgia.^[2] The developer created a total of 45 examinations in 1981 and 1982, which included a core examination for all teachers as well as examinations of specialized subject area.^[3] On December 15, 1981, plaintiffs, three black teachers, filed this lawsuit challenging the Alabama State Board of Education's requirement that applicants for State teacher certification pass a standardized test under the Alabama Initial Teacher Certification Testing Program ("AITCTP").^[4] They named as defendants the State Board, its individual members, and the State Superintendent of Education. On October 14, 1983, the court certified a class, consisting of all black persons who have been or will be denied any level teacher certification because they failed to pass the test administered under the program.^[5] On June 13, 1984, the court allowed a predominantly black university and an additional black teacher to intervene as plaintiffs.^[6]

In April 1985, the plaintiffs and the defendants settled this lawsuit by verbally agreeing to a proposed consent decree.^[7] The State Board subsequently renounced the settlement in response to public criticism, but this court ruled that the settlement was binding.^[8] On October 25, 1985, the court approved and entered the consent decree. On November 4, 1985, defendants filed a motion for the court to reconsider its finding that the parties had entered into a binding settlement. This court subsequently granted that motion, holding that although the parties had entered a settlement, a signed writing did not exist, and therefore principles of federalism prohibited it from enforcing the settlement.^[9] The Eleventh Circuit Court of Appeals reversed, holding that the agreement to settle was enforceable.^[10] On May 14, 1987, this court entered an order enforcing the consent decree. As part of the decree, the State Board was enjoined from using the old test and was required to issue permanent teaching certificates to a court-defined class of African-American teachers who had failed the certification test.

Although the defendants denied in the consent decree that the old test was psychometrically invalid, this court subsequently found in a separate case that it lacked validity. In Richardson v. Lamar County of Board of Education,^[11] an African-American teacher who had failed the AITCTP test alleged that her local school board's failure to renew her contract violated Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C.A. §§ 1981a, 2000e through 2000e-17, because the AITCTP test had a disparate impact on black teachers. After an extensive review of the development of the AITCTP test, the court found that none of the examinations that comprised the certification test possessed content validity because of five major errors by the test developer.^[12] The court also found that the test developer had made six major errors in establishing cut scores to decide who passed and who failed the test.^[13] Therefore, the court ruled that the teacher was entitled to relief because the test violated Title VII.^[14] The court's decision was upheld by the Eleventh Circuit Court of Appeals, which noted that the State Board did not even attempt to defend the validity of the AITCTP test and, "in fact, it conceded at trial that plaintiff need not relitigate the issue of test validity."^[15]

B. Major provisions of the decree^[16]

Under the consent decree, a new test would be developed for teacher candidates. Plaintiffs and defendants would name experts to serve on a three-member monitoring panel to oversee the test-development process.^[17] The test would be subject to restrictions based on p-values of questions.^[18] A p-value is a basic measure of a question's difficulty and simply reflects the proportion of test-takers that answer a question correctly.^[19] For example, if 55% of the students answered an item correctly, then its p-value is .55.^[20] Under the decree, Type I items are "those for which the item difficulty (i.e. p-value) for blacks and whites differs by no more than 5%."^[21] Type II items are those "for which the item difficulty for blacks and whites differs by more than 5% points but not more than 10%."^[22] Type III items are "those for which the item difficulty (i.e.p-values) differ[s] by a percentage which is greater than 10% but not more than 15%."^[23] Under the consent decree, all test items would be field-tested before use on an actual test.^[24] On the actual test, Type I items would be used exclusively "so long as they are available in sufficient numbers to provide comprehensive coverage of the objectives sought to be measured in the examination."^[25] Type II items could be used "only if Type I items are not available in sufficient numbers to provide comprehensive coverage of the objectives sought to be measured in the examination."^[26] The decree further provided that after the examination had been used a number of times, the test items would be reclassified to determine the actual difference in p-values among racial groups.^[27] If there were not a sufficient number of Type I and Type II items, then Type III items could be used on the test.^[28] Type III items could never comprise more than 10% of the total number of items on the test unless the monitoring panel agreed otherwise.^[29] The decree provided that if a candidate did not achieve a passing score on the test, then a separate formula could be used under which the candidate's college GPA counted 50% and test score counted 50%.^[30]

The decree further provided that any new test must be "developed and validated in accordance with the technical standards contained in the 1978 Uniform Guidelines on Employee Selection Procedures, 43 Fed.Reg. 38240, to the extent said guidelines are applicable, and the most current Standards for Educational and Psychological Tests published by the American Psychological Association, Inc."^[31]

C. Prior attempts to obtain a test developer

In a letter dated December 5, 1986, the Department of Education wrote Educational Testing Service ("ETS"), asking it to provide cost estimates for developing a new teacher test for Alabama to comply with the consent decree.^[32] ETS is a major testing company, whose teacher certification test is used by approximately 30 States.^[33] In January of 1987, ETS wrote back declining to develop a test.^[34] After ETS declined, the State Department of Education sent requests for proposals ("RFP's") for the redevelopment of the AITCTP test to major universities in Alabama and major test development companies around the nation.^[35] A bidder's conference was held on January 29, 1988.^[36] On March 25, 1988, National Computer Systems, Information Services ("NCS"), a major test developer, submitted a proposal to develop a test for $1,149,427.^[37] The proposal included the use of IOX Associates as a subcontractor to develop test items.^[38] Dr. James Popham, who is now serving as an expert for defendants, is the founder of IOX and has served as its director since 1968.^[39]

The Evaluation and Assessment Laboratory of the College of Education at the University of Alabama made a proposal for $805,027.^[40] However, the proposal was conditioned upon modification of the indemnification requirement in the RFP.^[41] The indemnification requirement stated as follows: "The contractor shall agree to indemnify, defend, and hold harmless the Alabama State Board of Education, its officers, agents and employees against all liability or loss that said parties may sustain as a result of suits, claims, damages, costs, or judgments (including but not limited to civil rights damage claims and suits, constitutional rights damage claims and suits, copyright infringement claims and suits) arising from the contractors's performance of the contract(s) described in this RFP."^[42] The indemnification requirement was an obstacle to other test developers as well. Both Auburn University and CTB-McGraw Hill, a major test development company, wrote to say that because of the indemnification requirement, they would not be making proposals.^[43] Three other companies wrote to say they would not be making proposals, one because of workload, one because the request had arrived too late, and one without giving a reason.^[44]

On June 13, 1988, NCS wrote Dr. C.C. Baker, Assistant Superintendent for Professional Services, reaffirming the company's interest in developing a test.^[45] In a letter dated September 1, 1988, Baker wrote back, stating that "The Alabama State Board of Education took action at their July 12, 1988, meeting to immediately suspend the teacher certification testing program. The AITCTP will not be redeveloped."^[46] In a letter dated July 22, 1988, the State Board notified the court that it had suspended the testing of teacher candidates in Alabama.

During the 1995 Alabama State legislative session, the Legislature enacted a law requiring that teacher candidates pass an examination as a condition for graduation.^[47] In response to the Legislature's action, plaintiffs filed a motion for further relief on August 9, 1995, requesting that the court issue a declaratory judgment and injunction requiring that any teacher certification test comply with the conditions of the consent decree. Defendants responded with motions to modify or vacate the consent decree. A trial was held on February 23, 1996. Rather than having numerous experts testify, the court allowed each party to designate one expert to summarize the deposition evidence and answer questions from the court.^[48]

II. DISCUSSION

A. Standards for modification of a consent decree

Modification of a consent decree, like modification of any judgment or order, is formally governed by Rule 60(b) of the Federal Rules of Civil Procedure. Rufo v. Inmates of Suffolk County Jail, 502 U.S. 367, 378, 380, 112 S.Ct. 748, 757, 761, 116 L.Ed.2d 867, (1992). Rule 60(b) authorizes a court to modify a final judgment or court order at any time if it finds that "the judgment has been satisfied, ... or a prior judgment upon which it is based has been reversed or otherwise vacated, or it is no longer equitable that the judgment should have prospective application, or [for] any other reason justifying relief from the operation of the judgment." Fed.R.Civ.P. 60(b)(5)-(6).

Traditionally, courts had required parties seeking modification of a consent decree under Rule 60(b) to demonstrate that continued application of the decree would result in "grievous wrong" on account of new and unforeseen conditions. United States v. Swift & Co., 286 U.S. 106, 119, 52 S.Ct. 460, 464, 76 L.Ed. 999 (1932). In recent years, however, the Swift standard has been rejected as too rigid for "institutional reform litigation." See, e.g., Hodge v. HUD, 862 F.2d 859 (11th Cir.1989) (per curiam); New York State Ass'n for Retarded Children, Inc. v. Carey, 706 F.2d 956 (2d Cir.), cert. denied, 464 U.S. 915, 104 S.Ct. 277, 78 L.Ed.2d 257 (1983); Philadelphia Welfare Rights Org. v. Shapp, 602 F.2d 1114 (3d Cir.1979), cert. denied, 444 U.S. 1026, 100 S.Ct. 689, 62 L.Ed.2d 660 (1980). The Supreme Court's decision in Rufo established a two-part standard for determining when modification of such decrees is appropriate. 502 U.S. at 383-384, 112 S.Ct. at 760.^[49]

First, the party seeking modification of the decree "bears the burden of establishing that a significant change in circumstances warrants revision of the decree." Rufo, 502 U.S. at 383, 112 S.Ct. at 760. The party may satisfy this initial burden "by showing either a significant change in factual conditions or in law." Id. at 384, 112 S.Ct. at 760. The Court identified several situations in which a significant change in factual conditions could warrant modification of a decree. Modification could be appropriate where a change in conditions has made compliance with the decree "substantially more onerous." Id. Modification could also be appropriate "when a decree proves unworkable because of unforeseen obstacles," or, lastly, "when enforcement of the decree without modification would be detrimental to the public interest." Id. In addition, modification of the decree may be appropriate where there has been an intervening change in the governing law, such that one or more of the obligations the decree imposes upon the parties has been made impermissible under federal law or, conversely, the law has changed to make legal what the decree was designed to prevent. Id. at 388, 112 S.Ct. at 762.

Second, if the moving party meets this standard, the court then considers "whether the proposed modification is suitably tailored to the changed circumstance." Id. at 383, 112 S.Ct. at 760. "A proposed modification should not strive to rewrite a consent decree so that it conforms to the constitutional floor," id. at 391, 112 S.Ct. at 764; rather, any modification to the decree must be directly responsive to the problem created by the changed circumstances. The Rufo court recognized that, in applying this two-part standard, a court must take into consideration the effect of the proposed modifications on the underlying purpose of the consent decree. Where the defendant requests a modification that is clearly designed to further the goals of the decree, for example, the court may be more flexible in its application of the two-part test. Id. at 381 n. 6, 112 S.Ct. at 758 n. 6. Similarly, if the proposed modification concerns only minor details of the decree and does not affect the decree's purpose, the court may forgo application of the Rufo standard altogether. Id. at 383 n. 7, 112 S.Ct. at 760 n. 7. On the other hand, if the modification would in any manner frustrate or undermine the decree's purpose, the court must proceed with extreme caution in reviewing the justification for the proposed changes. Id. at 387, 112 S.Ct. at 762.

B. Changes in factual circumstances

Under the first prong of the Rufo test, defendants bear the burden of showing that a significant change in facts has either made compliance with the decree substantially more onerous, has made the decree unworkable because of unforeseen obstacles, or when enforcement of the decree would be detrimental to the public interest. Defendants contend that the following factual changes warrant modification or vacatur of the consent decree: recent research has uniformly condemned p-value restrictions of the type used in the decree; ETS, the largest developer of teacher certification tests, will not develop a test in Alabama; any test which complied with the decree would be too difficult or easy to provide a meaningful evaluation of students; the cost would be prohibitive; and the monitoring panel is ineffective.

1. Recent research on p-value restrictions

Defendants contend that the consent decree should be modified or vacated because recent research has demonstrated that it is based on unsound psychometric principles. The limitation on adverse impact through restrictions in p-value differences is called the "Golden Rule" approach because the methodology was first used in a case called Golden Rule Life Insurance Company v. Mathias, 86 Ill.App.3d 323, 41 Ill.Dec. 888, 408 N.E.2d 310 (1980). In that case, the Golden Rule Insurance Company and five individuals who had failed the certification examination to become insurance agents in Illinois sued the State insurance commissioner and ETS, which had developed the test. Id. 41 Ill.Dec. at 890-91, 408 N.E.2d 310 at 312-13. In November 1984, the parties agreed to a settlement which included the use of restrictions in p-value differences.^[50] The consent decree in this case is modeled after the one in Golden Rule. Two years after the settlement in Golden Rule, the President of ETS denounced the settlement as "a mistake ... that has been used to justify legislative proposals that go far beyond the very limited terms of the original agreement."^[51] He stated that "We recognized that the settlement compromise was not based on an appropriate `bias prevention' methodology for tests generally. What was to become known as the `Golden Rule' procedure is based on the premise — which ETS did not and does not share — that group differences in performance on test questions primarily are caused by `bias.' The procedure ignores the possibility that differences in performance may validly reflect real differences in knowledge or skill."^[52]

A key point in the debate about the Golden Rule methodology is the difference between disparate impact and bias.^[53] As Dr. Popham, expert for defendants, explained at trial, disparate impact simply means "that one group scores differently [from] another group on a test."^[54] Therefore, if blacks pass a test at a lower rate than whites, the test has a disparate impact on blacks. Dr. Popham defines bias as "any quality of the test item, or the test, that offends or unnecessarily penalizes the examinees on the basis of personal characteristics, such as gender, ethnicity, and so on."^[55] For example, a math question that involved football scores might be biased against women.^[56] But the existence of a disparate impact does not necessarily indicate bias.^[57] A disparate impact may be due to legitimate differences in skills and knowledge, which could, for example, be attributable to the fact that one group has received better educational opportunities than another group.^[58] Defendants have provided substantial literature from experts in the field who are concerned that the Golden Rule methodology could improperly be used in place of anti-bias measures. On April 11, 1988, the Committee on Psychological Tests and Assessment of the American Psychological Association issued a statement saying that, "Whatever the merits or shortcomings of the Golden Rule agreement settlement, it cannot be regarded as a model for antibias legislation.... The procedure described in the Golden Rule agreement, however, was not originally developed to be a generally applicable method of reducing item bias. There is considerable concern that scientific judgments may be compromised by legal and political considerations promulgated as a result of the Golden Rule agreement."

In this case, however, the consent decree does not substitute the Golden Rule methodology for a review of test items to determine if they are biased. The Golden Rule methodology is outlined in ¶ 2 of the consent decree. The anti-bias provisions are contained in ¶ 6 of the consent decree, which state in pertinent part that: "The defendants shall include in any test development process, in addition to all of the other standards enunciated herein, a review of all examinations for racial bias by panels of black Alabama educators who have expertise in the content field area of the tests they are reviewing. This panel ... shall make recommendations to the test monitoring panel concerning their findings of bias, if any." Because the consent decree includes both the Golden Rule methodology and the anti-bias techniques typically used by test developers, the court concludes that it is not in conflict with the opinions of experts who warn that the Golden Rule methodology should not be used in place of anti-bias provisions.

The other major concern among critics of the Golden Rule methodology is that it could compromise the validity of tests by placing arbitrary limits on disparate impact. "Generally, validity is defined as the degree to which a certain inference from a test is appropriate and meaningful." Richardson v. Lamar County Bd. of Educ., 729 F.Supp. 806, 820 (M.D.Ala.1989) (Thompson, J.). A test which is valid will test the full range of skills and knowledge required by a teacher. Id. at 820-21. After the Golden Rule settlement, legislation was proposed in a number of States to require that all licensing examinations adhere to similar restrictions. Richard Jaeger, a past president of the National Council on Measurement in Education, stated that, "If judgments about test bias were made on the basis of the criteria proposed in these bills ... serious adverse consequences for students and educational institutions would result: Tests constructed in such a manner would result in scores for an individual that were less reliable and less valid indicators of that individual's abilities in the area being tested."^[59] Two other leading researchers, Robert Linn and Fritz Drasgow, noted that even if the Golden Rule settlement did not have a negative effect on the validity of the tests involved in that case, the fact that "it is based on pragmatics rather than on sound psychometric principles ... sets a bad precedent, one that could seriously undermine the validity of many other tests to which the procedures might be applied."^[60] In commenting on this case, attorney Michael Rebell stated that an invalid test with easier items "would ensure the certification of more applicants — both minority and nonminority — without assessing their competence by meaningful objective standards. Job access would be provided to minority candidates, but at the cost of undermining certification standards and possibly lowering the quality of education of future generations of school children."^[61]

In this case, the consent decree requires that the test be valid. As stated, ¶ 4 of the consent decree states that "Should the defendants elect to continue teacher certification examinations as part of their existing certification requirements, they shall, as soon as practicable after the entry of this decree, implement the development of subject matter examinations to assure that said examinations meet the specifications in paragraph two herein and are developed and validated in accordance with the technical standards contained in the 1978 Uniform Guidelines on Employee Selection Procedures, 43 Fed.Reg. 38240, to the extent said guidelines are applicable, and the most current Standards for Educational and Psychological Tests published by the American Psychological Association, Inc." (Emphasis added). The plain language of this paragraph dictates that the test must be validated in accordance with the most recent standards in the field. Furthermore, legal considerations dictate that validity trump the need to reduce disparate impact. The purpose of the consent degree is to try to develop a valid test which has a lower disparate impact on black candidates than the off-the-shelf tests that are currently available. The Equal Employment Opportunity Commission ("EEOC") Guidelines to Title VII state that "Where two or more selection procedures are available which serve the user's legitimate interest in efficient and trustworthy workmanship, and which are substantially equally valid for a given purpose, the user should use the procedure which has been demonstrated to have the lesser adverse impact." 29 C.F.R. § 1607.3(B).

Defendants contend that, given expert opinion, it would be fruitless to attempt to develop a valid test because "we know what the outcome is going to be."^[62] Yet the position of defendants is contradicted by the testimony of their own expert, Dr. Mehrens, who was asked whether it is possible to develop a valid test under the terms of the decree. Dr. Mehrens stated that "It's impossible to know whether a company could build that test until it was attempted. My guess is that you would not be able to meet the requirements of the consent decree."^[63] On the other hand, Dr. Pedulla and Dr. Poggio, who are experts for the plaintiffs, both stated that they believe a valid test can be developed within the limitations of the consent decree.^[64] When asked whether the p-value restrictions in the Golden Rule settlement undermined its validity Dr. Mehrens stated, "It's a little hard to answer that.... I think there probably could be debates in the field about whether content validity was maintained, even recognizing as we both do that the consent decree does allow provisions for attempting to maintain content validity."^[65] When asked to give a yes or no answer as to whether a content valid test could be developed if special validity components were followed, Dr. Mehrens stated that "I'll give you the same answer I gave earlier. There would be some debate about that in the field."^[66] Because there is a debate among experts in the field as to whether a valid test can be developed under these restrictions, the court concludes that recent research does not indicate that it would be a fruitless exercise to attempt to develop such a test.

2. Availability of a test developer

Defendants contend that the consent decree should be modified because "the primary testing company in the country will not implement a test in Alabama under the terms of the decree."^[67] In a letter dated December 5, 1986, the Department of Education wrote ETS, asking them to provide cost estimates for adapting some of its existing tests to the consent decree requirements and developing some custom-made tests for Alabama.^[68] However, at almost the exact same time, the President of ETS announced that the Golden Rule settlement had been a "mistake."^[69] In the spring of 1988, the President of ETS wrote an article heavily criticizing the Golden Rule methodology, and ETS has declined subsequent requests to develop a test in Alabama.^[70]

However, despite ETS's criticisms, a number of reputable test developers were interested in 1988 in developing a test for the State. On March 25, 1988, NCS made a proposal to develop a test for Alabama using IOX as a subcontractor.^[71] According to the proposal, NCS was a major test developer with over 600 full-time employees in its testing division.^[72] IOX, its subcontractor, had developed "high stakes" teacher tests for use in Tennessee, South Carolina, Arkansas, and Texas.^[73] When NCS/IOX did not hear from the State Board, they wrote again on June 13, 1988, to Assistant State Superintendent Baker, to re-express their interest in developing a test for Alabama.^[74] At the top of the letter is a handwritten notation stating "no response necessary per CCB 6/17/88."^[75] Presumably, the initials "CCB" stand for Dr. C.C. Baker. The State Board has presented no evidence why it rejected the NCS bid and why it voted in July of 1988 not to develop the AITCTP test.

Furthermore, the evidence indicates that three other organizations (the University of Alabama, Auburn University, and CTB McGraw-Hill) would have made proposals to develop a test if they had not been required to indemnify the State Board. In its letter dated March 9, 1988, CTB McGraw-Hill stated, "As you are no doubt aware, CTB/McGraw Hill is one of the largest test publishers in the country and is well known for its test development and psychometric/research capability. In the past, we have successfully bid and carried out a significant number of statewide testing programs." The letter further stated that "We were eager to become involved ... [but] of major concern to us in the RFP is that the contract indemnify, hold harmless and defend Alabama against all liability, including claims arising from suits based in civil rights, constitutional rights, and copyrights. At the bidders conference it was unequivocally confirmed that this RFP paragraph (3.32) was not negotiable. As a result, we have concluded that since this requirement is so vastly different from our normal procedure of assuming liability for negligence, errors, and omissions, in performing on a contract, and since it is not negotiable, we must decline to bid. If the Department of Education sees fit at a future time to alter its position on the paragraph of the RFP, we would be pleased to provide a proposal."^[76]

Auburn University responded in a similar manner. The Dean of Auburn's College of Education wrote that "Auburn University cannot agree to the requirements of the indemnification clause of the request for proposals. In particular, the University would have to limit its liability to events resulting from negligence as a contractor. The response by the State Department of Education to a question submitted at the bidder's conference indicated that the State would not accept such a limitation to the indemnification clause. The University Risk Manager has advised us that liability insurance is not available for civil rights claims and suits. Therefore, we cannot meet the strict requirements of the indemnification clause."^[77] In the indemnification section of its proposal, the University of Alabama stated that "Employees or agencies of the State of Alabama are unable legally to waive the state's immunity from suit. The University will negotiate the specific terms of the final contract according to Alabama law."^[78] There is no indication in the record that the State ever waived its indemnification requirement. The statements of Auburn University and CTB McGraw-Hill indicate that it is not a normal operating procedure to require a test developer to indemnify users of the test for liability arising from civil rights claims. The fact that ETS does not want to develop a test in Alabama does not make the consent decree unworkable. As the testimony of defendants' expert, Dr. Mehrens, indicated, there is a debate in the testing community regarding whether valid tests can be developed under the Golden Rule methodology. ETS and the academic researchers cited by defendants are on one side of that debate. However, the evidence reflects that there are academics who disagree, and there are reputable test developers who believe a valid test can be developed under the Alabama consent decree. Defendants have presented no evidence that those test developers are either disreputable or are no longer willing to develop a test in Alabama. Therefore, the court concludes that the consent decree is not unworkable for lack of a test developer.

3. Validity

Defendants contend that the requirements of the consent decree prevent the development of a test that is valid. As stated, "Generally, validity is defined as the degree to which a certain inference from a test is appropriate and meaningful." Richardson, 729 F.Supp. at 820. A test which is content valid will test the full range of skills and knowledge required by a teacher. Id. at 820-21. According to Dr. Popham, expert for defendants, the problem with the consent decree is that questions with low p-values tend to be either very easy, so that almost all test-takers answer them correctly, or very difficult, so that almost all test-takers answer them incorrectly.^[79] As a result, test developers are unable to test the full content of skills and knowledge required of a teacher and are unable to draw valid inferences from the test about the test-taker's capabilities.^[80]

While the court does not question Dr. Popham's knowledge and expertise in this field, the credibility of his testimony is seriously diminished by the fact that his company previously submitted a proposal to develop a test at a reasonable price which complied with the restrictions of the consent decree. At his deposition, Dr. Popham was asked if at the time of the consent decree, he would have developed a test under those restrictions, and Dr. Popham stated as follows:

"A: ... had we been asked to develop a test under these restrictions, I can tell you we probably — We would not have done it. I would have declined the invitation. But let's say Alabama offered an enormously large amount of money — sufficiently large that I relaxed my ethical standards to do the test, what we would have done is try to create a whole series of exceedingly difficult items or exceedingly easy items ...

Q: You said, Dr. Popham, that your firm would have declined to build this test?

A: Under those restrictions.

Q: Yes. The test prescribed by the consent decree.

A: Correct.

Q: Would your firm have declined to construct the test in 1985 or 1987?

A: Would have declined given the conditions of the consent decree?

Q: Yes sir.

A: I'm pretty sure we would have, unless we thought we could have persuaded someone somehow to modify the consent decree so it was not so restrictive ...

Q: You have made it clear to me that IOX would have declined to build this test. And I understand your testimony to be that they would likewise have declined to build it in 1985 and 1987 because of the same reservations that you have today.

A: That's right."^[81]

Dr. Popham further stated in his deposition that "I find it hard to believe that someone who's engaged in the day-to-day development of tests could have countenanced that agreement. It's hard to know why all of the parties agreed to [the consent decree], but I just can't believe any seasoned test developer would have agreed to it, because the basic dysfunctionality requirement should have been immediately apparent. It just wouldn't work. To create a test based on those kinds of requirements would have been an exercise in deceit."^[82]

Dr. Popham's testimony is contradicted by the fact that in 1988, his company did, in fact, make a proposal with NCS to develop a test for the Alabama State Department of Education under the terms of the consent decree.^[83] The proposal stated that IOX could create test items that "are consonant with the stipulations of the consent decree" and that "IOX staff have the expertise to create a redeveloped AITCTP that is legally defensible, instructionally illuminating, and fundamentally equitable for all examinees."^[84] A section of the proposal titled "Legal Defensibility" stated that "All phases of the test development work will be carried out in complete adherence to the Consent Decree and to the relevant guidelines embodied in the APA-AERA-NCME 1985 Standard for Educational and Psychological Testing and the 1978 Uniform Guidelines on Employee Selection. IOX personnel are thoroughly versed in these important legal considerations and will cleave to them assiduously."^[85] The record further reflects that, as Director of IOX, Dr. Popham wrote a letter dated March 17, 1988, to Mr. Jay Clark, Vice President of NCS, stating that "IOX is happy to serve as your subcontractor in this project, recognizing in the early phases of the work that we shall be playing a prominent role in developing the required examinations in a manner consonant with Alabama's requirements."^[86]

At trial, Dr. Popham tried to address the contradiction in his testimony and stated as follows: "When I was being deposed in December, and asked about whether or not we would submit a proposal to the state of Alabama, I absolutely didn't remember having done so. And you can say, well, how can you possibly forget having prepared a proposal? And the reality is that during that period we were probably producing around eight to twelve proposals a year. And as soon as they go out the door, after a couple of weeks, I forget about them ... And then since that time, we've done a lot more development of teacher-licensure tests, in particular subject areas, in areas that would pertain to the situation in Alabama. So I've had a lot more experience, and read more about the Consent Decree. So when we got to that point, and he asked would you have done it that way back then, I said of course not, but then later on, they sent me the proposal, and I realized that back then, I simply forgot about it."^[87]

Dr. Popham's testimony is further contradicted by the fact that three other organizations stated that they could submit a valid test which complied with the restrictions decree. In addition, Dr. Poggio, expert for plaintiffs, testified at trial that it is his opinion that "empirical research shows that a test could be constructed under Allen type provisions that samples full test domain and does not compromise validity."^[88] Because four test developers have indicated that they can develop a test within the restrictions of the consent decree and because there is a split of opinion among experts on this issue, the court concludes that defendants have failed to establish that a valid test cannot be developed and modification of the consent decree is warranted.

4. Cost of developing a test

Defendants contend that the cost of developing a test within the limitations of the consent decree would be prohibitive. Under Rufo, modification could be appropriate where a change in conditions has made compliance with the decree "substantially more onerous." 502 U.S. at 392-93, 112 S.Ct. at 764. "Financial constraints may not be used to justify the creation or perpetuation of constitutional violations, but they are a legitimate concern of government defendants in institutional reform litigation and therefore are appropriately considered in tailoring a consent decree modification." Id.

There are three basic types of tests available. First, the State could use a preexisting test without tailoring it to the needs of Alabama. In his deposition, Dr. Popham stated that NCS, ETS, and a company called National Evaluation Systems have each developed "off the shelf" tests that have been validated.^[89] Such a test would effectively cost the State nothing because the fee paid by each candidate to take the licensing examination would cover its cost.^[90] However, Dr. Popham recommends that Alabama tailor a preexisting test to suit its particular needs and circumstances. He stated that "what I'm hoping your superintendent and your policymakers would do is recognize that this is a rare opportunity to help all teachers in the state, including the candidates, ready themselves for a decent x plus one test. And therefore, there would be some cost associated in that, but it could be done at modest expenditure."^[91] Dr. Popham did not specify how much that "modest expenditure" would be but stated that it would be far less than a custom-made test.^[92] None of the off-the-shelf tests currently available would meet the p-value requirements of the consent decree, even if they were tailored to suit the needs of Alabama.^[93]

The final option would be a "develop-your-own" test custommade for Alabama. Dr. Popham stated "that in general, if I'm asked by state authorities to recommend off-the-shelf versus develop your-own, in our current economy, I recommend off-the-shelf. Its simply — Why not use those dollars to educate children as opposed to develop a brand new test from scratch?"^[94] Dr. Popham further stated that each State must make the policy choice of whether a custom-made test is worth the extra cost.^[95] Although his bid in 1988 for a series of custom-made tests was $1.15 million, Dr. Popham estimates that today, the same series of tests would cost approximately $2 million to $3 million because of inflation and the realization among test developers that tests cost more to develop than they had originally anticipated.^[96] Using $2.5 million as a current yardstick, he estimates that a test which complied with the consent decree would probably "cost two to three times that, but I really don't know, it hasn't been done."^[97] If the p-value restrictions are rigidly adhered to, he estimates that it could cost as much as 20 times that amount.^[98] In contrast, Dr. Poggio estimates that a valid, custom-made test could be developed at a total additional cost of approximately $8,000 to $10,000.^[99]

In this case, the court is required to determine whether financial costs have become so prohibitive that the consent decree should be modified. The court cannot make such a determination based on general estimates by experts which vary so dramatically and which are essentially nothing but wild guesses and are not rooted in sound analysis. The court asked Dr. Popham how much it would cost the State to obtain actual bids on how much it would cost today to develop a test that complied with the consent decree.^[100] Dr. Popham stated that the only costs to the State would be the cost to the agencies of developing a request for proposal and "tying up a couple of staff members for a month or more."^[101] Because of the importance of the issues involved and the relatively low cost of obtaining actual bids, the court concludes that defendants must produce more specific and concrete data on what it would cost to develop a test today before the court can evaluate whether excessive cost justifies modification of the consent decree.

5. Composition of monitoring panel

Defendants contend that "the way the monitoring panel is composed with members appointed who have allegiance to the parties will not result in independent oversight of the exam."^[102]Rufo provides that courts can consider modification of a decree "when a decree proves unworkable because of unforeseen obstacles." 502 U.S. at 384, 112 S.Ct. at 760. Under the consent decree, each party was allowed to name one member to the oversight panel. Dr. Mehrens, who is defendants' appointee to the panel, testified at his deposition that although the committee is supposed to act as an independent body, the parties picked panel members based on their views on particular issues, and he believes it would be difficult for panel members to leave those views behind.^[103] Dr. Mehrens believes such a situation would create an imbalance in the panel because two of the members were appointed by plaintiff groups and one by the defendants.^[104]

Defendants have not shown any reason why this obstacle was unforeseeable at the time they drafted the consent decree. They agreed to a panel in which each party appointed one member. Furthermore, defendants have not shown any instance in which disagreements among panel members prevented the panel from functioning effectively on an important issue. The evidence reflects that there has been very little for the panel to do because defendants decided in 1988 to suspend testing of teacher candidates. When Dr. Mehrens was asked if he is still on the panel, he said "If I'm on it, it's been totally inactive for eight years or so.... I deem that as a nonexistent panel, but maybe, in fact it still exists."^[105] Dr. Poggio testified at trial that the panel met for only a year and that he does not recall any major disagreements along partisan lines.^[106]

If defendants can demonstrate that the panel is not functioning effectively, Rufo provides that the consent decree can be modified if the court determines "enforcement of the decree without modification would be detrimental to the public interest." Id. at 384, 112 S.Ct. at 760. However, until evidence is presented of instances in which the panel is acting or has acted in ways detrimental to the public interest, the court will not consider modification of the panel's structure.

6. Limitations on test content

The consent decree states that "The examination shall test only those matters which are contained in the approved teacher education program curriculum requirements for Alabama's teachers." Dr. Mehrens states in his report that such a requirement is inappropriate because it was adopted from the case of Debra P. v. Turlington, 730 F.2d 1405 (11th Cir.1984), which is relevant to high school curricula but not to professional licensing tests.^[107] Dr. Mehrens also believes the requirement is inappropriate because the test should not be limited to what is taught in the teacher education programs.^[108] Defendants fail to show why these problems could not have been anticipated at the time they entered the decree or why they constitute changes in facts justifying modification of a consent decree under Rufo. Furthermore, Dr. Mehrens stated that "a reasonable argument could be made" that limiting the contents of the test to the curricular requirements of teacher education programs is only a problem if the State has no control over the curricular requirements of teacher education programs.^[109] Defendants have not made a showing that the State has no control over those curricular requirements. Therefore, the court concludes that defendants have failed to produce sufficient evidence to show that limiting the content of the test to that of approved teacher education programs requires modification of the consent decree.

7. Other issues

Defendants raise three additional reasons why the consent decree should be modified or vacated, none of which has merit. First, defendants contend it should be vacated because the Alabama Legislature recently passed a law requiring testing of teacher candidates. 1995 Ala. Act 262.^[110] The Supremacy Clause in Article VI of the United States Constitution dictates that to the extent that State and federal law conflict, State law must yield. Therefore, 1995 Alabama Act 262 does not alter the State's obligations under a federally approved consent decree. Second, defendants cite the fact that the State educational system has been found to violate the Alabama Constitution.^[111]Alabama Coalition for Equity v. Hunt, ___ So.2d ___, 1997 WL 7736 (Ala. Jan. 10, 1997). Defendants do not explain why the consent decree impedes compliance with that suit. Furthermore, this issue is not ripe for consideration by the court because the suit is still pending. Third, defendants contend that the consent decree is flawed because it provides that if a candidate does not achieve a passing score on the test, then a separate formula could be used under which the candidate's college GPA counted 50% and test score counted 50%. In his deposition, defendants' expert Dr. Popham said that he saw some value in using GPA as a measure in borderline cases, but that the 50/50 split was too arbitrary.^[112] Defendants have not shown how this constitutes a change in fact or why they could not have anticipated it at the time they entered the decree. Therefore, it cannot be considered as grounds for modification of the decree.

C. Changes in legal circumstances

Defendants contend that recent changes in judicial interpretation of the fourteenth amendment require that the consent decree be modified or vacated because it violates the equal protection clause.^[113] In Rufo, the Supreme Court stated that modification of a decree may be appropriate where there has been an intervening change in the governing law, such that one or more of the obligations the decree imposes upon the parties has been made impermissible under federal law or, conversely, the law has changed to make legal what the decree was designed to prevent. 502 U.S. at 388, 112 S.Ct. at 762.

1. Applicability of strict scrutiny

In Adarand Constructors. Inc. v. Pena, 515 U.S. 200, 115 S.Ct. 2097, 132 L.Ed.2d 158 (1995), the Supreme Court held that governmental classifications based on race must be treated with skepticism and must face strict scrutiny. Based on its prior holdings, the Court provided examples of the types of classifications requiring strict scrutiny: any preference based on racial, ethnic or ancestral criteria is inherently suspect and must necessarily receive a most searching examination. Id. at 223, 115 S.Ct. at 2111.

The use of race in the consent decree in this case is distinguishable from those examples. The EEOC Guidelines to Title VII provide that "The use of any selection procedure which has an adverse impact on the hiring, promotion, or other employment or membership opportunities of members of any race, sex, or ethnic group will be considered to be discriminatory and inconsistent with these guidelines unless the procedure has been validated in accordance with these guidelines." 29 C.F.R. § 1607.3(A). The guidelines further state that "Where two or more selection procedures are available which serve the user's legitimate interest in efficient and trustworthy workmanship, and which are substantially equally valid for a given purpose, the user should use the procedure which has been demonstrated to have the lesser adverse impact." 29 C.F.R. § 1607.3(B).

In this case, the consent decree is designed to see if a valid test can be developed which has a lesser disparate impact on African-Americans. If such a test is developed, less qualified black individuals will not receive preference over other more qualified individuals, nor will any person be treated differently on account of his race or ethnic origin. Instead, all test-takers will be subject to the same requirements, irrespective of race.

In the voting rights context, the Supreme Court has stated that strict scrutiny must be applied if "`traditional distracting principles"' are "subordinated to racial objectives." Miller v, Johnson, 515 U.S. 900, 917-19, 115 S.Ct. 2475, 2489, 132 L.Ed.2d 762 (1995) (quoting Shaw v. Reno, 509 U.S. 630, 647, 113 S.Ct. 2816, 2827, 125 L.Ed.2d 511 (1993)). The analogous comparison, which this court must ask, is whether traditional testing principles such as validity, reliability, and elimination of bias have been subordinated to racial considerations. Throughout this opinion, the court has emphasized that validity trumps all other considerations and that the p-value limitations should not replace the anti-bias measures traditionally used by test developers. Therefore, the court concludes that the consent decree does not subordinate traditional testing principles to racial considerations, and strict scrutiny of the decree is not required.

2. Application of strict scrutiny

Even if strict scrutiny were applied, the consent decree would not violate the equal protection clause of the fourteenth amendment. The "purpose of strict scrutiny is to `smoke out' illegitimate uses of race by assuring that the legislative body is pursuing a goal important enough to warrant use of a highly suspect tool." Richmond v. J.A. Croson, Co., 488 U.S. 469, 493, 109 S.Ct. 706, 721, 102 L.Ed.2d 854 (1989) (plurality opinion). It "also ensures that the means chosen `fit' this compelling goal so closely that there is little or no possibility that the motive for the classification was illegitimate racial prejudice or stereotype." Id. Strict scrutiny requires that racial classifications "be narrowly tailored to further a compelling governmental interest." Shaw v. Reno, 509 U.S. at 643, 113 S.Ct. at 2825; accord Ensley Branch, N.A.A.C.P. v. Seibels, 31 F.3d 1548, 1564 (11th Cir.1994). In Adarand, the Supreme Court emphasized that it "wish[ed] to dispel the notion that strict scrutiny is `strict in theory but fatal in fact.'" 515 U.S. at 236, 115 S.Ct. at 2117 (quoting Fullilove v. Klutznick, 448 U.S. 448, 519, 100 S.Ct. 2758, 2795, 65 L.Ed.2d 902 (1980) (Marshall, J. concurring in judgment)).

a. Whether past discrimination justifies remedial action

Affirmative-action jurisprudence makes clear that a State has a compelling interest in remedying its past and present discrimination. E.g., United States v. Paradise, 480 U.S. 149, 167, 107 S.Ct. 1053, 1064, 94 L.Ed.2d 203 (1987); Seibels, 31 F.3d at 1565. In determining whether there has been prior discrimination by a public employer that justifies remedial action, courts have looked to whether the evidence supports a prima-facie case of employment discrimination under Title VII. Croson, 488 U.S. at 501, 109 S.Ct. at 725-26; Wygant v. Jackson Bd. of Educ., 476 U.S. 267, 292, 106 S.Ct. 1842, 1856, 90 L.Ed.2d 260 (1986) (O'Connor, J., concurring); Peightal v. Metropolitan Dade County, 26 F.3d 1545, 1553 (11th Cir.1994). In determining whether a State actor has engaged in racial discrimination sufficient to justify remedial measures based on race, the court must conduct an evidentiary inquiry. E.g., Peightal, 26 F.3d at 1553. The governmental body must have a "`strong basis in evidence for its conclusion that remedial action was necessary.'" Croson 488 U.S. at 500, 109 S.Ct. at 725 (majority opinion) (quoting Wygant, 476 U.S. at 277, 106 S.Ct. at 1849 (plurality opinion)). The test can also be formulated as requiring "a prima facie case of a constitutional or statutory violation." Croson, 488 U.S. at 500, 109 S.Ct. at 725 (majority opinion). In assessing the strength of the evidence, the court cannot depend on societal discrimination alone. Id. at 500, 109 S.Ct. at 725. Race-conscious relief is justifiable only if discrimination is identified with some specificity in the governmental unit to which the relief is to apply. Id. at 504, 109 S.Ct. at 727; Peightal, 26 F.3d at 1553.

In this case, there has been a finding that defendants engaged in a statutory violation with regard to the testing of African-American teacher candidates. In Richardson v. Lamar County Board of Education, 729 F.Supp. 806, 817, 821-824 (M.D.Ala.1989) (Thompson, J.), this court found that the tests administered between 1981 and 1987 violated Title VII because they had a significant disparate impact on blacks and lacked validity. It is not relevant that the court's finding of a Title VII violation postdated the entry of the consent decree. In Seibels, the Eleventh Circuit faced a virtually identical issue, when it was required to determine whether affirmative action measures in a consent decree signed by the City of Birmingham and its personnel board could be justified as remedies for past discrimination. 31 F.3d at 1565-1568. The court stated as follows: "The City and Board may defend their programs by showing enough evidence of discrimination to create a strong basis for the conclusion that past or present discrimination warrants race-based remedies in departments in addition to the police and fire departments. It is not necessary (but would of course be sufficient) for the City and Board to show that, when they approved the decrees, they already had strong evidence of such discrimination. This case concerns only the prospective validity of the decrees, and prospective validity can be established just as well with new evidence as with old. If the City and Board can now show strong evidence of the need for affirmative action in a department, then future affirmative action in that department is justified." Id. at 1567-68. Because there has been a specific finding that the teacher tests in place prior to the entry of the consent decree violated federal anti-discrimination laws, the court concludes that the consent decree serves a compelling State interest.

b. Whether the decree is sufficiently narrowly tailored

After the court has determined that remedial provisions are justifiable, the court must determine whether the provisions of the consent decree are narrowly tailored. The Supreme Court has recognized that deciding whether relief is appropriate to remedying racial discrimination is necessarily a balancing process left to the discretion of the district court within certain limits. Paradise, 480 U.S. at 184, 107 S.Ct. at 1073; see also Fullilove v. Klutznick, 448 U.S. 448, 508, 100 S.Ct. 2758, 2790, 65 L.Ed.2d 902 (1980) (opinion of Powell, J., concurring). The Supreme Court and the Eleventh Circuit have established that there are four factors that must be analyzed to decide whether a remedy is narrowly tailored: "the necessity for the relief and the efficacy of alternative remedies; the flexibility and duration of the relief, including the availability of waiver provisions; the relationship of the numerical goals to the relevant labor market; and the impact of the relief on the rights of third parties." Paradise, 480 U.S. at 171, 107 S.Ct. at 1066; accord Seibels, 31 F.3d at 1568-69; Peightal 26 F.3d at 1557; In re Birmingham Reverse Discrimination Employment Litig., 20 F.3d 1525, 1545 (11th Cir.1994), cert. denied, 514 U.S. 1065, 115 S.Ct. 1695 (1995).

In this case, there is a necessity for relief because the court has determined that the prior teacher certification tests violated Title VII in that they were not valid and had a disparate impact on African-Americans. There are no effective alternate remedies because defendants' expert, Dr. Popham, has acknowledged that none of the off-the-shelf tests currently available would meet the p-value restrictions of the consent decree, even if they were modified to suit the needs of Alabama. The consent decree is flexible in that it allows test developers to first use Type I items, then Type II items, and finally Type III items, which have the largest difference in p-values. It is also flexible in the sense that validity trumps the need to limit disparate impact. In situations where these two goals of the decree conflict, validity will always be the primary factor. Although there are no sunset provisions in the decree, the decree will not need to continue indefinitely. Once a test is developed within the decree's specifications, the decree will become irrelevant. Furthermore, counsel for plaintiffs testified that, at the time the decree was drafted, plaintiffs deferred to defendants on the issue of a sunset provision, and defendants chose not to include one.^[114] Defendants have not contested that testimony.

Finally, in this case, there are not numerical hiring goals which relate to the labor market. Therefore, that factor is not relevant in determining whether the remedy is narrowly tailored. Finally, the attempt to limit disparate impact has a minimal effect on the rights of third parties. Less qualified black individuals will not be given preference over more qualified white individuals. Instead, the requirements of the test will be applied equally to each candidate, irrespective of race. Furthermore, unlike a promotion or hiring context, this is not a `zero sum' situation, in which any advantage to black teachers requires white teachers to be disadvantaged. Therefore, the court concludes that the consent decree does not trammel upon the rights of third-parties.

III. CONCLUSION

The court concludes by emphasizing two points. First, the thrust of this opinion is that the record before the court reflects that defendants have yet to make a good-faith effort to develop a test that both meets the requirements of the consent decree and is psychometrically sound or even to find out whether such a test can be developed. Unless and until they do, the need to modify, let alone vacate, the consent decree is premature. Second, any test developed pursuant to the consent decree must be psychometrically sound; it must be validated in accordance with the most recent standards in the testing field. Validity and soundness trump any need to redress discriminatory impact. The public deserves as much, and the court will approve nothing less.^[115]

For the foregoing reasons, defendants' motion to modify the consent decree, filed August 28, 1995, and defendants' motion in the alternative to vacate the consent decree, filed September 25, 1995, are denied, albeit without prejudice.

An appropriate judgment will be entered.

NOTES

[1] Richardson v. Lamar County Bd. of Educ., 729 F.Supp. 806, 817 (M.D.Ala.1989) (Thompson, J.).

[2] Id.

[3] Id. at 821.

[4] Allen v. State Bd. of Educ., 612 F.Supp. 1046 (M.D.Ala.1985) (Thompson, J.).

[5] Id.

[6] Because the positions of plaintiffs and plaintiff-intervenors are identical with respect to this issue and because they have submitted a joint brief, they are collectively referred to as "plaintiffs."

[7] Id. at 1049-50.

[8] Id. at 1050, 1052.

[9] Allen v. State Bd. of Educ., 636 F.Supp. 64 (M.D.Ala.1986) (Thompson, J.).

[10] Allen v. State Bd. of Educ., 816 F.2d 575 (1987).

[11] 729 F.Supp. 806 (M.D.Ala.1989) (Thompson, J.).

[12] Id. at 821.

[13] Id. at 822-24.

[14] Id. at 825.

[15] Richardson v. Alabama State Board of Education, 935 F.2d 1240, 1246 (1991).

[16] A copy of the decree was filed with the court on October 25, 1985.

[17] Consent decree at ¶ 5.

[18] Consent decree at ¶ 2.

[19] Expert Report of Dr. Poggio, plaintiffs' record exhibit P, at 2.

[20] Id.

[21] Consent decree at ¶ 1(b)(1).

[22] Id. at ¶ 1(b)(2).

[23] Id. at ¶ 1(d).

[24] Id. at ¶ 2(b).

[25] Id. at ¶ 2(c)(1).

[26] Id. at ¶ 2(c)(2).

[27] Id. at ¶ 2(d).

[28] Id.

[29] Id.

[30] Id. at ¶ 8.

[31] Id. at ¶ 4.

[32] Exhibit to motion to vacate consent decree, filed September 25, 1995.

[33] Exhibit 4 to motion to vacate consent decree, filed September 25, 1995.

[34] Exhibit to motion to vacate consent decree, filed September 25, 1995. In a letter dated March 30, 1995, ETS stated that it would not develop a test in Alabama unless the consent decree is changed.

[35] Plaintiffs' record exhibit F.

[36] Cover letter to plaintiffs' record exhibit F.

[37] Plaintiffs' record exhibit G.

[38] Id.

[39] Dr. Popham deposition at 8.

[40] Plaintiffs' record exhibit I.

[41] Id. at 16.

[42] Plaintiffs' record exhibit F at 13.

[43] Plaintiffs' record exhibits K and L.

[44] Plaintiffs' record exhibits M,N, and O.

[45] Plaintiffs' record exhibit H.

[46] This letter was not designated as an exhibit but was included in the material submitted by the parties for the record.

[47] 1995 Ala. Acts 262.

[48] The parties submitted a record of evidence on February 1, 1996, which contained numerous articles by experts in the field. They submitted a supplement to the record of evidence on February 6, 1996.

[49] In their brief filed on February 16, 1996, at 4, defendants contend that "the two-prong test for modifying a decree established in Rufo is not applicable to the case at bar because the Defendants are moving the court to vacate the degree." However, the Eleventh Circuit has stated that the principles in Rufo are applicable to both motions to vacate and motions to modify consent decrees. In United States v. City of Miami, 2 F.3d 1497, 1502 (11th Cir.1993), a firefighters union "filed a motion to dissolve the consent decree or, in the alternative, to modify the decree" in an employment discrimination case. The court discussed Rufo as binding precedent in a section titled "Standard for modification or termination of consent decrees in employment discrimination class actions." Id. at 1503-04. At the end of the section, the court stated that "We find that the principles articulated in Rufo and [Board of Education of Oklahoma City Public Schools v. Dowell, 498 U.S. 237, 111 S.Ct. 630, 112 L.Ed.2d 715 (1991)], are applicable to requests to modify or terminate decrees in employment discrimination class actions, like the one before us." In this case, as in City of Miami, defendants have moved to modify the consent decree or, in the alternative, to vacate it. Therefore, the court concludes that the two-prong test in Rufo is applicable to this case.

[50] Gregory Anrig, "Golden Rule": Second Thoughts, APA Monitor, January 1987, at 3.

[51] Id.

[52] Id.

[53] The terms "disparate impact" and "adverse impact" are considered synonymous and are used interchangeably in this case.

[54] Trial transcript at 80.

[55] Id.

[56] Id. at 81.

[57] Id.

[58] Id.

[59] Expert report of Dr. Mehrens at 12-13, exhibit 1 to the record of evidence. (quoting R.L. Jaeger, NCME Opposition to Golden Rule Legislation, Educational Measurement: Issues and Practice, 6(2) 1987, at 22).

[60] Expert report of Dr. Mehrens at 15 (quoting Robert Linn and Fritz Drasgow, Implications of the Golden Rule Settlement for T @ Construction, Educational Measurement: Issues and Practice, 6(2) at 13-17.

[61] Expert report of Dr. Mehrens at 20 (quoting Michael Rebell, Disparate Impact of Teacher Competency Testing on Minorities: Don't Blame the Test-Takers — or the Tests, 4 Yale L. & Pol'y Rev. 397.

[62] Trial transcript at 15.

[63] Dr. Mehrens deposition at 31.

[64] Trial transcript at 52; Dr. Pedulla deposition at 18.

[65] Dr. Pedulla deposition at 37-38.

[66] Dr. Mehrens deposition at 46-47.

[67] Motion to vacate consent decree, filed September 25, 1995, at 4.

[68] Exhibit to motion to vacate consent decree, filed September 25, 1995.

[69] Gregory Anrig, "Golden Rule": Second Thoughts, APA Monitor, January 1987, at 3.

[70] Gregory Anrig, ETS Replies to Golden Rule on "Golden Rule," Educational Measurement: Issues and Practice, 7(1) at 21 (1988).

[71] Plaintiffs' record exhibit G.

[72] Id. at II-1.

[73] Id. at 3.

[74] Plaintiffs' record exhibit H.

[75] Id.

[76] Plaintiffs' record exhibit L.

[77] Plaintiffs' record exhibit K.

[78] Plaintiffs' record exhibit I at 16.

[79] Expert report of Dr. Popham at 6-8, exhibit 7 to the record of evidence.

[80] Id. Defendants also contend that the decree is unwise because it enjoins the use of a separate test on basic professional studies. The test does allow that questions on basic professional studies be incorporated into all subject-matter examinations. In his report, Dr. Mehrens criticizes this as unwise because if the examination has only one cut score, it may be difficult to measure an examinee's skill in both areas. This analysis improperly assumes that the cut score could not be structured to assure that examinees meet minimum competency levels in both areas. Nothing in the consent decree prevents the test from being structured in such a way to assure the validity of the test. In any event, after further analysis, if it is necessary to modify the decree to accommodate some warranted changes, the court will consider them then.

[81] Dr. Popham deposition at 39-42.

[82] Id. at 138-39.

[83] Plaintiffs' record exhibit G.

[84] Id. at 2-3.

[85] Id. at III-23.

[86] Cover letter to plaintiffs' record exhibit I.

[87] Trial transcript at 77.

[88] Id. at 53.

[89] Dr. Popham deposition at 132-35.

[90] Trail transcript at 175.

[91] Id.

[92] Id.

[93] Dr. Popham deposition at 138.

[94] Id. at 127-28.

[95] Id. at 128.

[96] Trial transcript at 172.

[97] Id.

[98] Id. at 171.

[99] Id. at 19.

[100] Id. at 172.

[101] Id. at 173.

[102] Brief in support of motion to vacate consent decree, filed February 16, 1996, at 16.

[103] Dr Mehrens deposition at 80-86.

[104] Id. at 77-78.

[105] Id. at 59-60.

[106] Trial transcript at 56-57.

[107] Exhibit 6 to record of evidence at 18.

[108] Id. at 26-28.

[109] Id. at 27.

[110] Motion to modify consent decree, filed August 28, 1995, t 6.

[111] Motion to vacate consent decree, file September 25, 1995, t 4.

[112] Dr. Popham deposition at 84-87.

[113] Defendants do not make a claim that there has been a change in Title VII law requiring modification of the decree.

[114] Watkins deposition.

[115] In this vein, the court suggests that, as the experts proceed to develop a test, if a particular aspect of the decree is problematic, the parties should consider meeting with each to see if they can agree on a modification to address the concern before returning to court. In addition, the defendants should also first explore the possibilities of reasonable modifications before concluding that the decree must be vacated.