Ernest T. BROWN, et al., Plaintiffs,
v.
CITY OF CHICAGO, Defendant.
No. 95 C 1890.
United States District Court, N.D. Illinois, Eastern Division.
June 30, 1998.*1096 *1097 Craig Benson Futterman, Futterman & Howard, Chtd., Chicago, IL, Lesley A. Redman, Kenneth N. Flaxman, P.C., Chicago, IL, Kenneth N. Flaxman, Kenneth N. Flaxman, P.C., Chicago, IL, for Ernest Brown, David Byrd, Samuel E. Harris, Vance Kimber, plaintiffs.
James D. Holzhauer, Jeffrey Scott Piell, Angela K. Dorn, Mayer, Brown & Platt, Chicago, IL, Eileen B. Libby, City of Chicago, Joseph Robert Lipton, Law Department, Corporation Counsel, Chicago, IL, for City of Chicago, defendant.
MEMORANDUM OPINION AND ORDER
GETTLEMAN, District Judge.
I. INTRODUCTION
Did the defendant City of Chicago ("City") discriminate against minority Chicago police sergeants when it decided to make promotions to the rank of lieutenant based on an examination which the City developed with the acknowledged intention of avoiding an adverse impact on minorities? This case continues a series of attacks on successive attempts by the City to make police promotions based on criteria that would pass muster in the courts.
Plaintiffs are 44 minority (African-American and Latino) Chicago Police Department ("CPD") sergeants who took the 1994 lieutenant examination and were not promoted based on their scores. They claim that the City deprived them of equal employment opportunity in violation of Title VII of the Civil Rights Act of 1964, as amended 42 U.S.C. § 2000e, et seq. In light of the City's stipulation that the examination had an adverse impact on minority candidates, plaintiffs contend that, (a) the City has not met its burden of proving that the examination was a job related, content valid selection device, and (b) even if the test were valid, plaintiffs have demonstrated there was an equally valid, less discriminatory alternative selection method available which the City refused to use.
The case was tried to the court in November 1997. Extensive expert testimony and other evidence was presented. Thereafter, the parties submitted voluminous post-trial briefs and proposed findings and conclusions, and argued the matter to the court on March 6, 1998. Finally, on March 31, 1998, the parties submitted supplemental memoranda on the issue whether there existed an available, less discriminatory selection method than the rank order test. For the reasons set forth below, the court finds that the examination was content valid, but that the City had available a less discriminatory equally valid method of promotion utilizing merit selection along with rank order promotions.
II. LEGAL STANDARDS
In a Title VII case alleging discrimination in the promotion process, once the plaintiff proves a prima facie case by showing that the promotional method had an adverse impact on minorities as is stipulated in the instant case the burden shifts to the employer to prove that the test is valid. If the employer meets this burden, the plaintiff must prove that there was an available, equally valid, less discriminatory method for promotion that the employer refused to use. Albemarle Paper v. Moody, 422 U.S. 405, 425, 95 S. Ct. 2362, 45 L. Ed. 2d 280 (1975). Thus, the first issue presented is whether the City met its burden of proving that the test was valid, or "job related." Id.; 42 U.S.C. § 2000e-2(k)(1)(A).
In Griggs v. Duke Power Co., 401 U.S. 424, 91 S. Ct. 849, 28 L. Ed. 2d 158 (1971), the Supreme Court held that the employer must demonstrate that the test has "a manifest relationship to the employment in question" that reasonably measures job performance. Id. at 432, 436, 91 S. Ct. 849. "A test is job related if it measures traits that are significantly related to the applicant's ability to perform the job." Gillespie v. State of Wisconsin, 771 F.2d 1035, 1040 (7th Cir. 1985) (citing Griggs).
The Equal Employment Opportunity Commission ("EEOC") has promulgated Uniform Guidelines on Employee Selection Procedures ("EEOC Guidelines"), 29 C.F.R. § 1607, et seq., which draw upon current *1098 psychological literature on psychometrics as well as standards for test validation established by the American Psychological Association. Gillespie, 771 F.2d at 1040. Under the EEOC Guidelines (§ 1607.5B), an employment test may be validated under any one of three methods: criterion related; content validity; or construct validity. No one method is preferred; any of the three may be used by the employer to establish validity and shift the burden back to the employee.
In Gillespie (Id. at n. 3), Judge Coffey succinctly described these three methods:
A criterion-related validation study determines whether the test is adequately correlated with the applicant's future job performance. Wollack, Content Validity: Its Legal and Psychometric Bases, Personnel Management, Nov-Dec 1976, 397 at 402 (hereinafter "Wollack"). Criterion-related tests are constructed to measure certain traits or characteristics thought to be relevant to future job performance. Id. at 403. An example of an employment test that would be validated by the criterion-related validation method is an intelligence test. The content validation strategy is utilized when a test purports to measure existing job skills, knowledge or behaviors. Id. "The purpose of content validity is to show that the test measures the job or adequately reflects the skills or knowledge required by the job." Id. For example, a typing test given to prospective typists would be validated by the content validation method. Construct validity is used to determine the extent to which a test may be said to measure a theoretical construct or trait. Anastasi, Psychological Testing, 144 (1982) (hereinafter "Anastasi"). For example, if a psychologist gave vocabulary, analogies, opposites and sentence completion tests to a group of subjects and found that the tests have a high correlation with one another, he might infer the presence of a construct a verbal comprehension factor. Anastasi at 146.
In the instant case, the City has sought to establish the validity of the 1994 lieutenants' examination by showing that the test was content valid. Section 1607.14(C)(4) of the EEOC Guidelines provides:
[T]o be content valid, a selection procedure measuring a skill or ability should either closely approximate an observable work behavior, or its product should closely approximate an observable work product. If a test purports to sample a work behavior or to provide a sample of a work product, the manner and setting of the selection procedure and its level and complexity should closely approximate the work situation. The closer the content and the context of the selection procedure are to work samples or work behaviors, the stronger is the basis for showing content validity. As the content of the selection procedure less resembles a work behavior, or the setting and manner of the administration of the selection procedure less resemble the work situation, or the result less resembles the work product, the less likely the selection procedure is to be content valid, and the greater the need for other evidence of validity.
In determining whether an employment test is content valid, the court in Gillespie (771 F.2d at 1043) held that,
... the court must evaluate the test for: (1) the degree to which the nature of the examination procedure approximates the job conditions; (2) whether the test measures abstract or concrete qualities; and (3) the combination of these factors, i.e. whether the test attempts to measure an abstract trait with a test that fails to closely approximate the working situation. [Citing Guardians Ass'n of New York City v. Civil Service Commission, 630 F.2d 79, 93 (2d Cir.1980); Wollack at 405.06.]
Should the employer meet its burden of proving validity, the statute provides that the employee can succeed if he or she proves that there was an available, equally valid, less discriminatory method of promotion that the employer refused to use. 42 U.S.C. § 2000e-2(k)(1)(A)(ii). In such an event, the plaintiffs would be entitled to injunctive and affirmative relief. Franks v. Bowman Transp. Co., 424 U.S. 747, 770, 96 S. Ct. 1251, 47 L. Ed. 2d 444 (1976), Sheet Metal Workers v. EEOC, 478 U.S. 421, 448, 106 S. Ct. 3019, 92 L. Ed. 2d 344 (1986).
*1099 III. THE PARTIES' CONTENTIONS
Before addressing the facts and the application of the facts to the law, it is useful to understand the basic positions of the parties. Plaintiffs' argument rests to a large extent on a syllogism:
(a) The results of the 1994 lieutenants' examination had a gross adverse impact on minority applicants;
(b) Minorities as a group are as equally qualified as whites to be Chicago Police Department lieutenants;
(c) Therefore, the examination is, perforce, invalid because it eliminated applicants who were admittedly qualified.
The City defends its examination as content valid, claiming that it was designed to and did in fact measure knowledge, skills and abilities ("KSAs") required for the job of police lieutenant. Once having established that, the City contends that it need not explain why the test might have resulted in the failure to promote qualified applicants (a criterion-related method of proving validity that need not be employed once content validity is established).
Plaintiffs' argument that there was an available, equally valid, less discriminatory method of promotion that the City refused to use, relies on the City's own attempt to adopt a proposal to promote 20% of lieutenants based on a merit selection system rather than solely on the 1994 examination. The City argues (1) (without much conviction) that this method was not equally valid, and (2) (which much conviction) that even if it were equally valid, the addition of a merit selection component to promotions was unavailable because the state courts enjoined the City from using that method.
IV. THE FACTS
A. Agreed Facts
The following facts have been stipulated to by the parties, and, with certain stylistic changes, are hereby adopted by the court:
1. This case involves the 1994 lieutenants' promotional test prepared for the Chicago Police Department.
2. Plaintiffs are 44 African-American or Latino present or former members of the Chicago Police Department who hold the career service rank of sergeant and participated in the 1994 lieutenant promotional examination.
3. Plaintiffs have not been promoted to the rank of police lieutenant based on the results of the 1994 Chicago police lieutenant examination.
4. On March 28, 1995, plaintiffs filed a complaint in this action challenging the 1994 Chicago police lieutenant examination process under 42 U.S.C. § 1981 and 42 U.S.C. § 1983. On December 5, 1995, plaintiffs filed their First Amended Complaint, adding a Title VII disparate impact claim. On February 9, 1996 plaintiffs abandoned their 1981 and 1983 claims.
5. Plaintiffs contend that the test has a disparate impact upon African-American and Hispanic police sergeants and is an unlawful employment practice under Title VII of the Civil Rights Act of 1964 as amended, 42 U.S.C. § 2000e.
6. Plaintiffs do not claim that the City intentionally discriminated against plaintiffs or subjected them to disparate treatment.
7. Defendant City of Chicago concedes that the test had a disparate impact on African-American and Hispanic sergeants and contends that the test is job related and consistent with business necessity.
8. The 1994 lieutenants' test was taken by 765 police sergeants: 184 (or 24%) of those who took the test were African-American; 55 (or 7%) were Hispanic.
9. The City has made 108 promotions from the list. Six of those promotions (or slightly less than 6%) of those promotions have gone to minorities (5 of those promoted are African-American and 1 is Hispanic).
10. The parties agree that the statistical evidence makes out a prima facie case of disparate impact discrimination.
11. The 1994 promotional test consists of three subtests which shall be referred to in this order as follows: a written job knowledge test, an in-basket simulation, and an oral briefing exercise.
12. Scores on each component were combined to produce a final score.
*1100 13. Each component of the test had a disparate impact on minorities:
a. White sergeants received 193 of the top 207, 282 of the top 316, and 350 of the top 413 scores on the job knowledge test,
b. White sergeants received 181 of the top 203, 271 of the top 313, and 356 of the top 442 scores on the in-basket simulation.
c. White sergeants received 171 of the top 195, 303 of the top 383, and 412 of the top 540 scores on the oral briefing exercise.
d. White sergeants received 102 of the top 108 scores.
14. The parties agree that the statistical evidence makes out a prima facie case of disparate impact discrimination for each component of the 1994 lieutenant's test.
15. The CPD has promulgated a comprehensive set of departmental directives which establish CPD policy and procedure.
16. General Orders set forth CPD policy. A full set of the CPD General Orders that were in effect as of the date of the 1994 Chicago police lieutenant examination is contained in Defendant's Exhibit 7, which is in evidence.
17. Special Orders set forth CPD procedures. A full set of the CPD Special Orders that were in effect as of the date of the 1994 Chicago police lieutenant examination is contained in Defendant's Exhibit 8, which is in evidence.
18. Police actions are also governed by the criminal statutes of the State of Illinois and the City's Municipal Code.
19. All probationary officers are trained on the General Orders, Special Orders and Illinois criminal statutes upon hire.
20. There are four career service ranks in the CPD: police officer, sergeant, lieutenant and captain. Police officers are supervised by sergeants. Sergeants are supervised by lieutenants.
21. Lieutenants are promoted from the rank of sergeant.
22. At the time of trial, there were approximately 252 lieutenants in the CPD.
23. Prior to assuming the position of lieutenant, newly promoted lieutenants participate in a two-week pre-service training session. The training program exposes new lieutenants to management techniques, executive development issues and personnel issues.
24. The CPD is comprised of five bureaus: administrative, technical, staff services, operational services and investigative services. The operational services bureau is a field unit that includes the CPD Patrol Division.
25. The majority of newly promoted lieutenants are assigned to the Patrol Division.
26. District Law Enforcement, part of the CPD Patrol Division, is responsible for day-to-day law enforcement for the City of Chicago. It is comprised of 25 districts located throughout Chicago.
27. In each of the 25 districts, there is a district commander who is responsible for overall district operations.
28. Operations in each police district are divided into three hour shifts, called "watches." District operations during each watch are supervised by a "watch commander." The position of watch commander is held by an officer with the rank of lieutenant or captain.
Because the CPD is phasing out the rank of captain, the watch commander job is now performed more and more by lieutenants. There is no difference between what a captain does as a watch commander and what a lieutenant does as a watch commander.
29. As watch commander, the lieutenant is responsible for all activities that occur on a given shift in a district. These duties include overseeing the arrest, booking and bonding process. scheduling and assigning officers, conducting roll calls, inspecting personnel and equipment, conducting training, conducting facility inspections, supervising personnel assigned to district administrative activities, reviewing and approving all arrest reports and complaints, directing and reviewing disciplinary investigations of CPD personnel, monitoring the lockup, and personally responding to and assuming command at major *1101 criminal incidents occurring within the geographic confines of the district.
30. At the start of the shift, the watch commander meets with the watch commander from the previous shift to discuss any problems or issues that may be relevant to his or her tour of duty.
31. The watch commander reviews the commanding officer's book ("CO book") and any other paperwork or communications from the preceding watch commander, the district commander or any command member of the CPD to ascertain specific duties and responsibilities for his or her shift.
32. The watch commander inspects the district facility and the lockup facility to ensure the health and safety of all persons in CPD custody and to ensure compliance with applicable laws, rules and regulations. The watch commander also checks on the status of arrestee processing and the length of all detainments, and determines whether any prisoners remain who have yet to be released on bond.
33. The watch commander may review supervisor logs.
34. The watch commander may examine the manpower schedules. The watch commander is also responsible for adjusting work schedules in response to unanticipated absences.
35. Manpower scheduling is governed by specific CPD directives and by the collective bargaining agreement between the City and the police union, the Fraternal Order of Police ("FOP") (the "FOP contract").
36. There are at least two roll calls during each shift. During roll call, the officer conducting roll call conducts an inspection of all personnel. Once a week, the officer conducting roll call conducts a weapons and ammunition inspection of all personnel. During roll call, officers and supervisors receive their assignments for the shift. The officer conducting roll call reviews the CO book and notifies officers of court appearances, special work assignments or other items of interest contained therein. The officer conducting roll call also periodically provides roll call training during which training bulletins, new General Orders and Special Orders are reviewed and discussed with subordinates.
37. The watch commander also attends to various administrative duties.
38. The watch commander is responsible for reviewing and signing off on all overtime requests and authorizations.
39. The watch commander has the responsibility to review time due slips.
40. The watch commander conducts and/or oversees weapons discharge investigations concerning incidents in which a police officer has discharged his weapon or has been fired upon. The watch commander retains the responsibility for the investigation through any subsequent formal investigation if any person has been struck by a shot from an officer's weapon.
The watch commander is responsible for ensuring that the crime scene is protected, that proper notifications have been made, that injured persons have received medical attention, that all weapons discharge reports have been prepared, and that proper disciplinary procedures are implemented.
41. If an officer is injured on duty, the watch commander is required to review the injury-on-duty report that is prepared by the investigating supervisor.
42. The watch commander reviews activity reports prepared by subordinate officers.
43. The watch commander reviews case reports prepared by subordinate officers.
44. The watch commander reviews and approves summary punishment action requests ("SPAR") for subordinate officers. The SPAR system is an expedited disciplinary process for less serious transgressions by officers (e.g., traffic accidents, tardiness, failure to appear in court).
45. The watch commander reviews complaint register ("CR") investigations. CR investigations are initiated when officers are alleged to have committed more serious transgressions (e.g., use of excessive force, verbal abuse, failure to provide service), and may result in discipline of the officers up to and including discharge.
46. The watch commander reviews arrest reports prepared by subordinate officers.
*1102 Arrest reports are reviewed to ensure that they are complete, that charges are justified by the narratives provided by the officer, that there is probable cause to support the charges, that proper notifications have been made, and that arrestees who are eligible for bond are bonded out. Lieutenants must know the elements of crimes to make proper probable cause determinations.
47. The watch commander is responsible for ensuring that arrestee processing is done in compliance with CPD directives and orders.
48. The watch commander assigns officer counseling and other job duty assignments to sergeants.
49. The watch commander investigates incidents in which arrestees are hospitalized.
50. Throughout the shift, the watch commander monitors the activities in the district via the police radio. The watch commander is responsible for responding to and assuming command at major incidents (e.g., police shootings or hostage incidents).
51. The watch commander is required to tour the district periodically in a police vehicle to provide a command presence.
52. The watch commander supervises the personnel assigned to the district facilities, including all desk personnel, desk sergeants and watch secretaries.
53. The office facilities of watch commanders differ from district to district.
54. The watch commander's office is a focal point of many of the principal activities in the district. The watch commander's office is particularly busy at the change of shifts, when officers are getting their notifications.
55. While overall patrol operations for each district are supervised by a watch commander, field operations during each watch are supervised by field lieutenants. Field lieutenants directly supervise the sergeants in the field.
56. Field lieutenants are responsible for all field operations. They attend roll call, assist the watch commander in the inspection process and roll call training process, monitor, supervise and counsel sergeants and police officers, review forms and reports submitted by police officers and sergeants, ensure compliance with police policy and procedures while personally responding to major incidents in the field, investigate actions of police officers, conduct investigations of licensed premises, ensure overall compliance with the FOP contract and supervise check off roll call.
57. After roll call, the field lieutenant goes out into the district in an assigned police vehicle and supervises all field activities during the shift. Police vehicles are rotated among lieutenants from shift to shift. At the end of a shift, the field lieutenant turns the vehicle over to the lieutenant on the next shift who, in turn, turns the vehicle over to the lieutenant for the next shift.
58. Field lieutenants conduct investigations of weapons discharge incidents. As part of the investigation of an incident, the field lieutenant prepares a preliminary information weapons discharge worksheet and a weapons discharge report documenting the incident and investigation, and, if an officer is injured, an injury-on-duty report.
59. Field lieutenants may recommend summary punishment action requests for subordinate officers. Such recommending lieutenants investigate and submit SPARs, and review and approve SPARs initiated by subordinates.
60. Field lieutenants conduct CR investigations when a sergeant is the offending officer, and review CR investigations conducted by sergeants regarding police officers. The investigation is a formal due process investigation in which the accused has right to counsel. The formal investigation involves consideration of prior disciplinary and complimentary history and penalty recommendations. The investigation file includes a "Complaint Against Department Member" form and an "Administrative Proceeding Rights" form (in a non-criminal matter).
61. Field lieutenants conduct or ensure thorough investigations of licensed premises (e.g., taverns or other liquor establishments in which incidents occur).
62. Field lieutenants also respond to calls for police service. Lieutenants respond to the scene of major incidents (e.g., homicide) and are responsible to coordinate police activity at the scene, to ensure that proper *1103 notifications have been made, to ensure preservation of the crime scene, to ensure safety and health of all individuals involved and to provide necessary information to supervise officers.
63. Lieutenants ensure that complete investigations are conducted regarding CPD members who are suspected of intoxication and drug use.
64. Lieutenants review reports prepared by police officers and sergeants including, but not limited to, arrest reports, case reports, injury-on-duty reports, activity reports, information reports and time due slips.
65. Lieutenants review search warrants, complaints for search warrants, and search warrant data forms to determine whether there is sufficient probable cause to justify the warrant, to ensure that the warrant is accurate, complete, signed and properly planned, and to ensure that proper notifications have been made and that the warrant has been submitted to the State's Attorney's office and the court.
66. The field lieutenant does not have an office. Field lieutenants work out of the police car assigned to them for a particular shift.
67. Starting in 1992, the CPD developed and implemented a new policing strategy known as the Chicago Alternative Policing Strategy, or CAPS. The CAPS style of policing represented a departure from the CPD's incident-driven policing method, which focused on responding to service calls. The CAPS style relies upon a relationship between the CPD and members of the community to identify and solve problems in the community.
68. Under CAPS, police officers meet with community members to discuss problems of crime and disorder, prioritize the problems and develop strategies for addressing the problems. Based on these meetings, officers develop beat plans that chronicle specific criminal problems and strategies for response.
69. All 44 plaintiffs completed all three components of the 1994 Lieutenant examination. Their scores on each of the three components (the higher the number the better the score on each component) and their overall examination performance in relation to the 765 individuals who completed all three components of the examination (1 is the top scorer and 765 is lowest scorer) are as follows:
Name written oral in-basket final rank Brown, Ernest T 95 11 49 541 Bryant, Lloyd E 92 8 35 750 Butler, Desmond J 87 12 43 616 Byrd, Doris M 106 13 47 354 Carpenter, Reginald 57 13 37 719 Cole, Louis C 112 11 46 461 Crump, Henry E 96 12 33 670 Davis, Theodore R 105 7 35 740 Day, Sandra K 107 12 46 429 Eaglin, Paul R 90 8 38 740 Grayer, Mary A 106 15 44 258 Hardy, Judge F 111 14 35 448
*1104
Name written oral in-basket final rank
Hargrove, Larry 112 11 48 429
Harper, Juana J 97 14 51 282
Harris, Samuel E 89 12 44 587
Hicks, Eddie C 95 12 45 535
Jackson, Prentiss E 101 13 47 398
James, William M 92 12 36 662
Johnson, Forest L 89 9 35 735
Kimber, Vance T 99 12 47 475
Lewis Jr, Chris 112 9 44 611
Love, Robert A 113 12 36 543
Mastin, Barry 112 13 45 330
Mc Caster, Maurice L 107 10 38 657
Mial, Paul D 75 11 31 749
Moran Jr, Reyes P 82 13 31 699
Morgan, Cheriff A 82 13 44 576
Porter, Bernard 82 11 47 649
Raymond, Alberta M 118 13 44 298
Robinson, Holly C 117 10 56 316
Robinson, Yvonne 101 11 50 481
Rowland, Cisco D 110 11 45 489
Russell, Kevin 102 14 46 321
Shead, Brenda J 110 12 46 413
Shepherd, Eugene N 106 11 31 679
Silas, Nathan E 115 12 46 375
Steward, Dorothy M 102 13 36 602
Steward, Morris L 107 13 36 521
Thompson, Diane D 121 13 41 325
Tyler, Calvin E 97 12 36 642
Washington, Earl B 92 12 44 567
West, Gertrude 99 10 40 670
Wilindez, Joaquin 112 11 52 370
*1105
Name written oral in-basket final rank
Woods, Wanda R 93 9 43 697
V. ADDITIONAL FINDINGS AND CONCLUSIONS REGARDING TEST VALIDITY
A. Development, Design and Administration of the 1994 Chicago Police Lieutenant Examination
The 1994 Chicago Police Lieutenant Examination was developed by Barrett & Associates,[1] a human resources consulting firm specializing in employee selection, hiring and promotional testing. Barrett & Associates is headed by Dr. Gerald B. Barrett, who has a PhD in Psychology and a law degree, and who teaches courses in testing and measurement, personnel selection and performance evaluation, and personnel psychology and the law at the University of Akron. Dr. Barrett's curricula vitae was admitted into evidence as Defendant's Exhibit 2. Founded in 1973, Barrett & Associates has custom-developed more than 50 examinations for municipal police and fire departments. The court found Dr. Barrett to be a highly knowledgeable, qualified and credible witness.
Prior to developing the 1994 lieutenant examination, in 1993 Barrett & Associates developed a promotional examination for the position of Chicago police sergeant. In developing the 1993 sergeant examination, Barrett & Associates conducted a job analysis of the sergeant position. In that job analysis, Barrett & Associates interviewed approximately 90 incumbent sergeants about their duties and responsibilities in the CPD. Barrett & Associates also interviewed 28 lieutenants as part of the sergeant job analysis. In conjunction with subject matter experts from the CPD, Barrett & Associates developed a three-part sergeant examination comprised of a written job knowledge test, an in-basket simulation and an oral briefing exercise, the same format used for the 1994 lieutenant examination.
Barrett & Associates' "Report on the Development and Administration of a Promotion Process for the Job of Chicago Police Lieutenant" (received in evidence as Defendant's Exhibit 1) describes the actions taken to develop the examination. This description conforms to the testimony and other evidence, and will not be repeated in detail here. To summarize, Barrett & Associates conducted a job analysis to identify the major work behaviors and related knowledge, skills and abilities ("KSAs") for the job of Chicago police lieutenant by conducting extensive interviews with a broad spectrum of Chicago Police Department personnel (including diverse ranks and minority members), personal observation of the job and a review of all relevant departmental and organizational documents. Barrett & Associates personnel interacted and consulted with Chicago Police Department personnel to identify the major work behaviors and the materials that a Chicago police lieutenant must know to do his or her job. These materials include Chicago Police Department General and Special Orders, portions of the Illinois Criminal Code and Chicago Municipal Code, the FOP collective bargaining agreement, and a document describing the Chicago Police Department's Community Policing strategy ("CAPs").
In addition, Barrett & Associates used four Chicago Police Department subject matter experts[2] to review, comment on and criticize its work as it progressed. In addition to developing the test itself, this process resulted in the development and production of a Master Job Description (Def. Ex. 1, Appendix C) and, two weeks prior to the giving of the written test, a reading list for applicants to study (Def. Ex. 20). The court finds that *1106 the job analysis conducted by Barrett & Associates conformed to the standards of industrial psychology presented at the trial, as well as the EEOC Guidelines.
The evidence presented by the City demonstrated the following facts concerning the three components of the police lieutenant examination, which were designed to tap different elements of the job. The oral briefing exercise was utilized to measure the oral communication skills required on the job by lieutenants. The written job knowledge examination was utilized to measure the job knowledge required to perform the major work behaviors of the lieutenant job. The in-basket simulation was utilized to measure the administrative and organizational component of the job.
There were 1101 candidates eligible to take the test. The oral briefing exercise was administered to 915 candidates on June 1, 1994. Of the 915 candidates who participated in the oral briefing exercise, 765 chose to continue and complete the entire test battery. The written job knowledge test and in-basket simulation were administered to the 765 candidates on November 19, 1994.
1. Oral Briefing Exercise
The oral briefing exercise was designed to measure those job dimensions identified during the job analysis as vital to successful performance as a Chicago police lieutenant, such as analytical ability and organizational and oral communication skills. The exercise was designed to simulate a situation that could be encountered on the job by a Chicago police lieutenant. The candidate was to review information about a given problem (a new gang) and was directed to brief his or her subordinates on the problem. Each candidate was given explicit directions as to what type of oral briefing they were required to present. In the briefing package, each candidate was given 13 pages of police reports, Special Orders, and other materials in the same format as used by the CPD. The candidate was allotted twenty-five (25) minutes to review the materials and to prepare the presentation. After this preparation, the candidate was allowed a maximum of ten (10) minutes to present his or her briefing into three tape recorders.
The exercise utilized tape recorders, rather than live raters, allowing for the assessment of all eligible candidates in one day. Completing all testing in one day provided for the security of the assessment materials and ensured fairness of the assessment process. Exercise materials remained confidential and, thus, did not afford an unfair advantage to candidates who may have completed the exercise at a later time.
Tape recording also allowed the raters an opportunity to replay tapes if necessary. This increased the reliability of the scoring procedure. The use of tapes also allowed for blind rating, i.e., the raters did not know the identity, or the race, of the individuals they were rating. This minimized potential bias in the rating process.
Each candidate's presentation was rated in terms of characteristics including analysis, organization and oral communication by three trained raters, who scored the briefing according to an objective checklist. Def. Ex. 1, p. 107. All raters received diversity training. The raters also received specific training on how to rate the briefing tapes. The raters were trained on specific rating procedures and were provided practice in the application of the rating factors developed for each scoring dimension. In the training program, common rating errors were reviewed, as were strategies for avoiding errors. Each rater scored the exercises independently of the other raters. Then the three raters compared their ratings and any discrepancies were resolved. The raters reached consensus on one set of ratings for each candidate. If it was necessary in order to reach a consensus, the tapes could be played again for further review.
Chief Cadogan and Lt. Klein, subject matter experts from the CPD, reviewed the oral briefing exercise before it was administered. Chief Cadogan and Lt. Klein also reviewed the scoring checklists of important and relevant information to be utilized in rating the exercise. These subject matter experts evaluated the oral briefing exercise to ensure *1107 that it was fair, that it reduced the possibility of rater bias or favoritism, and that a sufficient amount of time was allotted for the completion of the exercise. The decision regarding the actual time allotted for the oral exercise was based on the past experience of Dr. Barrett and the feedback provided by the subject matter experts. The time allotted to candidates for review of the briefing materials in the exercise was longer than the time usually available to lieutenants before making such presentations on the job. The subject matter experts reviewed the oral exercise to ensure that it simulated subject matter and responsibilities that lieutenants encounter on the job. The subject matter experts made recommendations regarding the subject matter and format of the oral briefing exercise. Barrett & Associates made revisions to the oral briefing exercise based on their recommendations. Chief Cadogan and Lt. Klein determined that the types of skills required to respond to the oral component are the types of skills required for the job of lieutenant.
Prior to the actual use of the oral briefing exercise, Barrett & Associates pilot-tested the exercise by administering it to 29 graduate students. Because of confidentiality concerns, the pilot test could not be given to anyone who had contact with the City of Chicago or the CPD. The pilot test exercises were scored by trained raters at Barrett & Associates, the same raters who were used to rate the actual examination. The pilot test was conducted to ensure that Barrett & Associates' test administration procedures were not flawed and to determine the reliability of the rating process.
The oral briefing exercise was linked to the major work behaviors and responsibilities of a lieutenant, and to the KSAs set forth in the Master Job Description. A linkage chart documenting the linkage between testing dimension (e.g., analysis), the definition of the dimension, the specific ability being tapped, the related major work behaviors and the rationale was prepared by Barrett & Associates and is included in the Technical Report (Def. Ex. 1) at Appendix H. Every element of the oral briefing exercise was linked to major work behaviors and KSAs identified in the job analysis.
2. Written Job Knowledge Test
The written job knowledge component was a 150 question multiple-choice examination designed to test a candidate's knowledge of specific CPD General Orders, Special Orders, the Illinois Compiled Statutes, Chicago Municipal Ordinance, the FOP contract and the CAPS strategy. CPD lieutenants must know the General Orders, Special Orders, Illinois statutory law, the Municipal Code, FOP Contract and CAPS strategy to effectively perform the job of police lieutenant. The job knowledge test was designed to test the domain of job knowledge contained on the reading list.
Candidates were given two and one-half hours to complete the written job knowledge test. The multiple-choice format was selected to eliminate any grading bias. Candidates were instructed to select the one best answer to each question. The items on the written job knowledge component were drafted by members of the Barrett & Associates staff who had been trained previously in industrial psychology, item writing, and diversity. The test items were derived from the source materials and drafted so that the item stems (the questions themselves) reflected actual events encountered by lieutenants on the job, as indicated by the job analysis. The items on the written job knowledge test were written to sample the knowledge in the source materials identified during the job analysis as important for a lieutenant to know. The item writers relied on job analysis information to determine the number of questions to be included from various sources. The items on the written job knowledge component were designed to measure only the KSAs required for successful performance on the job as determined by the job analysis. Def. Ex. 1, p. 112. Each item on the written job knowledge test was explicitly linked to the major work behaviors and tasks identified in the Master Job Description. Each item on the written job knowledge test was linked to knowledge *1108 identified as important to successful performance on the job.
Barrett & Associates prepared a linkage chart which illustrates the linkage between the test items and the major work behaviors identified in the Master Job Description. A copy of the linkage chart is included in the Technical Report at (Def. Ex. 1) Appendix K. Appendix K demonstrates that each item on the written job knowledge examination tests for knowledge required for the job of lieutenant. The more important major work behaviors, as identified in the job analysis, were linked to a greater number of test questions. That is, more questions were included on the test to test the job knowledge of candidates regarding the more important major work behaviors.
Test items were written at a reading level that is lower than the reading level required to read the actual source materials on the job. Each item used on the written job knowledge test, including the stem and the five possible alternative answers, was reviewed by several item writers at Barrett & Associates, by the four subject matter experts from the CPD and by Dr. Barrett. Each item was reviewed against the original source documentation by the item writers and project supervisors from Barrett & Associates.
The written job knowledge examination was pilot tested. Ten graduate students took a sample test of 186 questions at Barrett & Associates' offices in Ohio. To ensure confidentiality and test security, each test taker signed a confidentiality agreement and the test was closely monitored in a secured room. Barrett & Associates used the results of the pilot test to eliminate inconsistencies and errors, and to refine the test items. Barrett & Associates also used the results of the pilot test to determine that the time allotted for the completion of the written job knowledge test was appropriate.
In order to ensure the relevance and accuracy of the questions on the written job knowledge component, each of the 150 written test questions, and several other potential test questions, were reviewed by CPD subject matter experts. The subject matter experts broke into groups, with each group teaming with a Barrett & Associates employee, and reviewed the stem and alternatives of hundreds of potential questions. The subject matter experts reviewed the test items to ensure that the items were derived from the source materials on the reading list published to candidates. The subject matter experts confirmed that the items on the written job knowledge test were derived from and related to the materials on the reading list. The subject matter experts reviewed the time allotted for the written job knowledge test to ensure that it was appropriate. The subject matter experts also reviewed the test items to ensure that the items utilized understandable language consistent with the language in the CPD directives, and to ensure they were related to the duties and responsibilities of a CPD lieutenant. The subject matter experts reviewed the alternative answers proposed for each test question to ensure there were no obviously implausible answers, to ensure that alternatives were accurate and to ensure that out of the five possible answers, two alternatives could not be correctly construed as the best answer. The subject matter experts recommended additions, deletions and modifications of several of the potential test items they reviewed. Barrett added, deleted or modified test items based on feedback from these CPD subject matter experts. The subject matter experts reviewed the universe of modified and updated test items a second time.
The analysis conducted by Barrett & Associates and the subject matter experts on every test item on the written job knowledge test, as well as those created but not used on the test, was documented by Barrett & Associates. A representative analysis is attached to the Technical Report at (Def. Ex. 1) Appendix L. Each item analysis documented the test item, the five possible multiple-choice answers, the correct answer, the source of the knowledge being tapped by the item, a rationale describing the major work behaviors being tapped by the item, a written justification for each multiple-choice answer indicating why the answer is right or wrong, the comments of the Barrett & Associates *1109 employees and subject matter experts who reviewed the items, the dates on which the items were reviewed, and the date and results of the pilot test of the item. The item analyzed in Appendix L was not used on the examination because it overlapped with another item. After the test questions were modified to comport with their recommendations, the CPD subject matter experts found that the written job knowledge test was related to the job of police lieutenant and that it tested knowledge that is important for a lieutenant to have.
The written job knowledge test tested for knowledge that a lieutenant must know in order to be effective on the job. The written job knowledge test was independently computer scored by Barrett & Associates and by Arthur Andersen. The scoring by Barrett & Associates and Arthur Andersen was compared and found to be identical.
3. In-basket Simulation
The in-basket simulation presented the candidates with a packet of information that included identical copies, or close approximations of documents and other items that a Chicago police lieutenant might come across on the job, e.g., General Orders, directives, memos, and reports. The in-basket simulation required candidates to perform several tasks that are actually performed by Chicago police lieutenants on the job, such as reviewing reports, investigating and disciplining subordinates and assigning tasks.
The in-basket exercise is a simulation of the lieutenant job; it is not intended to be an exact replica of the job. As a test, the simulation compressed several days or months of work into a short testing period to measure a candidate's administrative ability. The in-basket simulation required the candidates to step in and assume the duties and uncompleted tasks (e.g., review arrest reports) of a lieutenant who has become ill and incapacitated. This exercise closely simulated events that occur at the CPD; when lieutenants are unable to complete their work due to illness, reassignment or some other reason, other lieutenants are assigned to complete their unfinished work. Barrett & Associates prepared the in-basket simulation based on information collected from incumbent lieutenants regarding important issues and occurrences encountered in the performance of their work duties. Incumbents provided examples of actual situations that occurred on the job and offered a step-by-step explanation of how the situation was identified, specific actions taken, reasons for actions, how the situation was resolved, and procedures for documenting the incident.
Each candidate was given two hours and thirty minutes to review the set of materials and to take notes on those materials and the decisions they made regarding the issues presented in the materials. The candidate was then given 90 minutes to complete 60 multiple-choice questions based on the factual scenarios provided in the materials. The answer to each item in the in-basket simulation was contained in the accompanying in-basket materials. Candidates were allowed to refer to the materials and their notes while they were completing the multiple-choice questions. Simulation materials and questions were written at a reading level that is lower than the reading level required to read the actual forms and source materials on the job.
An objective, multiple-choice format was utilized in the in-basket simulation to eliminate grading bias and, thus, to increase test reliability. The research and literature in the field supports the use of a structured testing format such as the multiple-choice format used in the in-basket simulation. The use of a multiple-choice format increases the reliability of the simulation because it eliminates rater error and subjectivity. The use of a multiple-choice format does not diminish the validity of the in-basket simulation.
Dr. Barrett considered using live raters in the in-basket simulation rather than an objective multiple choice format. Dr. Barrett elected not to use that approach-referred to as "traditional in-basket exercise" by Dr. York-because that subjective approach is more often used as a teaching or developmental method, rather than as a selection method, to observe, critique and provide *1110 feedback to managers on their style of management.
The in-basket simulation was pilot tested. Graduate students participated in the simulation in a confidential and secure setting. Based on the results of the pilot test, the in-basket simulation was modified and fine-tuned. The materials and facts in the in-basket simulation were checked against the actual CPD documents to ensure that they did not contradict actual procedures of the CPD. CPD subject matter experts Cadogan, Klein, DeLopez and Shaw reviewed the in-basket component to ensure relevance and accuracy and to ensure that the materials used were consistent with the type of materials a lieutenant would confront on the job. Def. Ex. 1, p. 120. As with the written job knowledge test, the CPD subject matter experts broke into groups, teamed with Barrett & Associates employees, and reviewed the questions and materials from the in-basket simulation. The subject matter experts reviewed the items in the in-basket simulation to ensure that the items were derived from the source materials on the reading list published to candidates. The subject matter experts reviewed the test items to ensure that the items used understandable language consistent with the language in the CPD directives. The subject matter experts recommended additions to or deletions from the potential in-basket items and materials, which were incorporated by Barrett & Associates. After the recommended modifications were made, the subject matter experts reviewed the in-basket simulation a second time. The subject matter experts also reviewed the time allotted for the completion of the in-basket simulation. Based on their recommendations, the time allotted was increased and the complexity of the tested material was decreased. The subject matter experts found the in-basket component, as modified to reflect their recommendations, complete and appropriate. The subject matter experts found that the in-basket materials were very similar to the materials that a lieutenant would review in the course of his or her duties.
The in-basket simulation tested for knowledge, skills and abilities that a lieutenant must possess to be effective on the job. All items in the in-basket simulation were linked to major work behaviors and KSAs in the Master Job Description. A linkage chart documenting the testing dimension (e.g., application of rules and procedures), the definition of the dimension, the rationale, the related major work behaviors and the specific tasks in the simulation was prepared by Barrett & Associates and is included in the Technical Report (Def. Ex. 1) at Appendix N. The in-basket simulation was independently computer scored by Barrett & Associates and by Arthur Andersen. The scoring by Barrett & Associates and Arthur Andersen was compared and found to be identical. In order to maintain the confidentiality of the examination, candidates were not allowed to take copies of the test materials, answer sheets or notes out of the testing facility.
B. Conclusion: The 1994 Chicago Police Lieutenant Examination is Content Valid.
The court finds that the 1994 lieutenant examination, including each of its three components, is content valid because it measures a significant portion of the knowledge, skill and abilities of a Chicago police lieutenant. By utilizing three distinct types of testing procedure, the examination both approximated actual work behavior (the in-basket and oral briefing exercises) and tested objective knowledge that a police lieutenant must have to do his or her job. The City presented highly convincing evidence that the use of the multiple choice format for the in-basket and written knowledge components, and the equal weighting of each component, were valid methods.
All parties to this action agree on one thing: it was extremely unfortunate and unintended that minorities scored so poorly on the examination. As the City points out, however, the examination should not be judged content invalid based on the result that some qualified applicants did not get promoted. That inquiry is applicable to the criterion related method rather than the content related method of proving validity. Gillespie, 771 F.2d at 1040, n. 3.
*1111 The record in this case is clear that both the City and Barrett & Associates took extensive measures to avoid any kind of racial or ethnic bias in the development, administration and scoring of the examination. They identified and tested the applicants for the knowledge, skills and abilities needed to be a Chicago police lieutenant. Promotions were made in rank order to fill vacancies in the rank of lieutenant, a procedure that Dr. Barrett convincingly defended and that is approved by the EEOC Guidelines and industry standards in connection with a content valid test.
The City more than met its burden of proof of content validity, and plaintiffs failed to present convincing evidence to the contrary. Instead, plaintiffs identified certain test questions that were arguably graded improperly, certain conditions in taking a portion of the examination that were less than ideal, and certain flaws in the challenge procedures. In the context of the entire 1994 Chicago Police Lieutenant Examination, however, these imperfections were de minimus.
Plaintiffs also attack a number of individual questions in the written knowledge and in-basket portions, claiming that some of them did not test job knowledge that a lieutenant needed to know his first day on the job, that a lieutenant could easily look up the proper answers from relevant source materials on the job, and that the answers to certain other questions were incorrect. The court has reviewed these questions and the evidence concerning them, and finds that they did in fact test job knowledge required of a police lieutenant, that it would be impractical for a lieutenant to carry around with him or her all of the source materials needed to answer the various questions, and that the answers considered correct by Barrett & Associates were in fact correct.
VI. AVAILABILITY OF AN EQUALLY VALID LESS DISCRIMINATORY ALTERNATIVE
Having determined that the 1994 lieutenant's examination was valid, the burden shifts to plaintiff to establish that there was an available, equally valid, less discriminatory method for promotion that the City refused to use.
As the City acknowledges, once it received the test scores and determined that an adverse impact occurred, it was required to consider any equally valid, less discriminatory selection method that was available to ameliorate the disparate impact. 29 C.F.R. § 1607.3(B). The City found one alternative, using merit promotions in conjunction with the examination to promote 13 sergeants (minorities and non-minorities) to the rank of lieutenant (in addition to the 54 promotions that had at that time been made based on the examination) based on job related criteria. A panel comprised of CPD deputy superintendents and the commander of the training division reviewed nominations from CPD exempt rank personnel and recommended to the Superintendent who should be promoted.
The City's decision to promote 13 sergeant by merit selection resulted in a state court challenge by a non-minority candidate. That action, filed in the Circuit Court of Cook County on March 20, 1995 (eight days before the instant case was filed in this court on March 28, 1995), was based on provisions of the Chicago Municipal Code, which the plaintiff argued did not give the Commissioner of Police the power to use merit selection. The state court held a preliminary injunction hearing, at which the City sought to introduce evidence of the federal guidelines concerning employment examinations with an adverse impact. The plaintiff objected, and the judge prohibited any testimony on the subject, (although he did accept an offer of proof).
On March 29, 1995, the Circuit Court preliminarily enjoined the City from using any method of selection other than test rank order. The City appealed, arguing, among other things, that merit selection was necessary to alleviate the disparate impact of the 1994 examination. While the appeal was pending, the City was served with the complaint in the instant lawsuit, which it answered on May 30, 1995. On December 22, 1995, the Illinois Appellate Court, while praising the City's motives and without addressing *1112 the requirements of federal law, affirmed the injunction, holding that the City did not have the authority under state law to use merit selection. McArdle v. Rodriguez, 277 Ill.App.3d 365, 213 Ill. Dec. 709, 659 N.E.2d 1356 (1st Dist.1995).
As a result, although the City recognizes that the use of merit selection in conjunction with the examination was an equally valid, less discriminatory method, it maintains that the method was unavailable to it because of the injunction. It argues that Title VII does not authorize it to ignore a facially neutral, nondiscriminatory state law or a state court order enforcing that law. Plaintiffs agree that the use of the City's merit selection program in conjunction with the rank order promotions based on the 1994 examination is an equally valid, less discriminatory method of promotion.
This court disagrees with the City's position. Title VII contains a provision expressly preempting conflicting state law. 42 U.S.C. § 2000e-7 provides:
Nothing in this subchapter shall be deemed to exempt or relieve any person from any liability, duty, penalty, or punishment provided by any present or future law of any State or political subdivision of a state, other than any such law which purports to require or permit the doing of any act which would be an unlawful employment practice under this subchapter.
In the instant case, accepting as correct the state court's application of the Municipal Code, the resulting holding "requires" the City to violate Title VII. By passing the Civil Rights Act of 1964, and Section 2000e-7 in particular, Congress "intended to supercede all provisions of State law which require or permit the performance of an act which can be determined to constitute an unlawful employment practice under the terms of Title VII of the Act or are inconsistent with any of its purposes." Rinehart v. Westinghouse Elec. Corp., 1971 WL 174 (N.D.Ohio 1971). Under 29 C.F.R. § 1607.3(B), the City was required to use any available alternate selection procedure to ameliorate the adverse impact of the otherwise content valid test. The City knew that, and attempted to so argue to the state court. When the Illinois Appellate Court rejected the City's argument and upheld the injunction without considering the resulting violation of federal law, it interpreted an otherwise facially neutral Municipal Code provision in a manner inconsistent with the terms of Title VII. That interpretation was unconstitutional under the Supremacy Clause of the United States Constitution. Id. As held in Guardians Association v. Civil Service Commission, 630 F.2d 79, 104-105 (2d Cir. 1980), "Title VII explicitly relieves employers from any duty to observe a state hiring provision which purports to require or permit any discriminatory employment practice."
The City argues that once the injunction was affirmed, it was in a "damned if you do, damned if you don't" situation. It chose "don't," electing to argue to this court that the injunction proves that merit selection was unavailable. As held by the Seventh Circuit almost a quarter century ago, however, "the scheme of Title VII provides that employers are exempted from liability under state laws which require the doing of acts which constitute unlawful employment practices, not that reliance on state statutes resulting in discriminatory practices bars Title VII liability." Williams v. General Foods Corp., 492 F.2d 399, 404 (7th Cir.1974) (citations omitted).
Moreover, that untenable situation was entirely of the City's own making. As noted, the City was aware of its obligations under 29 C.F.R. § 1607.3(B) and that Title VII supercedes conflicting state law. It knew that it had an obligation to take whatever steps were necessary to implement the equally valid less discriminatory selection method. Therefore, once the state courts elected for whatever reason not to reach the issue, the City should have filed in the instant case either a counterclaim for a declaration that Title VII supercedes the Municipal Code, or (preferably) a third party claim against McArdle, bringing the issue and all parties squarely before this court to determine the City's obligations under federal law.
The City argues in a footnote that it could not have filed a declaratory judgment action *1113 prior to the initiation of any lawsuit claiming the results of the 1994 examination violated Title VII because no case or controversy existed, leaving the federal court without declaratory jurisdiction. This argument is to no avail because plaintiffs had already amended their complaint to include a Title VII claim prior to December 22, 1995, when the Illinois Appellate Court affirmed the preliminary injunction. Thus, an actual case or controversy existed at that time, which coincidentally, is the same date that the City filed its answer to the amended complaint. Therefore, once the injunction was affirmed, the City could have and should have brought the issue to this court, either as an affirmative defense or as a counter or third party claim for declaratory relief. It did neither, electing instead to ignore an equally valid less discriminatory selection method, thereby abrogating its obligations under 29 C.F.R. § 1607.3(B).
VII. RELIEF
Having determined that the 1994 lieutenant's examination was valid but that the City had available an equally valid less discriminatory method of promotion by combining rank order promotions with merit selection, the court must determine the relief, if any, to which these 44 plaintiffs are entitled. Unfortunately, the record is unclear in this regard.
The court is uncertain whether any of these 44 plaintiffs were among those selected for merit promotions or whether any of them have a cognizable argument that they were likely to have been so promoted. Accordingly, the parties are directed to address this issue on July 10, 1998, at 11:00 a.m.
NOTES
[1] The City retained Barrett & Associates to conform with the recommendation of a "blue ribbon panel" appointed by Mayor Richard M. Daley in 1990, that the City use outside consultants to design and administer police promotional examinations.
[2] These were Chief of Patrol John Cadogan and Lieutenant John Klein, who are white, Deputy Chief William Shaw, who is African-American, and Commander Joseph DeLopez, who is Hispanic.