Appellant Carl Davis,1 a black, is a disability determination examiner employed by appellee, State of Alabama Department of Education’s Disability Determination Service (“DDS”). He appeals from a judgment against him, and the class of similarly situated employees he represents (“the plaintiffs”), on their Title VII2 and section *128819813 claims 4 that DDS5 discriminated in its promotion practices and in selecting assistant supervisors. Recognizing that there were some flaws in the district court’s analysis, as well as the parties’ presentation, of the case, we affirm in part and reverse and remand in part.
Background
DDS is the state agency responsible for implementing the federal Social Security disability program in Alabama. DDS employees determine whether citizens’ applications for disability benefits should be granted. Each employee (with the exception of the Assistant Director and the Director) fills a position identified by a pay grade. The grades at issue here, in the order in which an employee would progress through them, are Examiner I (E-I), Examiner II (E-II), Examiner III (E-III, Supervisor I (S-I), Supervisor II (S — II), and Supervisor III (S-III). From 1976 to 1983, the only dates for which the record reveals workforce statistics, the workforce was 26.8% black.
In addition to the pay grade classification system, there are special job assignments, such as Assistant Unit Supervisor, Quality Assurance Specialist, and Vocational Specialist. These jobs do not carry increases in pay, but many employees believe they entail more desirable work. Because Davis makes distinct claims that DDS has discriminated against blacks in promotions, i.e., moving up through the pay grades, and in assignments to the position of Assistant Unit Supervisor, the procedures for promotions and job assignments will be discussed separately.
During the period this lawsuit covers, the promotions procedure was as follows. First, an employee had to work the positions in order (the “job-progression” requirement): one could not, for example, become an E-III from an E-I position. Second, there were certain minimum experience requirements: to become an E-II, an employee had to serve one year as an E-I; to become an E-III, an employee had to serve one year as an E-II; to become an S-I, an employee had to have permanent status6 as an E-III and “at least one year’s supervisory experience in disability determination”; and to become an S — II, an employee had to have permanent status as an S-I. Third, the employee had to apply for a higher position. Applications were solicited and accepted only during a period specified by a job announcement. These *1289announcements were not posted regularly.7 When a vacancy or a need for additional positions arose, the Director of DDS, Dr. John Shelton, would contact the Personnel Board of the State of Alabama (“SPD”), tell SPD he wanted to fill a position, and SPD would distribute the announcement. Applications were sent directly to SPD; no one at DDS knew who had and had not applied.
After receiving applications, SPD would develop a “register” for the particular pay grade. The register listed everyone who had applied and was eligible in order of what SPD determined their qualifications to be. The applicants were ranked by SPD on the basis of three criteria. Forty-five percent of the ranking was based on the applicant’s training in disability determination, college and post-graduate degrees and the like. Another 45% was derived from the applicant’s experience, the length of time an applicant had worked in disability determination and the positions in which he or she had served. These two factors combined were referred to as the training and experience (“T & E”) score. The final 10% of an applicant’s ranking depended on his or her evaluations by supervisors while working at DDS. An applicant’s scores on the last three evaluations were averaged to arrive at the 10% factor.
After the applicants were ranked, there was a further culling process: the highest ranking applicants were placed on a list called the “Certificate of Eligibles” (“CE”), or “certified.” Only applicants who made it onto the CE were actually interviewed for the position they sought. The number of persons on the CE varied with the number of positions to be filled. For the first position open, ten names had to appear on the CE. For every position thereafter, one additional name had to appear. Due to a court order issued in United States v. Frazer, 317 F.Supp. 1079, 1081 (M.D.Ala.1970), blacks who were next in line on the CE could not be “passed over,” although white applicants could be. ^
For most of the period this lawsuit covers, 1977-1982, the process of becoming an Assistant Unit Supervisor (AUS) at DDS was considerably less formal than the promotion process. There might or might not be a job announcement, and thus employees might or might not file a formal application. Employees at the E-I, E-II, and E-III levels apparently were all eligible for the job. In most instances, the unit supervisor, the area supervisor, and the Assistant Director simply selected the employee they thought best suited for the position. One unit supervisor, however, indicated that he was essentially told by Shelton, the director, who would be chosen to assist him.
In 1982, a reclassification study conducted by SPD merged the AUS position into the promotion system. From then on, only E-III’s were eligible to serve as an AUS, and employees were selected for the position just as they were for any other E-III position. Several of the employees serving as AUS’s at the time the position was reclassified were automatically given E-III status.
At trial, Davis relied on the testimony of several class members as well as an expert who presented several statistical studies. DDS also relied on the testimony of an expert, as well as Director Shelton, Personnel Director James Methvin, and several of the class members’ immediate supervisors. The district court, sitting without a jury, entered findings of fact and concluded that the plaintiffs had failed, on both of their claims, either to make out a prima facie ease or to prove that DDS’ various actions were not justified. The district court also found that the state merit system used by DDS was “a permissible use of the seniority system.” Final Judgment Order at 2 (Mar. 27, 1987).
I. DDS’ Promotion Practices
With respect to the plaintiff’s challenge to DDS’ promotion practices, the district court ruled that they had failed either *1290to make out a prima facie case or to “carry the burden of proof at the third stage of the presentation of evidence.” Final Judgment Order at 2 (Mar. 27,1987). The ambiguity of this statement appears to reflect the district court’s efforts to integrate its disparate treatment and disparate impact analyses. Because we believe that the law remains in sharper focus if those analyses are not confused, we address them separately.
A. The Allegation of Disparate Treatment
Whenever a plaintiff proceeds upon a theory of disparate treatment, he or she must establish by a preponderance of the evidence that the employer acted with discriminatory intent. Texas Department of Community Affairs v. Burdine, 450 U.S. 248, 253, 101 S.Ct. 1089, 1093, 67 L.Ed.2d 207 (1981); Hill v. Metropolitan Atlanta Rapid Transit Authority, 841 F.2d 1533, 1538 (1988). If the case is fully tried, the court generally should not refer to the various “stages of proof,”8 for the only issue at that point should be whether the plaintiff proved the employer’s discriminatory intent. United States v. Postal Service Board of Governors v. Aikens, 460 U.S. 711, 716, 103 S.Ct. 1478, 1481-82, 75 L.Ed. 2d 403 (1983); Burdine, 450 U.S. at 255 n. 10, 101 S.Ct. at 1095 n. 10. Whether the employer acted with such intent is a purely factual question subject to reversal only if the fact finder was clearly in error. Pullman-Standard v. Swint, 456 U.S. 273, 288-89, 102 S.Ct. 1781, 1789, 72 L.Ed.2d 66 (1982).
Applying these principles to the district court’s opinion, we are troubled by the court’s references to the plaintiffs’ “prima facie case” and “third stage” burden of proof, and to DDS’s “articulated legitimate nondiscriminatory reasons.” Even so, on reading the opinion as a whole, we are convinced that the court reached its conclusion that DDS acted without discriminatory intent without relying excessively on the tripartite evidentiary structure. It is clear from the court’s discussion that it believed there were significant flaws in the plaintiffs’ statistical evidence and that it credited the testimony of the DDS personnel over that of the class members. We therefore proceed to determine whether the district court’s finding was clearly erroneous.
The plaintiffs’ evidence at trial was both statistical and anecdotal. Their expert, a biostatistics professor at the University of Alabama, presented a study indicating that of all the promotions awarded, whites received a disproportionate number of the higher-level promotions and blacks a disproportionate number of the lower-level ones. See Record, PI. Exh. 23. Several plaintiffs testified that white employees hired concurrently with or after them had received promotions faster. See, e.g., id., Vol. Ill at 101-02, 283. Several plaintiffs also testified that their supervisors had stated that the Director, Dr. Shelton, had instructed the supervisors to lower certain class members’ evaluations scores. See, e.g., id., Vol. II at 38, Vol. Ill at 280, Vol. IV at 344. Carl Davis insisted that his supervisor had made this statement six years in a row. See id., Vol. II at 38. A woman named Lovie Marable related that Shelton had attempted to have her sign a release stating that she would pass up the E-III position for which she was next in line. See id., Vol. Ill at 294.
DDS responded to most of these allegations. A consultant versed in employment discrimination statistics testified that the plaintiffs' statistical study could not be relied upon because it did not address whether blacks had applied or been certified for the promotions at issue. See id., Vol. VI at 781-84. The Personnel Director, James Methvin, testified that only three whites were hired concurrently with or after Davis and yet promoted earlier, and explained that two of these had several years of experience prior to joining DDS and the last was working in the Mobile office where the competition presumably was not *1291as stiff.9 See id. at 691-93. Methvin also testified that there were 25 whites and 2 blacks who had been employed longer than Davis and were only E-III’s and that there were 6 whites employed longer than Davis who were still at the E-II level. See id. at 694. The supervisors charged with stating that Dr. Shelton had instructed them to lower blacks’ evaluation scores denied both making such statements or being so instructed. See id., Vol. V at 492, 576, 671. Dr. Shelton himself admitted that he had, on one occasion, questioned Davis’ supervisor about a score that, because of Davis’ backlog of cases, seemed too high, but he denied that he had generally instructed supervisors to lower the evaluation scores of blacks. See id. at 621. With respect to Lovie Marable, Shelton admitted that he had asked her to sign a release, and in fact had had similar conversations with two other blacks he did not think were “ready” for the positions they were in line for. See id. at 614, 664. DDS sought to justify this by presenting evidence of Marable’s poor performance on the job10 and explaining that DDS was under a court order preventing it from passing over blacks appearing on the CE, see id. at 613, whereas whites were passed over on 81 different occasions for reasons similar to those inspiring Shelton’s request for a release. Id., Vol. VI at 705.
Given the evidence from both sides, we cannot say that the district court clearly erred in finding that DDS acted without discriminatory intent in awarding promotions. Although the plaintiffs’ evidence, if believed in its entirety, would give rise to an inference of discriminatory intent, the district court was entitled to credit those witnesses it found most believable and accept one version of the events over another. See Anderson v. Bessemer City, 470 U.S. 564, 574-75, 105 S.Ct. 1504, 1511-12, 84 L.Ed.2d 518 (1985) (trial judge’s finding based on credibility choice between witnesses each telling internally consistent story uncontradicted by extrinsic evidence can “virtually never be clear error”); United States v. Yellow Cab Co., 338 U.S. 338, 342, 70 S.Ct. 177, 179, 94 L.Ed.2d 150 (1949). This is true with regard not only to anecdotal evidence but to expert testimony as well. See Middleton v. Dan River, Inc., 834 F.2d 903, 910 (11th Cir.1987) (finding absence of clear error where court believed one expert over another). Moreover, even if the district court had determined to credit the plaintiffs’ statistical evidence, that evidence standing alone would have been an insufficient basis on which to find discriminatory intent. See Carmichael v. Birmingham Saw Works, 738 F.2d 1126, 1131 (11th Cir.1984). From the evidence placed before it, the district court concluded that black DDS employees were not intentionally treated any differently from similarly situated white employees with respect to promotions. The court found that the lower evaluations of several of the class members resulted from deficiencies in their work performance and were not the product of any discriminatory intent of the DDS management. There was substantial, internally consistent evidence to support these conclusions. We thus affirm the district court’s finding that the plaintiffs failed to prove their claim of disparate treatment in promotions.
B. The Claim of Disparate Impact
As noted above, to prevail on a disparate treatment theory, a plaintiff must prove that his or her employer intentionally discriminated in some aspect to the plaintiff’s employment. A disparate impact theory is considerably different in that it does not require a showing of any animus on the employer’s part. See Dothard v. Rawlinson, 433 U.S. 321, 328, 97 S.Ct. 2720, 2726, 53 L.Ed.2d 786 (1977); Griggs v. Duke Power Co., 401 U.S. 424, 430-32, 91 S.Ct. 849, 853-54, 28 L.Ed.2d 158 (1971). It must be shown, however, that a “facially neu*1292tral” employment practice falls more harshly on one group than another. The burden then rests on the employer to show that whatever the unfortunate effect of the practice, the practice bears a “manifest relation” to the job in question, New York City Transit Authority v. Beazer, 440 U.S. 568, 587 n. 31, 99 S.Ct. 1355, 1366 n. 31, 59 L.Ed.2d 587 (1979), and is thus justifiable as a business necessity.11 Albermarle Paper Co. v. Moody, 422 U.S. 405, 425 (1977) (employer must “meet the burden of proving that its tests are ‘job related’ ”); Griggs, 401 U.S. at 431, 91 S.Ct. at 853 (“Congress has placed on the employer the burden of showing that any given requirement [bears] a manifest relationship to the employment”).
As the Supreme Court has recognized, some class claims may be such that they are cognizable under both a disparate treatment and a disparate impact theory. See International Brotherhood of Teamsters v. United States, 431 U.S. 324, 335 n. 15, 97 S.Ct. 1843, 1854 n. 15, 52 L.Ed.2d 396 (1977) (“Either theory may, of course, be applied to a particular set of facts.”); see also Griffin v. Carlin, 755 F.2d 1516, 1525 (11th Cir.1985) (authorizing challenge to promotion process under disparate impact as well as disparate treatment theories). The plaintiffs’ challenge to DDS promotions is just such a claim. The plaintiffs’ evidence consisted of allegations of intentional discrimination combined with statistics and testimony intended to show that something about DDS's promotion procedures, facially neutral though they w’ere, was operating to deny black employees promotions that similarly situated whites were getting. Thus, to the extent that intentional discrimination was alleged, the plaintiffs’ promotion claim is (and has been) properly analyzed as a disparate treatment claim. Yet to the extent that the plaintiffs alleged that DDS’s facially neutral promotion procedures operated to exclude them, the promotions claim is properly analyzed under a disparate impact theory.
1. The Practices Properly at Issue
Before we proceed to the merits of the plaintiffs’ impact claim, we think it crucial to clarify precisely what practices are properly subject to that impact challenge. DDS and SPD argue vigorously that the plaintiffs are attempting to retry the case on appeal insofar as they are complaining of any promotion practices12 other than the evaluations by DDS supervisors. Their objection is two-fold. They argue first that the plaintiffs did not present any evidence on the additional practices and never indicated that these practices were part of their promotions claim. In support of this assertion, they cite a post-judgment certificate issued by the district court in accord with Federal Rule of Appellate Procedures 10(e) that reflects the district court’s understanding that the promotions claim involved only the evaluations. They also point to the “proposed facts” submitted by the plaintiffs prior to trial, which the defendants say reflects the plaintiffs’ limited focus. The defendants also insist that the plaintiffs were not entitled to challenge several of the practices to which they now object because SPD was in the case from the beginning only as a "necessary party” under Federal Rule of Civil Procedure 19. In other words, the requirements of working the jobs in order, the minimum-experience requirements, the T & E scoring procedure, and the development of registers and CE’s were all exclusively in SPD’s domain, and the plaintiffs would have had to name SPD *1293as a full party defendant if these practices were to come properly into issue. In support of this assertion, the defendants point to a colloquy in the record where the plaintiffs’ counsel appears to acknowledge SPD’s Rule 19 status.
In reviewing the defendants’ arguments, it has become clear to us that they misapprehend the respective burdens in a disparate impact case. Whatever impression they and, apparently, the district court may have been under, plaintiffs alleging class-wide discrimination in promotions are not required to isolate the particular aspect of the promotional process that is responsible for the discriminatory impact. As this court held in Griffin v. Carlin, 755 F.2d 1516 (11th Cir.1985), some nine months before the trial here began, the purpose of Title VII
“is not well served by a requirement that the plaintiff in every case pinpoint at the outset the employment practices that cause an observed disparity between those who appear to be comparably qualified. Such a requirement in effect permits challenges only to readily perceptible barriers; it allows subtle barriers to continue to work their discriminatory effects, and thereby thwarts the crucial national purpose that Congress sought to effectuate in Title VII.”
Id. at 1529 (quoting Segar v. Smith, 738 F.2d 1249, 1271-72 (D.C.Cir.1984), cert. denied, 471 U.S. 1115, 105 S.Ct. 2357, 86 L.Ed.2d 258 (1985)). Thus, to make out a prima facie case under a disparate impact theory, plaintiffs need only put on evidence of a statistically significant disparity between promotions of blacks and similarly situated whites. It is then the employer’s responsibility to identify the various practices resulting in the promotion decisions and to justify as job-related whichever practices have actually caused the racial disparity.13 See id. at 1527-28.
Because this is currently, and was at the time of trial, the evidentiary framework under which cases alleging disparate impact proceed, DDS and SPD cannot be heard to complain that they thought the promotions challenge was directed only at the supervisory evaluations, or “service ratings.” It is true that the plaintiffs’ proposed facts read as if they believed that the perceived discrimination resulted primarily from the disproportionately low evaluation scores blacks received,14 but the defendants should have been aware that they would not be able to explain away any disparate impact of the promotion process as a whole simply because the plaintiffs alleged that the most discriminatory practice was the evaluations.
Moreover, much of the trial discussion of the additional practices they now claim are not properly in the case came from the defendants themselves. For in addition to challenging the significance of the plaintiffs’ statistics, and instead of simply proving why the subjective evaluations were necessary, the defendants chose to argue that any disparate impact of the promotions process resulted not from the evaluations but from the disparity in blacks’ and whites’ lengths of service, the primary factor in arriving at the 45% “experience” score.15 On this point, we turn again to *1294Griffin. In Griffin, as here, the plaintiffs alleged that the employer had discriminated in awarding promotions. The district court found for the employer, noting that the plaintiffs’ statistics were not meaningful because they did not address whether the black employees not promoted had made it onto a register similar to that involved here. This court reversed, and held that the defendants could not, on remand, rely on the practices leading to placement on the register unless those practices themselves were proven free from discriminatory impact. See Griffin, 755 F.2d at 1526-28; see also Watkins v. Scott Paper Co., 530 F.2d 1159, 1192-93 (5th Cir.) (experience requirement could not justify disparate impact where “[p]ast discriminatory practices have either prevented or discouraged many of [defendant’s] employees from transferring to many lines of progression and from gaining the experience [the defendant] deems necessary in a supervisor”), cert. denied, 429 U.S. 861, 97 S.Ct. 163, 50 L.Ed.2d 139 (1976); Stevenson v. International Paper Co., 516 F.2d 103, 117 (5th Cir.1975) (lack of experience in a particular line of progression does not justify disparity in appointments to supervisor where blacks were excluded from those lines of progression). This case presents an analogous situation in that the defendants here also sought to rebut the allegations of discriminatory impact by relying on practices that themselves might have been discriminatory. Thus, even if the additional practices were not originally in the case, they became subject to the plaintiffs’ challenge when the defendants relied on them to rebut the allegations concerning the evaluations.
The defendants’ argument concerning SPD’s status does not convince us otherwise, for two reasons. First, we are not at all certain, as a factual matter, that SPD was only a Rule 19 defendant. There is no mention of Rule 19 in plaintiffs’ motion for leave to amend the complaint, in SPD’s answer, in the order granting leave to amend, in the pretrial order, or in the final order. Indeed, the district court’s memorandum opinion recounts that “the court allowed the plaintiff to amend his complaint and add the Personnel Board of the State of Alabama as a party defendant.” Memorandum Opinion at 2 (Mar. 27, 1987) (emphasis added). In addition, the exchange relied on so heavily by the defendant 16 falls far short of an “admission” that SPD was only a Rule 19 defendant because it is impossible to tell whether plaintiffs’ counsel ever heard the comment to which he allegedly was agreeing. Under these circumstances, we are hesitant even to conclude that SPD was in the case only under Rule 19, much less to conclude that SPD’s status is an adequate basis on which to drastically limit the plaintiff’s case.
Second, and more importantly, we hold that an employer cannot delegate several aspects of its promotion procedure to another agency such as SPD and then escape liability if that agency develops discriminatory practices. Title VII’s remedial purpose requires that an employer be held accountable for practices it, in effect, adopts if those practices have a class-wide discriminatory impact. To hold otherwise would run contrary to cases holding employers responsible for the use of discriminatory tests developed and administered by an outside agency. See, e.g., James v. Stockham Valves & Fittings Co., 559 F.2d *1295310, 319, 339-40 (5th Cir.1977) (employer could be held liable for test developed by consultant), cert. denied, 434 U.S. 1034, 98 S.Ct. 767, 54 L.Ed.2d 781 (1978). It would also conflict with this court’s recognition that plaintiffs often are not privy to the information necessary to pinpoint both the precise practices and parties responsible for the discriminatory impact. See Griffin, 755 F.2d at 1527-28; see also Evans v. Sheraton Park Hotel, 503 F.2d 177, 183 (D.C.Cir.1977).
In sum, the defendants’ reliance on practices other than the evaluations brought those practices properly into issue, and, in fact, this is how Title VII cases were intended to proceed. For only when all of an employer’s practices are squarely faced and examined can Title VIPs purpose of rooting out discrimination be served.
2. The Merits
In attempting to show that DDS’ promotion practices had a disparate impact on black employees, the plaintiffs introduced statistics indicating that “the higher the job level to be promoted into, the less likely it is to go to a black.” Record, Vol. Ill at 190. In addition to presenting raw numbers in support of that thesis, the plaintiffs’ expert undertook both a standard deviation and a chi-square analysis of DDS promotions from 1977-1986. Using as his expectation benchmark the ratio of all blacks promoted to all employees promoted, which turned out to be 21.19%, the expert derived the following figures:
Job Promoted Total No. Into Occurrences Percent Black Occurrences Diff. From Expectation Standard Deviations
E-I 31 48.39 + 27.20 3.7
E-II 118 22.88 + 1.69 0.45
E-III 52 11.54 -9.65 1.7
S-I 21 9.52 -11.67 1.31
S-II/S-III 14 0.00 -21.19 1.94
Id., Plaintiffs’ Exh. 23. The chi-square measure resulting from these figures was, according to plaintiffs’ expert, 22.31, indicating that there was only a 2 in 10,000 chance that race and the level to which an employee was promoted were not related. The expert asserted that the chi-square measure was more likely to predict the “overall trend” than the standard deviation analysis.
The plaintiffs also put on evidence indicating that black employees were considerably more likely to receive low evaluation scores than white employees were. Using an expectation benchmark similar to that used for his promotion analysis — the ratio of all evaluations of blacks to all evaluations, or 24.47% — the expert computed the' standard deviations and the chi-square measure for the five17 possible evaluation scores. The results of the standard deviation analysis were as follows:
Standard Deviation Evaluation Ranking Total No. Occurrences Percent Black • Occurences Diff. From Expectation
2.61 1 (lowest) 10 60.00 35.53
4.46 2 28 60.71 36.24
8.46 3 199 50.25 25.78
2.08 4 681 27.90 3.43
9.23 5 (highest) 496 6.65 -17.82
*1296Id., Vol. Ill at 204-09. The chi-square measure indicated that there was only a 1 in 10,000 chance that race and the evaluation score were not related.18
The defendants responded with expert testimony and statistical evidence of their own. The defendants’ expert was critical of the plaintiffs’ expert for failing to account for several factors other than discrimination that might account for the disparity in black representation in the higher-paying jobs. See id., Vol. VI at 781-84. Chief among these, in the expert’s view, was length of service. An exhibit he had compiled indicated that job level and length of service were directly correlated. See id., Def. Exh. 47, Tab 2. Also significant was the percentage of blacks in the “feeder jobs” directly beneath each higher-level job. In other words, it was specious to expect blacks to comprise 21.19% of promotions to the S-I position if there were not that many blacks in the E-III position. A study the expert presented, using what allegedly was the “proper” benchmark, the percentage in the feeder jobs, indicated that blacks were getting a proportional share of the actual promotions. See id., Tab 4. A third factor the expert deemed important was the representation of blacks on the CE’s. An analysis employing that benchmark revealed that blacks were actually faring better than whites once the former made it onto the CE. See id., Tab 3.
With respect to the evaluations, the defendants’ expert did not quarrel with the fact that blacks’ evaluations were consistently lower. He focused instead on the fact that the evaluations had not, under a standard deviation analysis, resulted in a statistically significant disparity in blacks’ rankings on the register. See id., Vol. VI at 801. On rebuttal, the plaintiffs’ expert testified to the contrary. A standard deviation analysis, he claimed, was insufficient to show anything because the numbers in each group were small. A chi-square analysis, however, would make up for the fact that the pools were small, and it showed that the evaluations had a statistically significant effect on certification. See id., Vol. VII at 840, 843.
After sifting through all of this evidence, the district court found that “[DDS’ promotion] system did not produce an adverse impact on any class or individual.” Memorandum Opinion at 25 (Mar. 27, 1987). Relying on Maddox v. Claytor, 764 F.2d 1539 (11th Cir.1985), the court agreed with the defendants that the plaintiffs’ impact statistics were flawed because they did not expressly exclude blacks who had not applied or been certified for the higher-paying jobs. The court also found that to the extent that the “experience” factor was causing a disparity in the presence of blacks and whites in the higher-paying positions, it was “a permissible use of the seniority system.” Memorandum Opinion at 26 (Mar. 27, 1987).
We can accept neither of these holdings. Under the circumstances of the case, it was improper for the district court to recognize any kind of “seniority system” defense. This court has held that a seniority system defense under section 703(h) of Title VII, see 42 U.S.C. § 2000e-2(h) (1982), is an affirmative defense that must be expressly pleaded and included in the pretrial order to be viable. See, e.g., Jackson v. Seaboard Coast Line R. Co., 678 F.2d 992, 1012-13 (11th Cir.1982). The defendants failed to plead the defense and to have it included in the pretrial order, and in fact disavowed any reliance on the defense when the plaintiffs sought to clarify the *1297defendants’ position.19 See Record, Vol. VI at 804. Any such defense, therefore, was waived, and the district court should have recognized this.
Similarly, the district court’s emphasis on certification was misplaced. It is true that the plaintiffs’ statistical comparisons did not distinguish applicants or certified employees from the professional workforce in general. It is also true that in Maddox v. Claytor, 764 F.2d at 1539, this court discredited certain of the plaintiffs’ statistical studies for this very failure. See id. at 1550. In Maddox, however, it was undisputed that virtually all of the jobs to which blacks complained they were denied promotions were highly skilled positions. See id. at 1542 (“the Base lacked the resources to train its workers for [the white-collar] ICP positions”); id. at 1550 (the plaintiffs complained that they did not receive “a great number of positions calling for highly specialized skills”). Here, the positions to which the plaintiffs seek equal access were not “highly skilled” jobs, at least not in the sense that the typical E-I could not fulfill the requirements of a typical E-III. There was testimony, for example, that all of the examiners in a given unit took on the caseloads of other examiners when those examiners went on vacation or were pulled for work on special assignments. See Record, Vol. Ill at 285, 298, 327. There was also testimony from one of the defendants’ witnesses, a DDS supervisor, that
there’s not a lot of training going on. Everybody who’s been there has been working there from five to ten years, usually. And they’re pretty well conversant with how they’re supposed to do cases. So there’s very little training that you refer to that’s required. The primary training that’s done is in the QA (Quality Assistance, a particular unit) procedures. That is how you write up the case for return, this sort of thing. It’s rather mundane as far as anything complicated or foreign or anything like this. Everybody is conversant with what goes on. It’s just a matter of getting into the routine.
Id., Vol. V at 513.
In short, while we agree with the general proposition in Maddox that statistical comparisons offered in support of a disparate impact claim should be drawn as narrowly as possible to avoid the effects of factors other than discriminatory practices, see also Hazelwood School District v. United States, 433 U.S. 299, 308, 97 S.Ct. 2736, 2742, 53 L.Ed.2d 768 (1977), we cannot agree that the plaintiffs’ case failed because their statistics were not limited to those blacks certified. From the only evidence in the record on the point, it appears that an uncertified black serving as an E-I was just as likely to be qualified for an E-III position as a certified white serving in an E-II position, even if DDS’ procedures did not acknowledge that fact. Furthermore, the plaintiffs would have proven nothing if they had compared the rates of promotion for only those certified. For even if the only issue were the supervisory evaluations, as the defendants insisted, the plaintiffs were attempting to prove that discrimination was occurring in the certification process itself. By requiring that the plaintiffs limit their comparison pool to only those blacks who had been certified, the district court simply begged the question. See Jackson, 678 F.2d at 1016-17 (“the Brotherhood’s primary position ... that the appellees were not discriminated against ... because they had not satisfied the promotion requirements ... merely begs the question whether or not the promotion provision was justifiable as a business necessity”).
Whether the plaintiffs should have limited their statistical comparison to include only those blacks who had applied is a more difficult issue. On the one hand, it is conceivable that the disparity in black and white representation in the higher-pay*1298ing positions resulted from blacks’ failure to apply in numbers proportional to their presence in the DDS workforce.20 The plaintiffs thus might have adopted a benchmark for comparison purposes reflecting only those blacks who had applied. Cf. Williams v. Tallahassee Motors, Inc., 607 F.2d 689, 692 (5th Cir.1979) (plaintiff did not make out prima facie ease of disparate impact by comparing percentage of blacks hired with percentage of blacks in geographic area), cert. denied, 449 U.S. 858, 101 S.Ct. 159, 66 L.Ed.2d 74 (1980). On the other hand, had the plaintiffs done so, they effectively would have conceded that the requirement of working the jobs in order and the requirements that an employee spend a certain amount of time in a “feeder” position were non-discriminatory. For comparing the promotions of only those whites and blacks who applied would necessarily have excluded blacks who might well have been qualified but who learned from the eligibility requirements that an application would be futile. Cf. International Brotherhood of Teamsters v. United States, 431 U.S. 324, 367, 97 S.Ct. 1843, 1871, 52 L.Ed.2d 396 (1977) (plaintiffs should not be denied relief solely because discriminatory practices have been “so successful as totally to deter applications from members of minority groups”); Taylor v. Hudson Pulp & Paper Corp., 788 F.2d 1455, 1462 (11th Cir.1986) (non-applicant may make out prima facie case by showing application would have been futile), cert. denied, — U.S. -, 108 S.Ct. 345, 98 L.Ed.2d 371 (1987).
With this dilemma in mind, we conclude that the plaintiffs failed to raise a sufficient challenge to those practices that would not have automatically rendered blacks ineligible; specifically, the use of the service ratings, and the evaluation of an applicant’s training and experience. With regard to these practices, applicant statistics were available, and the plaintiffs should have employed them. With respect to the other practices, however — specifically, the requirement that the jobs be worked in order and the requirement that an applicant for promotion have served a certain amount of time in a “feeder” position — the evidence was sufficient to shift the burden to the defendants to prove that those practices were sufficiently job-related to justify their discriminatory impact. The plaintiffs’ statistics showed that blacks were underrepresented in the higher-paying positions,21 and the defendants acknowledged the disparity. Then, rather than breaking *1299the promotion process into its various components and explaining why each component was job-related, the defendants put on evidence that the disparity was caused by the fact that blacks as a group had not been with the agency as long as whites and thus had not had the same opportunity to work up through the ranks. This might have sufficed as an explanation if it were not entirely possible, in fact likely, that racial discrimination itself was responsible for the difference in blacks’ and whites’ lengths of service.22 As it was, instead of rebutting the plaintiffs’ case, the defendants bolstered the idea that certain requirements were perpetuating past discrimination, which, if true, would violate Title VII. See Giles v. Ireland, 742 F.2d 1366, 1380 (11th Cir.1984); Jepsen v. Florida Board of Regents, 610 F.2d 1379, 1383 (5th Cir.1980).
Under these peculiar circumstances, we think it most prudent to remand the case for further proceedings. Although in most instances, findings that a prima facie case of disparate impact has been made and that a defendant failed to respond with a showing of business necessity would warrant a judgment for the plaintiffs, see, e.g., Jackson, 678 F.2d at 1016-17, it is painfully clear to us that everyone involved in this case was laboring under serious misconceptions concerning how the case would proceed.23 We thus do not believe it would bring us any closer to the truth to end the case without offering the parties an opportunity to respond directly to our holdings today. The case should be retried so that the plaintiffs can more clearly show, with their own evidence, that the job-progression and minimum experience requirements perpetuated past discrimination and the defendants either that the requirements did not have such an effect or that they were necessary for the agency to operate. The parties should be aware, however, that those issues are the only promotion issues still in the case, and the only ones as to which evidence should be admitted.
II. The Selection of Assistant Unit Supervisors
In addition to challenging DDS’ promotion practices, the plaintiffs alleged below that DDS had discriminated against blacks in its appointments to the AUS positions. SPD is not implicated by this claim because the AUS appointments were handled entirely by DDS officials. In support of this contention, the plaintiffs again offered both statistical and anecdotal evidence. An exhibit prepared from numbers supplied by DDS indicated that of 36 appointments between 1977 and 1986, only 6 went to blacks. See Record, Pl.Exh. 11. Several plaintiffs testified that they had met with the Director, Dr. Shelton, in 1978, to complain that they were being denied the opportunity to serve as AUS’s, but that he had refused to identify the criteria for the AUS selections and had essentially dismissed them. See id., Vol. Ill at 105-07, 281-82. Finally, on cross-examination of the Personnel Director, the plaintiffs adduced evidence that the evaluation scores of black AUS’s were consistently lower than those of white AUS’s. See id., Vol. VI at 762.
DDS responded to this evidence with statistical evidence purporting to show that any disparity in the appointment of blacks was not statistically significant. See id., Def. Exh. 47, Tab 5 at 1-2. Comparing the *1300percentage of blacks in each of the jobs “feeding” the AUS position to those actually appointed,24 DDS’ expert concluded that the selections were not discriminatory. DDS also relied on supervisors’ testimony that many of the plaintiffs had not performed well enough to be appointed to the position. See, e.g., id., Vol. V at 506-07, 520-21, 542-43.
Unfortunately, all this evidence from both sides went for naught. For despite having recognized in the class certification that the promotion claim and the AUS claim were independent of one another, the district court made only one statement that can possibly be construed as a finding with respect to the AUS appointments: “Although [the plaintiffs] complained that they had been denied access to the position of AUS because the position was not posted and they had no knowledge of vacancies until the positions were filled, neither was denied access to a better job because of not having filled that position.” Memorandum Opinion at 23 (Mar. 27, 1987) (emphasis added). It seems rather clear to us from this statement that the district court thought that discrimination claims were not cognizable under Title VII unless they alleged some direct economic detriment. This conclusion was incorrect: Title VII prohibits discrimination not only in compensation, but in the “terms, conditions, or privileges of employment.” 42 U.S.C. § 2000e-2(a)(l) (1982); see also James v. Stockham Valves & Fittings Co., 559 F.2d at 333; Swint v. Pullman-Standard, 539 F.2d 77, 93 (5th Cir.1976).
As a result of this error, and the district court’s consequent failure to provide any reviewable findings, we have no choice but to remand this issue along with the remaining promotion issues. On remand, the district court should, on the evidence already presented, consider whether the plaintiffs showed that DDS intentionally discriminated against blacks in selecting AUS’s. The district court should also consider, on the evidence presented, whether the plaintiffs proved that the subjective selection procedure was having a discriminatory impact on blacks seeking the appointments, and if so, whether the subjective selection procedure was necessary to DDS’ operations. See Watson v. Fort Worth Bank, 108 S.Ct. at 2786 (approving disparate impact analysis for subjective employment practices).
III. Davis’ Individual Claim
Wholly apart from the class claims, Carl Davis alleged below that he was denied reclassification to an E-III position in 1982 on account of his race. According to Davis, SPD’s 1981 reclassification, effective 1982, gave all AUS’s automatic E-III status, and although he was working in 1981 as an AUS, he retained his E-II status. According to DDS, Director Shelton submitted three different requests for Davis’ reclassification to E-III, but SPD denied them because, unlike the other AUS’s who were reclassified, Davis had not yet served a year as an E-II and was thus not eligible to move to E-III status.
The district court accepted the defendants’ explanation for Davis’ treatment and found that neither agency had intentionally discriminated against Davis in failing to reclassify him. We cannot say that this finding was clearly erroneous. There was uncontradicted evidence that Shelton had sought to have Davis reclassified and that all of the other AUS’s reclassified had more than a year of experience as E-II’s. We therefore affirm the district court’s holding with respect to Davis’ individual, disparate treatment claim.
Conclusion
The district court’s judgment on the plaintiffs’ claim that DDS and SPD intentionally discriminated against blacks in awarding promotions is affirmed. The district court’s judgment on Carl Davis’ individual claim is also affirmed.
With respect to the plaintiffs’ allegation that DDS’ promotion practices had a dis*1301parate impact on black employees, and their claim that the AUS appointments were discriminatory, the district court’s judgment is reversed and remanded for further proceedings consistent with this opinion.
AFFIRMED in part, REVERSED and REMANDED in part.
. Larry Powers was also a plaintiff when the suit was filed, but his claim was voluntarily dismissed with prejudice from the case on October 8, 1982.
. In relevant part, Title VII provides that “[i]t shall be an unlawful employment practice for an employer — (1) ... to discriminate against *1288any individual with respect to his compensation, terms, conditions, or privileges of employment, because of such individual’s race.” 42 U.S.C. § 2000e-2(a)(l) (1982).
. In relevant part, section 1981 provides that ‘[a]ll persons within the jurisdiction of the United States shall have the same right in every State and Territory to make and enforce contracts, ... and to the full and equal benefit of all laws and proceedings for the security of persons and property as is enjoyed by white citizens.” 42 U.S.C. § 1981 (1982).
. Employment discrimination claims may be brought either under Title VII or section 1981. This court has held, however, that a section 1981 plaintiff must prove that an employer acted with discriminatory intent. See Stallworth v. Shuler, 777 F.2d 1431, 1433 (11th Cir.1985). Under Title VII, a plaintiff may succeed even without such a showing of intent if he proves that a facially neutral practice has a discriminatory impact on his racial group and the defendant does not prove that the practice is strictly job-related. See Connecticut v. Teal, 457 U.S. 440, 446, 102 S.Ct. 2525, 2530, 73 L.Ed.2d 130 (1982); Dothard v. Rawlinson, 433 U.S. 321, 324, 97 S.Ct. 2720, 2726-27, 53 L.Ed.2d 786 (1977). The reader should thus be aware that all references to disparate treatment implicate both the plaintiffs' section 1981 claims and their Title VII claims, while the references to disparate impact implicate only Title VII.
. Because it administers several of the promotion procedures challenged by the plaintiffs, the Personnel Board of the State of Alabama (“SPD”) is also a defendant/appellee. DDS & SPD argue that SPD has been in the case from the very beginning only as a necessary party under Fed.R.Civ.P. 19, and that to the extent Davis now challenges procedures implemented exclusively by SPD, he has expanded the case well beyond the basis on which it was tried. This dispute will be addressed later in the opinion. See infra part I.A.1.
. The record does not reveal the requirements for "permanent" status or how such status was distinguished from any other kind. Neither party has asserted that the distinction is meaningful to this lawsuit.
. According to the Director of DDS Personnel, James Methvin, there was no maximum time between job postings, but there was a minimum period. At least one year had to pass before a new job announcement was posted.
. See McDonnell-Douglas Corp. v. Green, 411 U.S. 792, 805, 93 S.Ct. 1817, 1825, 36 L.Ed.2d 668 (1973) (discussing the evidentiary framework for Title VII cases in terms of three stages of proof: the prima facie case, the rebuttal, and the showing of pretext).
. Davis was working in the Birmingham DDS office.
. Two of Marable’s supervisors testified that she committed an excessive number of technical errors in her work and that her average caseload and time to complete cases was about 30% higher than the average for all employees. See Record, Vol. V at 520-21, 576. James Methvin testified that in the year Marable was asked to sign the release, she had taken leave time 50 out of 52 weeks. See id., Vol. VI at 705.
. We are aware that four members of the Supreme Court recently have indicated that the burden of proof on the absence of business necessity rests with the plaintiff. See Watson v. Fort Worth Bank & Trust, — U.S. -, 108 S.Ct. 2777, 2790, 101 L.Ed.2d 827 (1988). A plurality opinion, however, is not binding precedent, and in the meantime, we are bound by several decisions of this court (as well as the Supreme Court cases cited in the text) stating flatly that the employer bears the burden of proving that a practice is job-related. See, e.g., Craig v. Alabama State University, 804 F.2d 682, 689 (11th Cir.1986); Crawford v. Western Electric Co., 745 F.2d 1373, 1384 (11th Cir.1984).
. The defendants do not assert that the plaintiffs have misrepresented Davis' individual claim or the claim involving the selection of AUS’s.
. Griffin placed this burden on the employer for sound policy reasons. Employees are unlikely to be aware of every method by which their employer selects persons for promotion, and yet the employer will be fully aware of what went into its decision. Again, we are aware that four Supreme Court Justices have stated that it is the plaintiffs responsibility to identify at the outset all of the practices he or she is challenging, see Watson, 108 S.Ct. at 2788, but until this view is accepted by the full Court, we are bound to follow Griffin. See supra note 11.
. For example, at one point in the pleading the plaintiffs propose to prove that “[t]here is a direct correlation between the disparity in blacks’ being promoted into the upper level jobs and the service ratings they receive from a predominately white supervisory force.” Supp. Record, Vol. I, Tab B at 9 ("Proposed facts by Plaintiff"). No similar statements concerning any of the other practices are made, although the practices themselves are described.
.This argument was undoubtedly the reason that the district court included a finding that “[t]he Alabama State Merit System utilized by the DDS in its promotional policies is a permissible use of the seniority system, and its utilization does not discriminate against any class or *1294group of people." Final Judgment Order at 2 (Mar. 27, 1987).
. The defendants cite the following passage from the record:
[SPD ATTORNEY]: Judge, does the Court understand and has understood [sic] all along that we’re in for Rule 19 purposes?
THE COURT: I thought you made that clear.
[SPD ATTORNEY]: I just wanted to make sure where we were.
THE COURT: I didn’t hear any objection to that statement.
[PLAINTIFFS’ ATTORNEY]: Your Honor, could I ask or beg the Court’s indulgence for your position regarding second depositions generally, for future reference?
THE COURT: I'm not going to give you any advice when I don’t have to.
[PLAINTIFFS' ATTORNEY]: Fair enough.
(Proceedings concluded at 11:35 a.m.)
Supp.Record, Vol. I, Tab A at 20-21. As indicated, this is the only reference to Rule 19 appearing anywhere in the record.
. Since three different ratings systems were used during the period covered by the lawsuit, the expert found it necessary to correlate certain ratings used under one system with those under another. Because two of the ratings system had five possible scores, the defendants did not particularly object to correlating scores under those systems. The defendants did object, however, to the expert’s equating scores under the one four-score system with those under the five-score system. See Record, Vol. Ill at 256. The expert explained that it seemed most reasonable to group the two lowest scores under the four-score system with the two lowest under the five-score system, and the two highest under the four-score system with the two highest under the five-score system.
. In order to ensure that the statistics were not skewed because they were based on evaluation occurrence rather than on the scores given particular persons, the plaintiffs expert also performed statistical analyses based on the highest and lowest scores black employees had ever received. The expectation benchmark for these studies was the ratio of blacks evaluated to the total number of employees evaluated. The chi-square under both these studies confirmed that reached when the evaluations were studied by their occurrence: under either approach, there was only a 1 in 10,000 chance that race and evaluation score were not related. See Record, Plaintiffs’ Exhs. 28, 29.
. It appears the defendants were willing to waive the defense because they mistakenly believed that DDS' reliance on "experience" was not properly an issue in the case. See supra part I.A.1. Hence, they thought they could assert seniority as the reason for the disparity without its being the subject of an independent disparate impact challenge, and there was no need to rely on it as a “defense.”
. The defendants insist that their statistics showed that blacks did not suffer any discriminatory impact if they did in fact apply. Their Exhibit 31 purports to show that blacks made up 20.7% of the applicants for the higher-paying positions and yet 24.4% of those actually promoted. We find this exhibit virtually worthless. First, it deals only with the E-II, E-III, and S-I positions, even though the S-II and S-III positions were also at issue. Second, the data from which the summary exhibit was drawn appears to come from randomly selected years: the E-II numbers come from 1981 and 1983; the E-III numbers from March and September 1981; and the S-I numbers from 1978 and 1982. It may be that these dates correspond to the dates on which the jobs were posted, but the record does not indicate that the dates are anything but random, and the fact that the E-II numbers are drawn from two months in the same year conflicts with testimony that a year had to pass between announcements. Third, the total number of applicants listed is about 200 persons lower than the total number of persons certified appearing on yet another of the defendants’ exhibits, a practical impossibility if employees had to apply to be certified. Compare Record, Def.Exh. 31 (referring to 188 total applicants) with id., Def.Exh. 30 (referring to 377 employees certified).
. We are aware that the plaintiffs’ statistical evidence, as reproduced above, showed that the standard deviations from one particular level to the next were not legally significant, i.e., in none of the categories were there more than two standard deviations. See Castaneda v. Partida, 430 U.S. 482, 496 n. 17, 97 S.Ct. 1272, 1281 n. 17, 15 L.Ed.2d 498 (1977) (“As a general rule ..., if the difference between the expected value and the observed number is greater than two or three standard deviations, then the hypothesis that the [results were] random would be suspect to a social scientist.”). This does not mean, however, that the plaintiffs’ statistical studies were invalid. Several courts have approved chi-square analysis as an alternative to standard deviation analysis, and the analysis did serve the limited purpose the plaintiffs claimed for it: it showed that blacks were much less likely to be in the higher-paying positions. See, e.g., Coates v. Johnson & Johnson, 756 F.2d 524 (7th Cir.1985); Capad v. Katz & Besthoff, Inc., 711 F.2d 647 (5th Cir.1983).
. The Personnel Director testified that there were no blacks working for DDS in the 1960’s, and that from 1970-1974, only four blacks were hired. See Record, Vol. VI at 731-32. There was also testimony that SPD was operating under a court order entered in United States v. Frazer, 317 F.Supp. 1079 (M.D.Ala.1970). The court in Frazer found that the State of Alabama had discriminated in its hiring and promotion practices. See id. at 1087-88.
. The plaintiffs apparently were unaware that the benchmark used for their statistical comparisons had to be drawn as narrowly as possible in order to be considered probative of disparate impact, and the defendants apparently were unaware that practices other than the supervisory evaluations were legitimately in the case. It seems clear that if the defendants had been so aware, they would not have offered the statistics on length of service, and we would be faced directly with determining whether the plaintiffs’ statistics alone were sufficient to make out their prima facie case.
. We cannot help but note that this benchmark assumes that certain AUS positions were reserved for certain pay grades, an assumption that does not appear to be supported by the record.