Rossini v. Ogilvy & Mather, Inc.

615 F. Supp. 1520 (1985)

Carlotta ROSSINI and Jane Zukofsky on behalf of themselves and all persons similarly situated, Plaintiffs,
v.
OGILVY & MATHER, INC., Defendant.

No. 78 Civ. 1713.

United States District Court, S.D. New York.

August 27, 1985.

Vladeck, Waldman, Elias & Engelhard, P.C., New York City, for plaintiffs; Judith P. Vladeck, Joseph J. Garcia, Anne C. Vladeck, Laura S. Schnell, New York City, of counsel.

Davis & Gilbert, New York City, for defendant; Patricia Hatry, Howard J. Rubin, New York City, of counsel.

OPINION

GAGLIARDI, Senior District Judge.

Plaintiffs commenced this action against Ogilvy & Mather, Inc. ("O & M"), an advertising agency, alleging employment discrimination based on sex in violation of Title VII of the Civil Rights Act of 1964, 42 U.S.C. § 2000e et seq. ("Title VII"). The case was tried without a jury from October 3, 1983, through November 15, 1983. In its opinion, Rossini v. Ogilvy & Mather, Inc., 597 F. Supp. 1120 (S.D.N.Y.1984), the court discussed but did not rule on the claim of salary discrimination. This decision constitutes the court's findings of fact and conclusions of law, pursuant to Rule 52(a), Fed.R.Civ.P., on the salary discrimination claim.

Background

The court reserved decision on the claim of salary discrimination to allow revision of the relevant trial exhibits to incorporate previously omitted data on O & M "officers." Such revisions became necessary when the court found erroneous a prior ruling that precluded plaintiffs' discovery of the personnel files of O & M's officers.^[1] The court directed that the additional evidence would be received only to permit modification of existing exhibits to reflect the addition of officer data.

In order that neither side can take advantage of the initial erroneous discovery ruling, the court directs that, to the extent possible, the regression studies including officers replicate those based solely upon non-officers. In particular, neither side is to use this reopening of the evidence as an opportunity to add or delete contested independent variables or to alter questioned data used for non-officers.

Rossini v. Ogilvy & Mather, Inc., supra, 597 F.Supp. at 1167.

Despite this clear instruction, both sides have chosen to act as if the court were continuing trial of the salary discrimination claim. Both have unnecessarily submitted exhibits which do not comply with the court's order. The defendant has introduced studies which add variables to previous tables. Plaintiffs have submitted tables which incorporate data specifically excluded at trial. The studies that do not conform to the court's order accordingly are excluded from consideration.^[2]

The evidence introduced at trial on the issue of salary discrimination is summarized in the November 1984 opinion, and will not be repeated here. In addition, the court now has before it plaintiffs' direct report (ex. P-1-A) ("plaintiffs' 1985 report"), plaintiffs' reply report (ex. P-1-B) ("plaintiffs' 1985 reply"), defendant's direct report (ex. D-703) ("defendant's 1985 report), and defendant's reply report (ex. D-704) ("defendant's 1985 reply"), all of which incorporate the officer data. The parties have also submitted briefs addressing the results of the 1985 reports and a stipulation filed June 24, 1985, "in lieu of further expert testimony."^[3]

Discussion

I. O & M's Objections to Plaintiffs' Studies

A. Relevance of the Statute of Limitations

O & M argues that little weight should be given to the salary studies that include employees hired prior to the limitations date of May 30, 1975. It is true that primary consideration should be given to studies that reflect only O & M's employment decisions after that date, i.e., regression analyses of salaries of employees hired on or after May 30, 1975, or of salary increases for all O & M employees after that date. See Sobel v. Yeshiva University, 566 F. Supp. 1166 (S.D.N.Y.1983); Melani v. Board of Higher Education of the City of New York, 561 F. Supp. 769 (S.D.N. Y.1983) (hereinafter Melani v. Board). Some consideration must be given, however, to studies of salaries of all O & M employees, even though inclusion of those hired before May 30, 1975, means that the data necessarily reflect employment decisions both preceding and following the limitations date.

Such studies may provide background evidence relevant to the claim of actionable salary discrimination within the limitations period. See United Airlines v. Evans, 431 U.S. 553, 97 S. Ct. 1885, 52 L. Ed. 2d 571 (1977). Moreover, where, as here, a pattern and practice of discrimination is alleged, pre-limitations date employment decisions may be actionable as part of a continuing violation of Title VII which continued into the limitations period. See Melani v. Board, supra, 561 F.Supp. at 780-81.

O & M asserts that pre-limitations date salary decisions should serve neither as background evidence nor as proof of a pattern and practice of salary discrimination, since in 1976 O & M changed its procedures for setting salaries. The memoranda prepared and testimony given by Frances Devereux, who became director of personnel at that time, indicate that the female presence among the salary decision-makers was increased, but do not necessarily establish that the procedures followed after 1976 altered prior practice in other ways. Before 1976, salary decisions were made by the executive committee of the board of directors, an almost exclusively male group of ten people. By 1976, salary decisions were being made by a smaller salary committee composed of Devereux; Bill Phillips, a top O & M official; and either Jules Fine, for review of media department salaries, or Reva Korda, for review of creative department salaries.^[4]

Statistical evidence, however, supports O & M's position that salary practices differed for the pre- and post-May 30, 1975, periods. Plaintiffs prepared a Chow test to assess whether there are statistically significant differences between the regression models arrived at for salaries of pre-limitations date hires and those calculated for post-limitations date hires. Plaintiffs' amended Chow test (incorporating officer data) indicates statistically significant differences between regressions for personnel hired before and after the limitations date.^[5] Thus, the pre-May 30, 1975, salary decisions provide relatively little useful background evidence and little indication of a continuing employment practice.

This does not mean that the court should disregard entirely salary studies which cover all O & M employees, however. Plaintiffs have argued that all post-limitations date salaries are relevant to the issue of whether actionable salary discrimination exists, even when they reflect pre-limitations date employment decisions. As the District of Columbia Court of Appeals stated in Valentino v. United States Postal Service, 674 F.2d 56, 71 n. 26 (D.C.Cir. 1982):

Statistics tuned to the proper time period are more probative than statistics not so tuned, but categorical rejection of the latter is not warranted. See EEOC v. American Nat'l Bank, 652 F.2d 1176, 1188-89 (4th Cir.1981); Movement for Opportunity and Equality v. General Motors Corp., 622 F.2d 1235, 1244-45 (7th Cir.1980); Vuyanich [v. Rebublic National Bank of Dallas], 505 F.Supp. [224] at 360 [(N.D.Tex.1980)]; B. Schlei & P. Grossman, supra, at 326-27. See also Hazelwood, [School Dist. v. U.S.], 433 U.S. [299] at 308 n. 13, 97 S.Ct. [2736] at 2742 n. 13 [53 L. Ed. 2d 768 (1977)].

Further, the weight of authority holds that where pay differentials are based exclusively on gender, each discriminatory paycheck constitutes in effect a new violation of Title VII. See Bartelt v. Berlitz School of Languages of America, 698 F.2d 1003, 1004-05 (9th Cir.1983), cert. denied, ___ U.S. ___, 104 S. Ct. 277, 78 L. Ed. 2d 257 (1984); Hall v. Ledex, Inc., 669 F.2d 397, 398-99 (6th Cir.1982); Higgins v. State of Oklahoma ex. rel. Okl. Emp. Sec., 642 F.2d 1199, 1200 n. 2 (10th Cir.1981); Boyd v. Madison County Mutual Insurance Co., 653 F.2d 1173, 1176-77 (7th Cir.1981), cert. denied, 454 U.S. 1146, 102 S. Ct. 1008, 71 L. Ed. 2d 299 (1982); Parker v. Baltimore & Ohio Railroad Co., 555 F. Supp. 1177, 1182 (D.D.C.1983); Licari v. National Maritime Union, 21 Fair Empl.Prac. Cas. 1638 (S.D.N.Y.1979); but see Sobel v. Yeshiva University, supra, 566 F.Supp. at 1188 n. 57.

The Court of Appeals for the Second Circuit in Title VII class actions has examined together salaries of employees hired before and after the relevant limitations date, but has carefully scrutinized any showing of disparity that results from the inclusion of pre-limitations date hires. In Coser v. Moore, 739 F.2d 746 (2d Cir.1984), the court had before it regression studies indicating no gender-based disparity in raises after the limitations date or in gross salaries of employees hired after the limitations date. Rather than simply find no discrimination on the basis of those studies, the court looked at the broader studies, which "arguably show[ed] that defendants discriminated with respect to salary before [the limitations date] and that a residual effect of that discrimination continue[d] to exist." Id. at 754. The court then found, however, that the pre-limitations hires were "a very small group" and that any salary difference attributable to them disappeared when certain "unique categories of employees" were excluded.

Such careful scrutiny is appropriate here, where the statute of limitations properly should insulate from review certain salary differences which result from employment decisions made prior to the limitations date. Consider the hypothetical case of similarly qualified male and female account executives who both earned $15,000 in April, 1975. On May 1, 1975 (prior to the limitations date), the man, solely on the basis of his gender, was promoted to account supervisor, a position with somewhat more responsibilities, and given a raise in recognition of those responsibilities. The gender-based promotion decision is not a continuing violation of Title VII. Assuming no pattern and practice of discrimination, the promotion is protected from review by the statute of limitations. In addition, the change in responsibilities which results is a legitimate basis for a post-May 20, 1975, salary differential. Cf. Ste. Marie v. Eastern Railroad Association, 650 F.2d 395, 400-02 (2d Cir.1981) (no violation of Title VII where apparent salary discrimination resulted from disproportionate appointment of men to technical and managerial positions prior to limitations date). Since the legitimacy of the salary discrepancy derives from the differences in job duties, one way to screen out any substantial effect from alleged discrimination which is no longer actionable may be to control for job title (as a measure of job duties) in studies which include employees hired before the limitations date.

In sum, the court will give some consideration to the regression studies that cover employees hired before May 30, 1975. Those studies, however, will be evaluated with special care so that the court does not treat as evidence of actionable discrimination salary differences resulting from pre-limitations date employment decisions that themselves do not constitute continuing violations.

B. Constructed Data for Age at Highest Degree

O & M argues that plaintiffs inaccurately constructed data in a manner which seriously undermines their regression analyses. For many of their studies,^[6] plaintiffs' expert utilized an independent variable controlling for an O & M employee's experience as calculated from a presumed age at completion of formal education. For example, in some cases plaintiffs' expert assigned as the age at highest degree age 18 for high school graduates, age 20 for junior college or technical school graduates, age 22 for college graduates and age 24 for those with graduate degrees. In certain other cases, age 18 was assigned as the age at highest degree for anyone who did not graduate from college.

O & M established at trial that it had previously transmitted to plaintiffs personnel files indicating the actual dates of graduation for many of the O & M employees for whom plaintiffs had constructed such data. For example, studies in the 1983 "yellow report" (plaintiffs' 1983 submission entitled "Revised Tables to Report and Reply Report of Donald E. Wise") that included 449 O & M employees utilized constructed data on experience for 168 employees whose personnel files indicated their actual dates of graduation. For those 168, plaintiffs' constructed information was erroneous in 120 (approximately 72%) of the cases. Plaintiffs' 1983 Reply Report used constructed data for 239 employees, 79 of whom had their actual date of graduation indicated in their personnel files. In approximately 82% of the cases in which such information was available, the constructed date of graduation differed from the real date.

Where errors in the data base are "egregious," the value of studies based on the data is severely limited. Such gross errors render the studies suspect even in the absence of a showing that the mistakes are "systematic," thereby affecting the "accuracy of the signal of whether or not group [race or gender] status is being taken into account by the employer...." Vuyanich v. Republic National Bank of Dallas, 505 F. Supp. 224, 306 (N.D.Tex.1980); cf. Melani v. Board, 561 F.Supp. at 780 (missing data for 20% of employees in regression analysis not of a magnitude to undermine plaintiffs' prima facie showing). In those instances where plaintiffs had access to the employee's actual date of graduation, the error rate in the constructed data approaches the egregious. There is no basis for an inference that the data constructed for the remaining employees were any more accurate.

Moreover, this is not an instance in which "`plaintiffs cannot legitimately be faulted for gaps in their statistical analysis [because] the information necessary to close those gaps was possessed only by defendants and was not furnished either to plaintiffs or to the Court.'" Trout v. Lehman, 702 F.2d 1094, 1102-03 (D.C.Cir.1983), vacated on other grounds, ___ U.S. ___, 104 S. Ct. 1404, 79 L. Ed. 2d 732 (1984), quoting Trout v. Hidalgo, 517 F. Supp. 873, 883 (D.C.1981); cf. Melani v. Board, supra, 561 F.Supp. at 780 n. 18 (where defendant stopped collecting relevant employee data during the course of Title VII litigation, plaintiffs' studies not flawed by failure to include that information). Here, in many instances, defendant had provided plaintiffs with accurate information prior to trial and there was no need for constructed data.^[7]

The court concludes that errors in the constructed graduation data seriously undermine the value of plaintiffs' Tables 1 through 4, and Tables VI-1 and VI-2.^[8] The problems with the constructed data also limit the weight to be accorded to Table 6.^[9]

C. Omission of Employees Departing in 1975, 1976, or 1977

O & M argues that plaintiffs' data base is seriously flawed because it omits all professional employees (other than officers) who left O & M in 1975, 1976, or in 1977. Testimony at trial established that plaintiffs' data base was limited to the O & M employees appearing on the PERS 3600 tape, that is, to employees who were working for O & M in 1978, 1979 or 1980. Accordingly, plaintiffs' studies based on their own data include only 449 non-officer employees, and omit as many as 300 employees who left O & M between 1975 and 1978.

The court finds that this omission casts great doubt upon the utility of the studies based on plaintiffs' own data. The omission is serious, verging on the egregious, because O & M provided plaintiffs with the personnel files of employees not included on the PERS 3600 tapes. O & M's data base included those employees, utilizing information which was gathered directly from the personnel files. Plaintiffs' studies premised on their own data base are thus of limited evidentiary value.^[10]

D. Appropriate Measure of Statistical Significance

O & M contends that plaintiffs' regression studies do not reflect a level of statistical significance sufficient to establish a prima facie case of intentional sex discrimination. O & M asserts that the court should be hesitant to draw an inference of discrimination from any salary disparity with a statistical significance of less than 3 standard deviations.^[11]

In Hazelwood School District v. United States, 433 U.S. 299, 311 n. 17, 97 S. Ct. 2736, 2743 n. 17, 53 L. Ed. 2d 768 (1977) (emphasis added), the Supreme Court compared the actual number of black teachers hired by the defendant with the number of black teachers one would expect, by the laws of probability, given the labor pool and under an unbiased hiring system:

[T]he difference between the observed number of 15 Negro teachers hired ... would vary from the expected number of 62 by more than six standard deviations. Because a fluctuation of more than two or three standard deviations would undercut the hypothesis that decisions were being made randomly with respect to race [Castaneda v. Partida, supra,], 430 U.S. [482] at 497 n. 17 [97 S. Ct. 1272 at 1281 n. 17, 51 L. Ed. 2d 498 (1977)], ... these statistical comparisons would reinforce rather than rebut the [plaintiff's] other proof. If, however, the 5.7% areawide [black population] figure is used, the expected number of Negro teachers hired ... of 23 would be less than 2 standard deviations from the observed total of 15.

In EEOC v. American National Bank, 652 F.2d 1176, 1192 (4th Cir.1981), cert. denied, 459 U.S. 923, 103 S. Ct. 235, 74 L. Ed. 2d 186 (1982) (emphasis supplied), the Fourth Circuit stated:

[I]t appears that well short of three standard deviations the probability levels for chance as explanation [for a disproportionately low number of minority employees] have already dropped far below the point at which courts of law — concerned with proof by the "greater weight" or "preponderance" of the evidence — would presumably have discarded the hypothesis of chance. Just short of two standard deviations — specifically at 1.96 — the probability of chance is only 5 in 100; at just over two and one half, it is only 1 in 100; by three it is less than 1 in 100. W. Hays & R. Winkler, Statistics: Probability, Inference and Decision 218-19, 381-82 (1971). For this reason, authority can be found for the proposition that most social scientists, applying laboratory rigor to rule out chance as even a theoretical possibility rather than the law's rougher gauge of the "preponderance of the evidence," are prepared to discard chance as an hypothesis when its probability level is no more than 5%, i.e. at approximately two standard deviations. Id. at 394.

From all this we conclude that courts of law should be extremely cautious in drawing any conclusions from standard deviations in the range of one to three. Above this range, with standard deviations of more than three the analysis may perhaps safely be used absolutely to exclude chance as a hypothesis, hence absolutely to confirm the legitimacy of an inference of discrimination based upon judicial appraisals that disparities are, to the legally trained eye, "gross.".... Within the range of one to three standard deviations, where the probability of chance as explanation for revealed underrepresentation declines precipitately from only 5% at two standard deviations to less than 1% at three, we do not see how a court can properly find the only other hypothesis — discrimination — dispelled by this analysis alone.

In that case, the court, on the basis of minority hiring figures reflecting standard deviations from one to three, reversed the district court's finding of no discrimination. Nevertheless, EEOC v. American National Bank has been cited by some courts and by O & M for the proposition that courts should also exercise "extreme caution" in drawing an inference of intentional discrimination from studies reflecting standard deviations in the range of 1 to 3.^[12]See EEOC v. Federal Reserve Bank of Richmond, 698 F.2d 633, 647-48 (4th Cir.1983), rev'd on other grounds sub nom. Cooper v. Federal Reserve Bank of Richmond, ___ U.S. ___, 104 S. Ct. 2794, 81 L. Ed. 2d 718 (1984); Gay v. Waiters' and Dairy Lunchmen's Unions, 694 F.2d 531, 551 (9th Cir. 1982). O & M contends that, in light of those decisions, even those of plaintiffs' studies which are not flawed by the problems described above do not establish a prima facie case because they generally reflect student t values lower than three.

The court finds that the cases citing EEOC v. American National Bank, supra, do not purport to create a general Title VII rule that a prima facie case of intentional discrimination can be established through statistics only where the figures indicate disparities significant at the confidence level reflected by three or more standard deviations. In EEOC v. Federal Reserve Bank of Richmond, supra, the court found that plaintiffs had not established a prima facie case of discriminatory promotions where properly calculated standard deviations for four years of employment figures were all less than two. In Gay v. Waiters' Union, supra, the Ninth Circuit found that plaintiffs had not established their respective individual cases of discriminatory hiring, where the standard deviation for the variance between the actual and expected numbers of minority hires at one hotel was 2.46 and at another was 1.30. In the absence of significant anecdotal evidence of intentional discrimination, the statistical disparities were not sufficient to establish the discriminatory intent required for plaintiffs' disparate treatment cases.

While courts evaluating salary and work force statistics in Title VII suits have limited the weight given to disparities significant at levels below two standard deviations, see Sobel v. Yeshiva University, supra, 566 F.Supp. at 1173-74, they have not required plaintiffs to show disparities significant at three standard deviations or more to establish a prima facie case of discrimination. See Segar v. Smith, 738 F.2d 1249 (D.C.Cir.1984), cert. denied, ___ U.S. ___, 105 S. Ct. 2357, 86 L. Ed. 2d 258 (1985) and cases cited therein. Accordingly, this court will consider those studies whose student t values suggest, with a 95% level of confidence that the disparities contained therein are not the result of chance.^[13]

E. Job Titles

O & M argues that many of plaintiffs' regression analyses cannot support a prima facie case because they fail to control for job title. Plaintiffs' response is that variables controlling for that factor were appropriately excluded because (1) job title is largely employer-assigned and therefore can mask discriminatory behavior and (2) O & M uses an unusually large number of job titles in connection with its professional staff, titles which in many instances do not reflect actual differences in job functions. This court has held that

In instances in which assignment to rank is controlled by the employer and arguably is tainted by discrimination, courts have discredited studies which purported to measure salary discrimination by holding rank constant. See James v. Stockholm Valves, 559 F.2d [310], 332 [(5th Cir.1977), cert. denied, 434 U.S. 1034, 98 S. Ct. 767, 54 L. Ed. 2d 781 (1978)]; Mecklenberg v. Montana Board of Regents, 13 Employ.Prac.Dec. [P 11, 438] 6496 [(D.Mont.1976)]; cf. Presseisen v. Swarthmore College, 442 F.Supp. [593,] 619 [E.D.Pa.1977, aff'd mem., 582 F.2d 1275 (3rd Cir.1978)] (requiring regression analysis to include rank level in view of proof that assignment to rank is not discriminatory); see also Finklestein, 80 Colum.L.Rev. [737,] 740-42, 748-49 [1980].

Melani v. Board, supra, 561 F.Supp. at 783.

O & M considers their use of job titles less problematic than assignments of rank which do no more than create a hierarchy of employees performing largely similar tasks, as in college teaching. At O & M, job titles, at least to some extent, reflect job function as well as hierarchy. Cf. Coser v. Moore, supra, 739 F.2d at 753 (regression studies flawed by failure to control for job function within broad employment categories). Thus, as plaintiffs' expert acknowledged, an O & M professional employee may well be assigned to the creative, legal, or accounting department as a result of that individual's training and skills, factors not easily subject to employer manipulation.

In addition, plaintiffs in the instant case have not made any significant showing of discriminatory assignments of job title. Although certain positions seem disproportionately male, plaintiffs have introduced no analyses which study gender differences in job titles while holding constant relevant measures of productivity. Cf. Melani v. Board, supra, 561 F.Supp. at 783 (rank not an appropriate independent variable in salary regression, where logit studies indicate females less likely to be assigned to higher ranks than males with same productivity characteristics.)

On the other hand, the court cannot ignore plaintiffs' argument that its studies need not account for job title because of the sheer number of such classifications at O & M. The record indicates that there have been as many as a hundred job titles at one time at O & M, and that many classifications are occupied by only one or two individuals. A number of high ranking O & M officials have complained about a meaningless profusion of job titles. In addition, notations on O & M's records suggest that differences in titles do not always strictly correspond to differences in job functions. Periodically, they were used by high ranking O & M officials to justify salary differences among their subordinates.

Still, it is undisputed that O & M professional staff perform extremely varied job functions. Accordingly, any meaningful regression study of professional staff salaries must make some allowance for those variations in duties. As this court stated in Melani v. Board, supra, 561 F.Supp. at 777:

[I]f employees hold positions of a widely disparate nature a regression analysis must control for the differences in job title by including either variables measuring additional qualifications specific to certain job titles or a variable for the job titles themselves. See Valentino v. United States Postal Service, 674 F.2d 56, 70-71 (D.C.Cir.1982).

See Coser v. Moore, supra; Trout v. Lehman, supra, 702 F.2d at 1102-03.

In this case, some of the legitimate differences attributable to different job functions can be factored from the analyses by controlling for the employee's department. Since the number of departments is far less than the number of job titles, it is reasonable to require, at a minimum, that plaintiffs present analyses which include department as a variable. For this reason, the court concludes that in order to establish the basis for a prima facie case, plaintiffs' studies are not required to include a variable for job title, but should control for the more significant differences in job function indicated by the employee's department. Cf. Boyd v. Bechtel Corp., 485 F. Supp. 610, 619 (N.D.Cal.1979) (where use of job titles may lessen sample size and consequent usefulness of studies, control for sufficiently similar job families is appropriate). Accordingly, the court finds that a prima facie case cannot be premised upon plaintiffs' Tables VI-1 and VI-2 or on Tables 1 and 2. Those tables fail to include department or job title as a variable or to otherwise control for job functions.^[14]

F. Other Omitted Independent Variables

O & M contends that even if plaintiffs' studies need not include a variable reflecting job title, the omission of independent variables measuring other allegedly significant productivity factors prevents plaintiffs' studies from establishing a prima facie case. Specifically, O & M suggests that plaintiffs' studies must control for former salary, military experience, quality of education, and nature of prior experience.

The probative value of a regression analysis depends in part upon the inclusion of all major variables likely to have a large effect on the dependent variable. Wilkins v. University of Houston, 654 F.2d [388] at 402 [(5th Cir.1981)]; Fisher, supra, at 713. "[A] properly done study begins with a decent theoretical idea of what variables are likely to be important." Fisher, supra, at 715.... Three kinds of evidence may be offered in support of a regression model; direct testimony as to what factors operated in the decision-making process under challenge, what kinds of factors generally operate in decision-making processes of the kind under challenge, and expert testimony concerning what factors can be expected to influence the process under challenge according to principles of economic theory." D. Baldus & J. Cole, Statistical Proof of Discrimination Sec. 8.22 at 70 (1980 & 1982 Supp.) (hereinafter Baldus & Cole). The strength of the factual foundation supporting a regression model may be a factor in assessing whether the group [race or gender] status coefficient indicates discrimination or the influence of legitimate qualifications which happen to correlate with group status. Baldus & Cole, supra, Sec. 8.021 at 66 (1982 Supp.).

Eastland v. Tennessee Valley Authority, supra, 704 F.2d 613, 623 (11th Cir.1983), cert. denied, ___ U.S. ___, 104 S. Ct. 1415, 79 L. Ed. 2d 741 (1984) (hereinafter Eastland v. TVA).

Both the factual foundation for and the legal appropriateness of including the above-mentioned independent variables in studies of O & M are seriously disputed. While O & M correctly contends that courts have approved the use of those variables in Title VII regression analyses, no particular constellation of independent variables is required for a prima facie showing of salary discrimination.^[15]

Title VII regression studies, moreover, need not account for every factor that conceivably might explain differences in salaries or promotions. See Valentino v. United States Postal Service, supra, 674 F.2d at 71 n. 24; Segar v. Civiletti, 508 F. Supp. 690, 696 n. 2 (D.D.C.1981); Finklestein, supra, 80 Colum.L.Rev. at 744-45.

Melani v. Board, supra, 561 F.Supp. at 779; see DeMedina v. Reinhardt, 686 F.2d 997, 1008 (D.C.Cir.1982); cf. Trout v. Lehman, supra, 702 F.2d at 1101 ("if the plaintiffs account for the effect of extraneous variables to the extent reasonably permitted by the available data and the evidence presented strongly supports an inference of discriminatory treatment, the District Court properly may conclude that the plaintiffs have made out a prima facie case.")

The court need not examine the independent variables in each of the regressions introduced in plaintiffs' direct case, since virtually all of those studies, with the possible exception of the study summarized in Table 6, are seriously flawed by the other problems discussed above. Table 6 represents the only evidence introduced in plaintiffs' direct case whose probative value is not seriously undermined by failure to control for job function.

The analysis summarized in Table 6 includes O & M employees hired before the limitations date and relies in part on constructed data for age at highest degree. For these reasons, it must be evaluated with special care. Table 6 included independent variables reflecting job title, education, training and experience. It did not control for prior salary or military experience. In addition, the education variable did not reflect the quality of institution attended and the training variable did not distinguish among different types of technical training received outside the formal college setting. Finally, the prior experience variables did not specifically reflect the type of work in which the employee was engaged before coming to O & M.

In view of the testimony of O & M officials responsible for setting salary, the court finds that the analysis underlying Table 6, which purports to show genderbased salary differences during 1975-79, may well have omitted some of the "major variables likely to have a large effect on the dependent variables." Eastland v. TVA, supra, 704 F.2d at 623. While none of the O & M decision-makers testified that particular weight was given to military experience, they did testify that salary was based in part on the nature and extent of the employee's prior experience. O & M, moreover, recruited employees at certain institutions, and certainly viewed as valuable particularly rigorous academic backgrounds. The general nature of the independent variables used in the analysis underlying Table 6, in addition to the problems noted earlier, leads the court to conclude that it must limit the probative weight given that study.

G. Plaintiffs' Prima Facie Case

From this analysis, the court finds that of all the studies introduced by plaintiffs in their direct case, only Table 6 arguably serves to establish a prima facie case of salary discrimination. The other studies are too seriously flawed by their failure to control for job function, their omission of employees who left O & M before 1978, and their reliance on constructed dates of graduation. Table 6 shares the last defect and is limited by the use of overly general measures of prior experience and prior education and training. For these reasons, the studies introduced in plaintiffs' direct case do not establish a prima facie case of salary discrimination.

The studies introduced by plaintiffs in their rebuttal, however, are not characterized by the same flaws. For the most part, those studies use O & M's data base and employ O & M's more specific variables with respect to prior experience and education or training. Most of the studies do not rely on constructed data; some of them reflect statistically significant wage disparities (student t values of two or more). Although the rebuttal studies do not control for military experience, they arguably would be sufficient to establish a prima facie case of gender-based wage discrimination.

The court, however, need not resolve that question. O & M has introduced significant evidence of legitimate reasons for the salary disparities reflected in plaintiffs' studies. Thus, even if it is assumed that plaintiffs have established a prima facie case, O & M has met its burden of production. Accordingly, under United States Postal Service v. Aikens, 460 U.S. 711, 103 S. Ct. 1478, 75 L. Ed. 2d 403 (1983), the court moves to the broader question of whether plaintiffs have established, by a preponderance of the evidence, class-wide salary discrimination.

II. O & M's Evidence of Non-Discrimination

The statistical evidence in O & M's case consists of a number of regression studies which, like those of the plaintiffs, were designed to measure salary differences between men and women at O & M while controlling for productivity-related factors. O & M's regressions differ from plaintiffs' primarily in their inclusion of a greater number of independent variables, variables intended to provide more precise control for characteristics purportedly related to employee productivity. Generally, consideration of these variables reduced the size of (unaccounted-for) dollar or percentage salary differences between men and women and also reduced the student t values measuring the statistical significance of those differences.

Plaintiffs object to a number of the independent variables utilized by O & M in its studies. Plaintiffs also contend that O & M unjustifiably deleted from certain studies large employee groups, such as those without college degrees, those occupying the EEO category of "manager," and those whose first job upon completing school was at O & M. Thus, plaintiffs argue, O & M's regressions included too large a number of independent variables in proportion to the number of individuals studied.

A. O & M's Inclusion of Additional Independent Variables

As discussed earlier, regressions of the type at issue here should include "all major variables likely to have a large effect on the dependent variable." Eastland v. TVA, supra, 704 F.2d at 623. The court is to determine which variables are appropriate on the basis of the testimony concerning the nature of the defendant employer's business and the expert testimony regarding productivity factors relevant in the advertising industry or generally relevant under the theories of labor economics. Id.

(i) Prior Work Experience, College Major and Tenure at O & M

Plaintiffs criticize defendant's expert's use of distinct independent variables for employees' prior experience in advertising, prior experience as a professional in a field other than advertising, and prior experience in a non-professional position. O & M's expert based those distinctions on discussions with high level O & M management, where he was told what was repeated at trial, i.e., that employees with prior professional experience in advertising are often compensated differently from those with prior experience only in non-advertising positions or non-professional positions. The court therefore concludes that it was appropriate for O & M to distinguish between different types of prior experience in its independent variables.

Plaintiffs similarly criticize O & M's classification of the employees' college majors into six groups. The evidence suggests, however, that differences in college majors may provide non-discriminatory explanations of salary differences. For example, it may be reasonable to compensate art majors differently from accounting majors. See Valentino v. United States Postal Service, supra, 674 F.2d at 71 & n. 22.

Plaintiffs also criticize as overly specific O & M's categorization of the different types of experience employees have had during their tenure at O & M. O & M's expert met that objection by using plaintiffs' method for controlling for job tenure at O & M in O & M's reply report and later studies.

In sum, the court concludes that O & M has corrected any problem in its measure of job tenure and that O & M's use of additional variables to classify different types of prior experience and college majors is proper, since the classifications represent factors likely to have a significant effect on salary.^[16]

(ii) College Standing

Plaintiffs also criticize the method used by O & M to indicate that an employee had attended a highly-rated college. The variable O & M developed to control for this factor was based upon the Gourman Report, a ranking of colleges and universities in the United States which identifies fifty-five outstanding institutions. The Gourman Report does not list among the highly ranked schools a number of undisputedly excellent colleges attended exclusively or primarily by women such as Wellesley, Mount Holyoke, Smith, Vassar, and Barnard.

O & M's expert suggested that the proper question with regard to a disputed variable is "whether the variable potentially represents a sex-neutral basis for the determination of worker qualifications and, hence, salaries." While quality of the college attended by an employee may be an appropriate productivity-related factor, see Melani v. Board, supra, 561 F.Supp. at 774, a measure of quality which for no apparent reason omits highly respected women's colleges must be suspect.

In response to plaintiffs' criticism, O & M's expert prepared studies which used a "modified" college standing variable which included as highly-rated schools the women's colleges noted by plaintiffs' expert. Substitution of the modified variable does not always produce a salary disparity between men and women larger than that seen with use of the original college standing variable.^[17] Thus, while the modified college standing variable is more appropriate, it is unclear whether the use of the unmodified form skewed O & M's results in any meaningful fashion.

(iii) Military Experience

Plaintiffs criticize the use of variables controlling for military experience in some of O & M's regression analyses. Although plaintiffs do not dispute that time spent in the military may provide individuals with experience or training relevant to their work at O & M,^[18] they contend that it nevertheless must be excluded because military experience is a trait shared almost exclusively by males.

In response, O & M's expert explained why regression analyses of gender-based salary differences appropriately may include independent variables representing productivity characteristics held primarily by members of one gender:

Military experience — it's true that men are more likely to have it than women, no doubt about that. But the same thing is true of many of the variables here. There are more men that have MBA's than women, and so on.

The fact that a variable — that women score differently on a variable than a man is no reason to reject it from the analysis by itself....

Well, suppose that military training did not have any effect on the pay of men. In other words, that men with training [do not] get more than men without. Then the regression coefficient [for the independent variable for military experience] would come out to be zero, and it would have no effect on the estimated pay difference [by gender] at all.

In other words, the regression model is fixed that a variable which has no effect on the determination of pay will ... get a coefficient of zero.

It will just get kicked out as not having any influence.

I should add that there is another kind of regression, another kind of statistical analysis where that would be true. We have not used that method.....

Wise and I agree completely that the right method to use here is standard multiple regression....

So since we used the right method, there is no reason in the world not to use military experience.... If it turned out that it had no effect on the pay of males, it would get a zero coefficient and wouldn't matter [in reflecting any pay differential between men and women.]

On cross-examination, O & M's expert repeated that if military training were irrelevant, that is, if it did not correlate with salary differences among men, those variables would not bias the regression coefficients, but would reduce the chance of finding statistically significant differences in the salaries of men and women. This does not appear to be the case here, however.

At trial, plaintiffs' expert conceded that failure to consider military experience might artificially inflate any salary disparity between men and women. Plaintiffs' expert also testified that it might be reasonable to rely on studies taking military experience into account. Although plaintiffs argue in their memorandum that inclusion of the military experience variables serves no purpose other than reducing the number of degrees of freedom in the analysis (see discussion infra at 1535), they do not provide proof that this is the case. A number of paired studies are in evidence, where the members of each pair differ only with respect to use of the military experience variables. Despite this, plaintiffs have not referred the court to anything within O & M's studies which indicates that military experience is irrelevant, or that the differences between the paired studies are attributable solely to differences in degrees of freedom.

This is not a case where the challenged variables are "patently collinear" with the other experience variables and thus unjustifiably repetitive. Cf. Sobel v. Yeshiva University, supra, 566 F.Supp. at 1181 (variables which necessarily overlap because each is a subset or composite of the others — e.g., job tenure, total medical experience, and total experience as indicators of experience of medical faculty members — are "patently collinear"). Given the nature of the variables and the expert testimony in this case, the court cannot simply assume that military experience is irrelevant or that the other prior experience variables adequately control for time spent in the military. In the absence of such evidence, the court concludes that O & M has established a sufficient basis for use of the variables and that any artificial reduction of the statistical significance of genderbased disparities is insignificant.

(iv) Other Training Variables

Plaintiffs also criticize O & M's studies for their distinctions among types of noncollege training, such as training in data processing, secretarial work, writing, or technical skills. Plaintiffs' chief objection is to the separate treatment of secretarial training, a qualification shared almost exclusively by O & M's female employees.

For the reasons set forth in O & M's expert's testimony about military service, the court finds that use of an independent variable controlling for secretarial training is not inherently suspect. In addition, the court notes that O & M revised some of its regressions, substituting plaintiffs' expert's single "other training variable," which does not differentiate among the types of non-college schooling. As in the case of the revised college standing variable, use of plaintiffs' single "other training" variable increased the observed male-female salary differentials in some, but not all, of the years studied. The court finds the plaintiffs have not established either that O & M's several prior training variables are inappropriately specific, or that use of those variables skewed O & M's results in any meaningful fashion.

(v) Former Salary

Plaintiffs have two objections to O & M's use of an independent variable to control for an employee's salary prior to O & M. First, plaintiffs argue that because prior salary reflects sex discrimination in the marketplace, analysis which controls for prior salary serves to mask genderbased salary differences at O & M. Second, since regression studies that control for prior salary must exclude from consideration the significant number of employees hired by O & M immediately upon completion of their education, plaintiffs argue that the prior salary variable substantially reduces the number of individuals studied, thus making detection of significant gender-based disparities less likely.

O & M's expert acknowledged that "a potential disadvantage with the use of this [prior salary] variable is that it may reflect, in addition to skill and quality, sex-biased employment practices by some other employer," but relies on the fact that a number of Title VII and Equal Pay Act (29 U.S.C. § 206(d)(1)) decisions have mentioned prior salary as a legitimate explanation for salary differences between men and women. Virtually all of the cases cited by O & M, however, refer to prior salary insofar as it is reflective of the employee productivity factors of experience, education or supervisory skills, see Lemons v. City and County of Denver, 620 F.2d 228 (10th Cir.), cert. denied, 449 U.S. 888, 101 S. Ct. 244, 66 L. Ed. 2d 114 (1980); EEOC v. Aetna Insurance Co., 616 F.2d 719, 723 (4th Cir.1980); Horner v. Mary Institute, 613 F.2d 706 (8th Cir.1980), or of the varying market place demand for different jobs, or, in universities, for different academic specialties. See Lamphere v. Brown University, 685 F.2d 743, 750 (1st Cir.1982) (academic department); Christensen v. State of Iowa, 563 F.2d 353, 356 & n. 7 (8th Cir.1977) (market value of various jobs is permissible basis for salary differences although "relying on `market price' as a basis for paying women lower wages than men doing substantially equal work [is] a violation of the EPA."); Briggs v. City of Madison, 536 F. Supp. 435 (W.D.Wis.1982) (disparity in prior salary reflective of different job functions performed by men and women). Since O & M's studies include variables controlling for experience, education, and field of expertise or department, there is little need to include prior salary as an indirect method of measuring those factors.^[19]

O & M's expert also asserted that prior salary serves as a proxy for less concrete productivity-related factors such as an employee's "drive" or "creativity." The incremental utility of prior salary in reflecting those unquestionably important characteristics, however, may well be outweighed by the risk that sex-discrimination by other employers will be masked. This seems particularly likely where, as here, there is no evidence that the prior salaries were nondiscriminatory,^[20] and where consideration of prior salary can shed no light on the creativity or drive of the significant numbers of O & M employees who had no prior employment. Accordingly, the court concludes that it will place primary reliance on those O & M studies which do not control for former salary.

B. Alleged Overfitting

Plaintiffs also argue that O & M's regression studies, regardless of which independent variables are included, are characterized by the problem of "overfitting." Overfitting is the use of too many independent or explanatory variables relative to the number of observations, e.g., the number of employees studied. By introducing an inappropriately large number of independent variables, a study characterized by overfitting approaches a perfect but meaningless explanation of each employee's salary. According to plaintiffs' expert "as generally used in literature [overfitting] means that you have got so many variables in the [regression] model that the ... coefficients of the [independent] variables are unreliable; unreliable in the sense that [they differ from] study to study; they move around a lot, that they don't make sense on the[ir] face in many cases...."

Overfitting is related to the number of degrees of freedom in a regression analysis. According to plaintiffs' expert, the number of degrees of freedom in a regression is, roughly, the number of observations less the number of independent variables. Determination of an appropriate number of degrees of freedom is a relative matter; the statistical literature sets no precise point below which the number of degrees of freedom is impermissibly small. Rather, according to plaintiffs' expert, a subjective determination must be made in any particular study.

Plaintiffs have failed to provide the court with any evidentiary basis for concluding that O & M's studies are systematically characterized by the problem of overfitting. Plaintiffs' expert cited one or two apparently outlandish coefficients for variables in O & M's studies. On cross-examination, however, he conceded that such anomalous results are not uncommon even in appropriate regression studies, including those prepared on behalf of the plaintiffs.

Plaintiffs' expert also indicated that while the questioned independent variables concerning former salary and job title were deleted from the studies in O & M's 1983 reply, the number of degrees of freedom might be inadequate, and result in overfitting, because O & M added a host of other independent variables measuring education, experience and other productivity characteristics. O & M, however, has shown that while its initial study of salaries of men and women with certain productivity characteristics (which controlled for job title) had 94 independent variables and 127 degrees of freedom, the 1983 reply report study (which did not control for job title but added other independent variables) actually had a total of 93 independent variables and 177 degrees of freedom.

In sum, the court finds that plaintiffs, who carry the burden of persuasion at all times in this proceeding, have not provided enough expert testimony for the court to make the subjective determination that defendant's studies are characterized by overfitting and an insufficient number of degrees of freedom.

C. Evaluation of O & M's Studies

(i) Salaries of Employees Hired after May 1975

The centerpiece of O & M's statistical evidence is a group of studies which concern only those personnel decisions effected after May, 1975. Using defendant's "Supplementary Report Variables," Table A (column 1), as amended by plaintiffs to include managers and officers, shows only one year of statistically significant salary differences between men and women sharing certain productivity characteristics and hired after May, 1975.^[21] This is true even though defendant's variables do not include job title, former salary, or military experience. Using the less specific non-college training and modified college standing variables suggested by plaintiffs, Table A (column 3) indicates significant differences in three years. As discussed earlier, however, the court does not disapprove of defendant's non-college training variables and is not convinced that use of defendant's college standing variables necessarily skews the results of the analysis. Further, these significant differences disappear when military experience is added as a variable.^[22]

Plaintiffs reanalyzed Table A ("2d Table A") using "experience" data based on constructed dates of graduation for those O & M employees with high school degrees or who did not complete college, thereby adding approximately 40 individuals to the number studied in Table A and increasing the chance that a given disparity will be statistically significant.^[23] This second Table A shows four years of statistically significant salary differences. Using the modified non-college training and college standing variables, the results are similar.

A number of factors prevent the court from relying on 2d Table A, however. As discussed earlier, O & M showed that where the actual data were available, plaintiffs' method for constructing age at highest degree has resulted in significant errors. Although the actual data are not available for the employees for whom plaintiffs constructed graduation dates in 2d Table A, plaintiffs have suggested no reason why the constructed information should be more accurate for those individuals. Moreover, plaintiffs have offered no reason for constructing the age at highest degree only for those employees who completed high school or failed to complete college.

As noted earlier, where plaintiffs' suggested "other training" and modified college standing variables are used in Table A (column 3), statistically significant differences appear for three years.^[24] With the addition of military experience as a variable, however, those significant differences disappear.^[25] Plaintiffs' expert was cross-examined with regard to the result produced by adding the military experience variables.^[26]

Q You said before that you didn't think that military experience would make a difference in your conclusions in certain studies.

Hypothetically, if in fact adding military experience and military experience alone to one of your studies changed the result from significant to insignificant, would you think it would be more reasonable to rely on the study that included military, would that cause you to review your conclusion?

A That would cause me to look more carefully at military experience as a variable.

Q But might it be reasonable to —

A It would cause me to review my decision.

Q It would. It might be reasonable to rely on the study that included military experience?

A Yes; it might be.

Q Do you still stand by that testimony?

A Yes. It might be. Since the addition of military experience so dramatically alters the results, the court is compelled to limit the weight given to the results in Table A which reflect plaintiffs' modified variables.

In addition, O & M has submitted a study of salary at hire for employees beginning work at O & M after May 1975. The results show no statistically significant salary difference between men and women,^[27] even when the analysis utilizes plaintiffs' modified training and college standing variables, and deletes job title and former salary as variables.

The court therefore concludes that the record indicates at most one year with a statistically significant salary gap between men and women hired after 1975 by O & M. When military experience is controlled for, no statistically significant salary differences appear for that group.

(ii) Raises for Employees between 1975 and 1979

O & M also studied the salary decisions effected after the limitations date by analyzing the raises given to O & M employees between 1975 and 1979.^[28] Those studies, which controlled for age, education, other training, time since degree, college major, standing of college, prior work experience, tenure and department, identified no statistically significant differences between men and women in salary increases, whether computed as dollar increases or as percentage increases. Although the probative weight of the studies of raises is limited somewhat by use of the unmodified college standing variable, it is enhanced by the exclusion of former salary and job title as variables. Accordingly, the court considers those studies additional evidence that salary decisions within the limitations period were effected without regard to gender.

(iii) Studies of All Employees at O & M from 1975 to 1979

A number of regression studies analyze the salaries of all O & M employees during the years 1975 to 1979, regardless of each employee's date of hire. As discussed earlier, such studies must be evaluated with special care because of the limitations date issues. Table C (as amended by plaintiffs to include managers and officers), which compares salaries of male and female college graduates while controlling for a number of factors including job title, shows a statistically significant difference in salary between men and women for one year. That study, however, is limited in its probative value because it excludes a large number of employees who are not college graduates.^[29]

Somewhat similar studies which do not control for job title (the various versions of Table D) analyze the salaries of those male and female employees who are exempt from the overtime provisions of the Fair Labor Standards Act, 29 U.S.C. § 201 et seq.^[30] Limitation of the studied population to exempt employees is not unreasonable. Although some members of the plaintiff class and their male counterparts receive overtime wages, it is problematic to analyze the earnings of such employees in the same manner as the salaries of exempt employees who receive no overtime regardless of the number of hours worked per week. Studies which cover the exempt employees indicate statistically significant salary disparities in only one year, even though former salary and job title are excluded as variables.

Plaintiffs have produced a version of Table D ("2d Table D") which includes more employees by using the constructed age at highest degree for high school graduates and employees with some college. This 2d Table D shows two years of statistically significant salary differences.^[31] Because of the reliance on constructed dates of graduation, however, 2d Table D deserves little consideration. Further, as plaintiffs' expert acknowledged, such a pattern, where regression results change from significant to insignificant or vice versa with the addition of just a few employees or another independent variable, suggests that the results are "sensitive" and that there may be problems with the regression model. See generally Sobel v. Yeshiva University, supra, 566 F.Supp. at 1181.

In response to an early version of O & M's Table D that excluded managers, plaintiffs submitted a distinct set of regressions studying salary differences by gender for O & M employees during the years 1975-1979, excluding managers. Those studies, presented in Table I, control for department, prior work experience, tenure at O & M, education, age, time since degree, college major, other training and college standing. They do not control for job title. Using defendant's versions of the "other training" and "college standing" variables, Table I (column 1) indicates significant disparities for two years. Using plaintiffs' versions, Table I (column 3) indicates statistically significant salary disparities for three years.^[32]

O & M revised Table I (column 1) to include managers and control for military experience. When this was done, the number of years reflecting statistically significant wage differences between men and women was reduced to one.^[33] Thus, the results in Table I also appear sensitive and, as a result, somewhat questionable.

The more serious flaw in the Table I studies, however, is the failure to control for differences in job function in a manner more precise than that afforded by simply controlling for job department. Failure to carefully control for differences in job duties is particularly problematic where, as in Table I, a study includes employees hired before May, 1975, and thus may reflect the effects of discriminatory but time-barred employment decisions. Accordingly, the court finds the studies included in Table I to be of limited utility.

In summary, O & M has rebutted any inference of a pattern and practice of salary discrimination which may arise from plaintiffs' studies of the earnings of O & M employees between 1975 and 1979. The few showings of statistically significant disparities largely disappear when military experience is added as a variable. Since such shifts from statistically significant to statistically insignificant results suggest the sensitive nature of the disparities indicated in plaintiffs' studies, it is especially difficult to consider those studies sufficient to support a finding of systematic and intentional discrimination.

Plaintiffs' analysis of salary differences for "all employees for whom all relevant data are available" ("2d Table C") provides plaintiffs' most significant evidence of salary discrimination.^[34] The study underlying 2d Table C included all O & M employees, whether hired before or after the limitations date. It controlled for department, job title, prior work experience, tenure, education, age, time since degree, college major, other training and standing of college. Even so, it revealed three years of statistically significant salary differences between men and women.

O & M never submitted at trial a study comparable to 2d Table C which controlled for military experience.^[35] (O & M did submit similar studies on the same individuals, with controls for military experience but not for job title.) Thus, plaintiffs' 2d Table C is some evidence of gender-based salary discrimination at O & M. The weight of that evidence, however, is reduced by the possibility that some of the statistically significant disparities in 2d Table C are sufficiently sensitive to be reduced to insignificance, were military experience to be included as a variable. In addition, it is limited by the incorporation of pre-limitations date personnel decisions, since there is considerable evidence that O & M's salary practices changed in 1976.^[36]

Most significantly, the court finds that the results reported in 2d Table C are too weak to support the entire burden of proving a class-wide case of intentional salary discrimination. For two of the five years studied, the results do not reflect statistically significant salary disparities. While the study indicates salary disparities significant at the level of two standard deviations for the remaining three years, in only one year is the disparity significant at the level of three standard deviations. Although strong statistical proof is not required in all disparate treatment cases, its role in the instant case is critical, since there is no anecdotal evidence supporting the figures^[37] and the most probative evidence, i.e., that which is finely tuned to the period after May 30, 1975, indicates no gender-based salary discrimination. Cf. Hazelwood School District v. United States, supra, 433 U.S. at 307-08, 97 S. Ct. at 2741 ("gross statistical disparities" may constitute prima facie proof of discrimination); Wilkins v. University of Houston, 654 F.2d 388, 402-03 n. 18 (5th Cir.1981), vacated on other grounds, 459 U.S. 809, 103 S. Ct. 34, 74 L. Ed. 2d 47 (1982) (in claim based entirely on statistics, gross statistical disparities evidenced through multiple regression analysis might establish prima facie proof of salary discrimination). Accordingly, since 2d Table C fails to provide a particularly strong showing of salary discrimination, the court concludes that plaintiffs have not established by a preponderance of the evidence a pattern and practice of intentional salary discrimination.

Conclusion

For the foregoing reasons, plaintiffs' claim of salary discrimination is dismissed. Plaintiff Zukofsky's individual salary discrimination claim is also dismissed in its entirety. The clerk shall enter judgment.

So Ordered.

NOTES

[1] Since at least at the level of vice-president, an "officer" title makes no difference in an O & M employee's job functions, it is useful to compare both male officers and male non-officers with the female non-officers in the plaintiff class.

[2] The studies excluded from consideration include:

Pl. 1985 Report, Ex. 284 (earlier versions excluded at trial),

Pl. 1985 Report, Ex. 285 (earlier versions excluded at trial),

Pl. 1985 Reply, Table 14,

Def. 1985 Report, Table 5,

All Versions of Table 16 and Table 17 (Pl. and Def. 1985 Reports and Reply Reports),

Def. 1985 Reply Table C +, and

Def. 1985 Reply, Second Table A# ("High School" and Some College) (does not replicate study in evidence at trial).

Def. 1985 Reply, Table A# ("All Professionals and Managers for Whom All Relevant Data are Available") essentially replicates O & M's trial exhibit 698(b) and is admissible.

[3] Submitted with the stipulation were certain exhibits not previously in the record.

[4] Reva Korda served until 1980, when she was replaced as head of the creative department by Norman Berry.

[5] Table II (Pl. 1985 Report, Ex. 301) Chow Test Results F Statistics* with Degrees of Freedom ------------------------------------------------------------------------- Using Plaintiffs' Defendant's Other Training Supplementary Variable and Report Variables Using Plaintiffs' Suggested (excluding former Other Training College Standing salary) Variable Variable ------------------------------------------------------------------------- 1979 2.41* (258,76) 2.47* (270,71) 2.43* (270,71) 1978 2.72* (265,77) 2.74* (278,71) 2.70* (278,71) 1977 3.06* (208,74) 3.22* (220,69) 3.28* (220,69) 1976 2.53* (167,68) 2.64* (178,64) 2.70* (178,64) ------------------------------------------------------------------------- * Indicates an F test that is statistically significant at the 0.05 level.

[6] The studies which rely on constructed dates of graduation include Tables VI-1 and VI-2 (in Pl. 1984 Report, as amended, and Pl. 1985 Report), Tables 1 and 2 (in Pl. 1984 Reply, as amended, and Pl. 1985 Report), and Tables 3, 4, 5 and 6 (in Pl. 1984 Reply and Pl. 1985 report). The precise method used to estimate "age at highest degree" varies somewhat. See Defendant's post-hearing memorandum of law, n. 3, 4. The 2d Table A and 2d Table D, described infra, n. 20, 28, also rely on constructed data.

[7] O & M's expert apparently deleted from each regression analysis individuals for whom he could not obtain data for the relevant variables in that regression. Plaintiffs' expert acknowledged that procedure as a common one, and, in fact, followed it in his own studies, except in the instances in which he constructed the age at highest degree.

[8] The court notes that Table 5 (see Pl. 1985 Report and Pl. 1984 Reply) which uses plaintiffs' own data productivity variables, indicates no sex discrimination in raises of O & M professional employees for the period from 1976 to 1979.

[9] Plaintiffs argue that it is inappropriate for defendant to challenge the reliability of plaintiffs' studies containing estimated experience data, since defendant successfully opposed introduction at trial of substituted evidence not containing "made up" data. See n. 2, supra; Rossini v. Ogilvy & Mather, Inc., 597 F. Supp. 1120, 1163 (S.D.N.Y.1984). Plaintiffs misconstrue the relevance of the rulings at trial. Because their offer of proof was rejected, and because acceptance of such proof at this time would in essence "reopen the evidence" and exceed the scope of the court's ruling with respect to officer data, the offered evidence is in no way before the court. That the evidence might have been admissible in other circumstances is irrelevant. Having had the evidence excluded, the defendant is entitled to argue from all studies in evidence, and to treat the excluded evidence as non-existent. To require that the defendant do otherwise would be to give weight to the excluded evidence. The same reasoning, however, leads the court to conclude that it is improper for defendants to argue that the court should consider for any reason Def. 2d Table C (page 9 of Def. 1985 Reply).

[10] Those studies include Tables VI-1 and VI-2 (in Pl. 1984 Report, as amended, and Pl. 1985 Report), and Tables 1 and 2 (in Pl. 1984 Reply, as amended and Pl. 1985 Report). Of course, those problems do not affect the validity of plaintiffs' studies which rely on O & M's data base.

[11] At trial, both experts referred to student t values, the measure of statistical significance used by both sides, as roughly analogous to the number of standard deviations. This court has stated:

A student t is a statistic concerning the sample standard deviation, as opposed to the population standard deviation. It is a test of the validity of statistics that are drawn from a limited sample. R. Hoel & R. Jessen, Basic Statistics for Business and Economics, 187-90 (2d ed. 1977).

Sobel v. Yeshiva University, supra, 566 F.Supp. at 1176 n. 32.

[12] The underlined portion of the quotation from EEOC v. American National Bank can also be read as suggesting that if the statistical evidence is such that observed disparities vary by one to three standard deviations from what would be expected under a situation of no discrimination, a court should not automatically conclude there is no discrimination, but should seek additional evidence.

[13] Plaintiffs' submissions in 1983 contained a number of studies comparing raw averages of male and female salaries, some of which reflect differences by job title. Because those studies do not control for experience, education and other productivity-related factors, they are of marginal usefulness, particularly in view of the wide range of skills and backgrounds in O & M's employees. Accordingly, the court has not discussed those studies at length, nor has it accorded them any significant probative value. See Pouncy v. Prudential Insurance Co., 668 F.2d 795, 802-03 (5th Cir.1982); Melani v. Board, supra, 561 F.Supp. at 777.

[14] Tables VI-1 and VI-2 appear in Pl. 1984 Report, as amended, and in Pl. 1985 Report. Tables 1 and 2 appear in Pl. 1984 Reply, as amended, and in Pl. 1985 Report. Tables 3, 4, 5, and 6 appear in Pl. 1984 Report and Pl. 1985 Report.

[15] Generally, courts have found the particular independent variables cited by O & M to be an effective part of a Title VII defendant's rebuttal rather than a necessary component of the regressions plaintiffs must adduce for a prima facie case. See Valentino v. United States Postal Service, supra, 674 F.2d at 71 n. 24 (quality of education); Agarwal v. McKee & Co., 19 Fair Empl. Prac. Cas. (BNA) 503 (N.D.Cal.1977), aff'd, 644 F.2d 803 (9th Cir.1981) (prior salary).

[16] O & M's use of an independent variable for age is not seriously disputed. Age is frequently included in regression analyses of this type as a relevant measure of experience. See Agarwal v. McKee & Co., supra, 19 Fair Empl.Prac.Cas. at 507; Melani v. Board, supra, 561 F.Supp. at 774.

[17] See, e.g., Def. 1983 Reply, Table A; Def. 1983 Ex. 698(b), Table A #; Def. 1985 Report, Table 12.

[18] This would be the case, for example, where an employee wrote copy for a military newspaper while in the service. O & M's studies took into account the length of military experience and the rank held in the military.

[19] Neither plaintiffs' nor defendant's studies directly take into account the level of the job an employee held before coming to O & M. Analysis of prior salary may serve in part to control for the impact of that factor.

[20] On cross-examination, O & M's expert testified that he had studied the prior salaries of male and female O & M employees. On re-direct, he stated that those studies indicated no significant gender-based differences. O & M, however, did not introduce into evidence that analysis, or any of the underlying data. Plaintiffs' expert testified initially that he had no opinion as to whether the prior salaries of O & M employees reflected gender discrimination. On rebuttal, the court sustained an objection to plaintiffs' introduction of a study of that question, on the ground that the study should have been submitted as part of plaintiffs' case in chief.

O & M objected to the introduction of a 1983 survey of advertising salaries by Adweek magazine. Decision was reserved. The court now finds the survey inadmissible because, although plaintiffs prior to trial were aware of the survey, they failed to list it as an exhibit in the pre-trial order. Such an omission is particularly critical with regard to a statistical document which requires considerable study for purposes of cross-examination. However, even if the Adweek report were admissible, it presents too many methodological problems to be given significant weight by this court.

[21] See note 24.

[22] See note 25.

[23] See Pl. 1985 Report, Ex. 283, 2d Table A, entitled:

Table A

Average Female/Male Salary Differences for Employees Hired Subsequent to May 1975

....

Plus All "High School" and "Some College"

[24] Table A (Pl. 1985 Report) Average Female/Male Salary Differences for Employees Hired Subsequent to May 1975 All Professionals and Managers for Whom All Relevant Data are Available ---------------------------------------------------------------------------- Defendant's Using Plaintiffs' Supplementary Using Plaintiffs' Other Training Report Variables Other Training Variable and (excluded former Variable Suggested College salary)+ Standing Variable Year ---------------------------------------------------------------------------- 1979 $-1,564 $-1,873 $-2,122* (-1.52) (-1.80) (-2.02) 1978 -1,736 -2,040* -2,140* (-1.75) (-2.03) (-2.09) 1977 -2,171* -2,178* -2,067 (-2.07) (-2.07) (-1.96) 1976 -2,556 -3,763* -3,694* (-1.68) (-2.70) (-2.60) ---------------------------------------------------------------------------- * Indicates a difference that is statistically significant at the 0.05 probability level. + The variables used in this study include department, prior work experience, tenure, education, age, time since degree, college major, other training and college standing. Job title is not included.

[25] Table A# (With Military Experience Added) (Def. 1985 Reply Report) Average Female/Male Salary Differences for Employees Hired Subsequent to May 1975 All Professionals and Managers for Whom All Relevant Data Are Available [Includes Officers] --------------------------------------------------------------------------- Using Plaintiffs' Defendant's Other Training Supplementary Variable and Report Variables Using Plaintiffs' Suggested College (excluded former Other Training Standing salary)+ Variable Variable --------------------------------------------------------------------------- 1979 $-1420 $-1634 $-1873 (-1.46) (-1.66) (-1.88) 1978 -1604 -1905 -1937 (-1.62) (-1.87) (-1.88) 1977 -1323 -1359 -1254 (-1.07) (-1.14) (-1.04) 1976 -1218 -1502 -1238 (-0.87) (-1.11) (-0.88) --------------------------------------------------------------------------- None of these differences is statistically significant at the 0.05 probability level. + The variables used in this study include department, prior work experience, tenure, education, age, time since degree, college major, other training, college standing and military experience. Job title is not included. [Same as Tables 3 and 12 of the Defendant's 1985 Report.] # These studies use defendant's data base as "corrected" by plaintiffs' expert in his reply report. See also Def. 1983 Ex. 698(b), Table A#.

[26] The military experience variables had a similar impact when added to the parallel studies (without officers) introduced at trial.

[27] Table 11 (Def. 1985 Report) Average Female/Male Differences In Salaries At Hire for Employees Hired After May 1975 ----------------------------------------------------------------------------------- 1 2 3 4 Using Plaintiffs' Defendant's Defendant's Using Training First Report Supplementary Plaintiffs' Variable & Variables (includes Report Variables Other Suggested former (excludes former Training College Standing salary)# salary)+ Variable Variable ----------------------------------------------------------------------------------- -579 -1024 -1245 -1434 ----------------------------------------------------------------------------------- 5 6 Corrected for Education, Experience, Corrected for Education, Time since Year of Hire, Job Title at Hire Degree, Age, Experience, Year of Hire ----------------------------------------------------------------------------------- -566 -1220 ----------------------------------------------------------------------------------- None of these differences is statistically significant at the 0.05 probability level. # The variables used in this study include education, experience, year of hire, and salary prior to hire by O&M. This is the same study as Table VII-11 from Defendant's 1983 Report, p. 59. Job title is not included. + The variables used in this study include department, prior work experience, education, age, time since degree, college major, other training, and college standing. Job title is not included.

This table was originally Exhibit D-698(d). Ex. 698(d) essentially reproduced Table B of O & M's reply report (1983), but included managers. Part 6 of D. Ex. 698d added a study reflecting a particular combination of independent variables, about which defendant's expert had been examined during O & M's direct case.

[28] TABLE 1 (Def. 1985 Report) Average Male/Female Differences In Salary Increases -------------------------------------------------------------------- Year Dollars Percentages -------------------------------------------------------------------- 1978-79 -204 - 0.5 1977-78 + 84 + 0.1 1976-77 -392 - 1.7 1975-76 +159 + 1.7 _____________________________________________________________________

None of the differences is statistically significant at the 0.05 probability level.

The variables used in this study include age, education, other training, time since degree, college major, standing of college, prior work experience, tenure and department.

This table was originally Exhibit D-698(h).

[29] Table C (Pl. 1985 Report, Ex. 283) Average Female/Male Salary Differences For All Professionals and Managers College Graduates -------------------------------------------------------------------------- Year Number of Observations -------------------------------------------------------------------------- 1979 $ -993 302 (-1.50) 1978 -1,232 303 (-1.97) 1977 -1,060 253 (-1.69) 1976 -1,698* 218 (-2.26) 1975 67 181 (0.08) ----------------------------------------------------------------------- * Indicates a difference that is statistically significant at the 0.05 probability level.

The variables used in this study include department, job title, prior work experience, tenure, education, age, time since degree, college major, other training, and standing of college.

These results do not change substantially when military experience is added as a variable, as the following chart shows:

                                   Table C#
                               (Pl. 1985 Reply)
                    Average Female/Male Salary Differences
                      For All Professionals and Managers
                                College Graduates
                   Salary              Student t                 Number of
  Year           Difference              Value                 Observations
---------------------------------------------------------------------------
  1979            $  -966                (-0.98)                    365
  1978             -1,060                (-1.29)                    367
  1977             -1,159                (-1.36)                    309
  1976             -1,854*               (-2.00)                    265
  1975             -1,216                (-1.16)                    214
---------------------------------------------------------------------------
* Indicates statistically significant at the 0.05 probability level.

The variables used in this study include department, prior work experience, tenure, education, age, time since degree, college major, other training, standing of college, and military experience.

This table was originally Exhibit D-698(g).

[30] Table D (Pl. 1985 Report) Average Female/Male Salary Differences for All Professionals and Managers Defendant's Defendant's First Supplementary Report Report Variables (includes Variables (excludes former salary and former salary and job titles)# job titles)+ All Employees (except officers) All Exempt° Employees for Whom All Relevant Data Year (including officers) Are Available ------------------------------------------------------------------------------------------- 1979 $ -1,333 -375 (-1.12) 1978 -1,808 -937 (-1.75) 1977 -2,609* -1,038 (-2.32) 1976 -1,982 -473 (-1.73) 1975 661 -2,855* (0.55) ------------------------------------------------------------------------------------------- * An asterisk indicates a statistically significant difference at the 0.05 probability level. # The variables in this study include job title, prior work experience, tenure, education and salary prior to hire by O&M. This result is reproduced from 1983 Report, Table VII-3, p. 47. + The variables used in this study include department, prior work experience, tenure, education, age, time since degree, college major, other training, standing of college and military experience. ° All employees exempt from the overtime provisions of the Fair Labor Standards Act.

[31] See Pl. 1985 Report, Ex. 283, 2d Table D, entitled:

Table D

Average Male/Female Salary Differences for all Professionals and Managers

....

Plus All "High School" and "Some College"

[32] Table I (Excerpted from Pl. 1985 Report, Ex. 301) Salary Regressions Using Plaintiffs' Other Defendant's Using Training and Supplementary Plaintiffs' Suggested Report Variables Other College (excludes former Training Standing Source Year salary)+ Variable Variable -------------------------------------------------------------------------------------------- Corrected 1979 -1,331 -1,508 -1,731 Results (-1.29) (-1.47) (-1.69) with Student 1978 -1,712 -2,005* -2,122* t Values (-1.91) (-2.26) (-2.38) All Employees 1977 -2,313* -2,410 -2,443* (-2.61) (-2.74) (-2.77) 1976 -2,721* -2,680* -2,735* (-3.09) (-3.14) (-3.19) --------------------------------------------------------------------------------------------- * An asterisk indicates a statistically significant difference at the 0.05 probability level. + Variables used include department, prior work experience, tenure, education, age, time since degree, college major, other training, standing of college. Job title is not included.

[33] TABLE 14 (Def. 1985 Report) Average Female/Male Salary Differences All Employees ----------------------------------------------------------------------- 1979 $ -1056 1978 - 930 1977 -1451 1976 -1850* (2.23) 1975 -1075 ----------------------------------------------------------------------- * Indicates a difference that is statistically significant at the 0.05 probability level. The number in the parenthesis is the t statistic.

The variables used in this study include age, education, other training, time since degree, college major, standing of college, prior work experience, tenure, department and military experience.

This table was originally Exhibit D-698(c).

[34] TABLE C (Pl. 1985 Report, "2d Table C") Average Female/Male Salary Differences for all Professionals and Managers All Employees for Whom All Relevant Data Are Available (includes officers) --------------------------------------------------------------------- Year Number of Observations --------------------------------------------------------------------- 1979 $ -941 356 (-1.51) 1978 -1,603* 357 (-2.66) 1977 -1,942* 308 (-3.08) 1976 -1,958* 270 (-2.91) 1975 707 221 (0.80) --------------------------------------------------------------------- * Indicates a difference that is statistically significant at the 0.05 probability level.

The variables used in this study include department, job title, prior work experience, tenure, education, age, time since degree, college major, other training and standing of college.

[35] Defendant's 1985 Reply Table C+ is excluded from consideration. See n. 2, supra.

[36] 2d Table C includes all employees, regardless of dates of hire. 2d Table C does control for job title, however, which may screen out some effects of discrimination which is no longer actionable. See discussion supra at 1522-1524.

[37] The court does not treat as significant certain interview evaluation forms introduced by plaintiffs in which high-ranking O & M officials on a handful of occasions referred to the appearance of female job applicants or used other terms clearly related to gender, e.g., "an intelligent Betty Boop." Although the court does not condone such comments, it notes that in many instances the individual at issue actually was hired by O & M, and that in any event, such sporadic comments fell very far short of providing a meaningful indicator of discriminatory intent. Men applicants too were sometimes referred to by their appearance, which suggests occasionally sloppy interviewing practices, rather than a pattern of gender discrimination.