846 F.2d 1373
William F. CURTIN, Petitioner,
v.
OFFICE OF PERSONNEL MANAGEMENT, Respondent.
No. 87-3192.
United States Court of Appeals,
Federal Circuit.
May 18, 1988.
Dennis L. Friedman, Philadelphia, Pa., for petitioner.
Robert A. Reutershan, Asst. Director, Commercial Litigation Branch, Dept. of Justice, Washington, D.C., argued for respondent. With him on the brief were Richard K. Willard, Asst. Atty. Gen., David M. Cohen, Director and Larry R. Steffes.
Before MARKEY, Chief Judge, FRIEDMAN and ARCHER, Circuit Judges.
ARCHER, Circuit Judge.
William F. Curtin petitions for review of the decision of the Merit Systems Protection Board (MSPB or board), Docket No. PH300A8610103, upholding the employment practices of the Office of Personnel Management (OPM) related to the examination for the position of administrative law judge (ALJ) in the federal government. Curtin challenges the examination under 5 C.F.R. Sec. 300.104(a) which provides that "[a] candidate who believes that an employment practice which was applied to him or her by the Office of Personnel Management violates a basic requirement in Sec. 300.103 is entitled to appeal to the ... Board."
I.
Curtin challenges the employment practices used by OPM to select persons for the position of ALJ, alleging that the ALJ examination violates the policies set forth in "Subpart A--Employment Practices" of 5 C.F.R. Part 300 (1988). These policies and requirements are contained in sections 300.102 and 300.103, which provide in relevant part:
Sec. 300.102 Policy.
This subpart is directed to implementation of the policy that competitive employment practices:
(a) Be practical in character and as far as possible relate to matters that fairly test the relative capacity and fitness of candidates for the jobs to be filled;
(b) Result in selection from among the best qualified candidates;
(c) Be developed and used without discrimination because of race, color, religion, sex, age, national origin, partisan political affiliation or other non-merit grounds; and
(d) Insure to the candidate opportunity for appeal or administrative review, as appropriate.
Sec. 300.103 Basic requirements.
(a) Job analysis. Each employment practice of the Federal Government generally, and of individual agencies, shall be based on a job analysis to identify:
(1) The basic duties and responsibilities;(2) The knowledges, skills, and abilities required to perform the duties and responsibilities; and
(3) The factors that are important in evaluating candidates....
(b) Relevance. (1) There shall be a rational relationship between performance in the position to be filled (or in the target position in the case of an entry position) and the employment practice used. The demonstration of rational relationship shall include a showing that the employment practice was professionally developed.
OPM is responsible for the recruitment and examination of ALJs. To carry out this activity, OPM has constructed and implemented a multi-step examination process. To establish minimum qualifications, applicants must demonstrate initially that they have had seven full years of administrative law or litigation experience at the federal, state or local level, part of which must be at grade or responsibility levels just below the ALJ position sought.
Applicants who possess the minimum qualifications are permitted to continue with the next portion of the examination in which applicants are ranked based on achievements described in a Supplemental Qualification Statement (SQS). According to the board, the SQS measures "achievements in five specific areas: (1) knowledge of rules of evidence and trial procedure; (2) analytical ability; (3) decision-making ability; (4) oral communications ability and judicial temperament; and (5) writing ability." Moreover, it noted that "applicants are advised to describe qualifying personal experience which will demonstrate they possess the knowledge, skills and abilities which OPM has declared essential for the successful performance of ALJ duties." Based on a rating guide, the applicants' achievements in the five areas are scored on a scale of 1 (unacceptable) to 5 (outstanding) and aggregated to establish a basic rating for successful applicants who achieve the minimum acceptable score.
Based on the SQS ratings and anticipated hiring needs, applicants are invited to participate in the final three-step examination process, consisting of a written demonstration, a personal interview, and a personal reference inquiry. Curtin concedes that this final rating process is "professionally and scientifically developed" and "content valid" as required by 5 C.F.R. Part 300.1
In this appeal, Curtin challenges only the SQS portion of the examination. He contends that
the SQS portion (a) doesn't measure what it purports to measure, (b) is not an accurate measuring instrument in that it cannot accurately measure an applicant's knowledge and abilities in the areas of knowledge of rules of evidence and trial procedures, analytical ability, decision making ability, oral communication ability, judicial temperament and writing ability, (c) contains a bias in favor of applicants who have more varied job experiences than those who do not, (d) is unfair, in that certain applicants are self-limited by positions they previously held and cannot achieve the full score available, (e) is unfair, in that verification of an applicant's achievements was an important aspect of the exam's scoring and evaluation and was not performed by OPM examiners, and (f) even if it were an accurate measuring instrument, which it is not, the total examination weight attributable to the SQS portion as compared to the three remainder portions of the examination is disproportionately high and totally lacking in scientific bias [sic: basis] or justification.
The SQS is a behavioral consistency method of unassembled examining. An unassembled examination is one which does not require a written test but relies upon evaluation of education and experience or, in this case, past achievements. The board stated that the principle of behavioral consistency is based on "the well-accepted notion that human behavior tends to be consistent" and that the behavioral consistency method has "been shown to be the most valid predictor of success in a particular endeavor." It also found that the method is "professionally acceptable" and that "there is convincing expert opinion that it is most appropriate in the circumstances here."2
The board concluded that
the employment practices under attack actually yield fairness and equality in employment selection. It is not at all unusual for employers, when considering applicants for employment, to review an applicant's training and experience in an attempt to predict the applicant's future job performance.... The use of benchmarks to measure past achievements enables the application of considered judgment by the rater because past actual experiences are being exalted over more passive criteria such [as] education.... The employment practices used by OPM have been shown to measure the probability of job success.
II.
The scope of review of decisions of the Merit Systems Protection Board is narrowly defined by statute. The board's decision must be affirmed unless it is found to be:
(1) arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law;
(2) obtained without procedures required by law, rule or regulation having been followed; or
(3) unsupported by substantial evidence.
5 U.S.C. Sec. 7703(c) (1982); Covington v. Dep't of Health and Human Servs., 750 F.2d 937, 941 (Fed.Cir.1984).
The employment practices of the Federal Government are required to be "based on a job analysis to identify: (1) The basic duties and responsibilities; (2) The knowledge, skills and abilities required to perform the duties and responsibilities; and (3) The factors that are important in evaluating candidates." 5 C.F.R. Sec. 300.103(a) (1988). And there must be a "rational relationship between performance in the position to be filled ... and the employment practice used," including proof that "the employment practice was professionally developed." 5 C.F.R. Sec. 300.103(b) (1988). The term "employment practices" includes "the development and use of examinations, qualification standards, tests, and other measurement instruments." 5 C.F.R. Sec. 300.101 (1988).
Curtin questions OPM's compliance with these provisions stating that the SQS portion of the examination contains "obvious, significant and substantial laws [sic: flaws] both in its construction and in its implementation." In doing so, however, he concedes that "the SQS exam was based on a job analysis which identifies the basic duties and responsibilities, the knowledges, skills and abilities required to perform the duties or responsibilities, and the factors that are important in evaluating ALJ candidates." He also acknowledges that in "the construction of the SQS exam, 84 critical job tasks of the ALJ job were identified, as well as 18 types of knowledge and ability essential for the performance for the 84 critical job tasks" and that a "panel of 8 law judges then determined the five types of knowledge and ability which should be evaluated by means of the SQS exam." He also does not challenge the board's finding that the SQS exam was professionally developed.
Curtin contends that "[i]dentifying the types of knowledge and ability and the critical job tasks ... is not synonymous with the identification of certain behaviors which are purportedly measured by the SQS portion of the examination." Curtin's position seems to be that although OPM has properly identified the knowledge, skills and abilities required to perform the duties of an ALJ, an objective test, or some test other than the one used, would have been a more valid measure of the ALJ requirements. He says that OPM has not identified what behaviors are measured by the SQS portion of the examination or addressed how the behaviors that may be measured by the SQS are related to the job analysis.
The board properly stated that it was irrelevant whether a test other than the SQS, such as an objective test, could have been used. In reviewing the challenged OPM employment practice, the board was not obligated to determine what might be the most preferable ALJ examination. It was only required to decide whether OPM's ALJ examination bears a rational relationship to the job performance of an ALJ and meets the employment practices criteria of Subpart A, 5 C.F.R. Secs. 300.101-104 (1988).
The board found that "OPM has established that the SQS portion [of the examination] measures 'achievements' that would reflect knowledge, skills and abilities, of ALJ applicants, which ... is what OPM purports it to measure." (Emphasis in original.) It observed that Curtin's discontent is apparently based on the fact that the SQS does not measure knowledge, skills and abilities of the applicants directly. In finding that an indirect measurement should not result in the invalidation of the examination, the board relied on evidence that the behavioral consistency method of candidate selection is at least comparable in validity to general mental ability tests. It also relied on a study which showed that work sample is the best predictor of success on the job and expert opinion testimony that the SQS method was the most appropriate for the position under consideration.
We are convinced that the board's findings are supported by substantial evidence. Curtin's first two contentions, i.e., the SQS does not measure what it purports to measure and is not an accurate measuring instrument, are therefore unpersuasive.
Curtin also challenges the SQS portion of the examination because OPM does not routinely verify the achievements claimed by each applicant and because the weights assigned to the various components of the examination are arbitrary.
The failure to routinely verify the achievements claimed by applicants was found by the board not to undermine the validity of the examination. The board said that:
research shows that applicants seldom falsify claims which are readily verifiable. Moreover, applicants are considered to be honest and to give a full picture. Accordingly, raters generally rely on what applicants tell them.... On the flip side, claims prone to be falsified are difficult to verify. More importantly there is also evidence that false claims would not effect [sic] the validity of this particular examination because other portions of the examination are likely to reveal discrepancies or inconsistencies.
The board, thus, heard testimony and gave ample consideration to Curtin's arguments concerning the lack of verification of the achievements listed by applicants on the SQS portion of the examination. Its findings are supported by substantial evidence and we have been directed to no evidence to the contrary.
The board found that "the scoring procedure ... is consonant with accepted professional principles and is rationally based." The board explained that "the rationale upon which OPM based its rating procedure was that the greater the validity of a particular part of the examination, the more weight should be assigned that part." The board went into a detailed analysis of the procedure used to determine the scoring and found "OPM's assignment of weights for the ALJ examination to be consistent with accepted principles of the profession." The board in its analysis gave careful consideration to the expert testimony presented to it. Again, the board's determination is supported by substantial evidence.
Curtin's remaining contentions--(1) that the SQS contains a bias in favor of applicants who have more varied job experience, and (2) that certain applicants are self-limited by positions they previously held--were adequately considered by the board. While Curtin contends that applicants are "self-limited to a benchmark achievement" in certain areas, pointing particularly to the areas "of oral communications ability and judicial temperament," the board found "there is worthy testimony that the benchmarks used ... are useful in evaluating those essential characteristics."
The board observed that the "final numerical score given candidates is based upon the complete examination" which includes the written and oral portions of the examination process. It said there are varied ways of evaluating specific skills incorporated into the testing procedure. Moreover, the board discussed examples of how the SQS would be useful in evaluating the particular skills which Curtin contends would not be adequately measured. Given the fact that the entire examination, not just the SQS portion, is used to evaluate applicants, Curtin's contentions, if true, regarding bias based on more varied job experience or certain applicants being self-limited would not be sufficient to undermine the validity of the SQS portion of the examination. The SQS bears a rational relationship to performance in the position to be filled and is a valid employment practice considering the examination as a whole. Although the benchmarks of the SQS may not have provided a perfect measurement of certain skills, such as oral communication, judicial temperament or the like, the board clearly found that they were "useful in evaluating those essential characteristics."
Reduced to basics, Curtin's challenge is that OPM might have developed a better technique in one portion of the examination process for assessing the behavior of applicants for the position of ALJ. However, the board does not sit to second guess the judgments of the professionals who developed the examination for OPM, nor does this court. The board and this court are required only to assure that there is a "rational relationship between performance in the position to be filled ... and the employment practice used" including proof that "the employment practice was professionally developed." 5 C.F.R. Sec. 300.103(b) (1988). The board found that these requirements were satisfied. It stated:
The uncontradicted evidence is that the test has been developed and applied in a manner wholly in consonance with professionally accepted examination development and validation principles.... Preparatory to adoption of this test, OPM conducted a job analysis. It systematically examined the job of an ALJ in the context in which it is performed and then choose [sic] a selection technique thoroughly in accord with professionally sanctioned concepts for this type of candidate selection. Viewed in its totality, the ALJ examination is a valid measure of fitness of the candidates.
The board concluded that
the preponderant evidence attests that the employment practice in question--the ALJ examination--is based upon a job analysis which identifies (1) the basis [sic] duties and responsibilities, (2) the knowledge, skills and abilities required to perform the duties and responsibilities; (3) the factors that are important in evaluating candidates and that there is a rational relationship between performance in the position to be filled and the employment practice used.
We are convinced that the board properly analyzed and applied the policies and requirements of the regulations relating to employment practices to the facts and that its findings are supported by substantial evidence. See 5 U.S.C. Sec. 7703(c) (1982).
III.
Curtin contends that various rulings by the MSPB with respect to discovery and evidentiary matters constituted an abuse of discretion and prejudiced his right to a fair and impartial hearing.
Procedural matters relative to discovery and evidentiary issues fall within the sound discretion of the board and its officials. Spezzaferro v. FAA, 807 F.2d 169, 173 (Fed.Cir.1986); 5 C.F.R. Sec. 1201.41(a) and (b) (1988). This court will not overturn the board on such matters unless an abuse of discretion is clear and is harmful. Spezzaferro v. FAA, 807 F.2d at 173. If an abuse of discretion did occur with respect to the discovery and evidentiary rulings, in order for petitioner to prevail on these issues he must prove that the error caused substantial harm or prejudice to his rights which could have affected the outcome of the case. Cornelius v. Nutt, 472 U.S. 648, 657-659, 105 S. Ct. 2882, 2888-2889, 86 L. Ed. 2d 515 (1985); Smith v. United States Postal Serv., 789 F.2d 1540, 1545 (Fed.Cir.1986); 5 C.F.R. Sec. 1201.56(b)(1) (1988); see 5 U.S.C. Sec. 7701(c)(2)(A) (1982).
Curtin alleges that several rulings with respect to discovery denied him a fair and impartial hearing. In particular, he points to the Administrative Law Judge's (ALJ) refusal to compel responses to his first and second sets of interrogatories and document requests. The ALJ carefully considered Curtin's motions to compel and in his analysis provided convincing reasons for refusing to compel responses to particular discovery requests. For example, in his order of April 15, 1986, the ALJ refused to compel answers to some interrogatories because they were not relevant and to others because they were unduly burdensome.
Curtin also alleges that the ALJ "failed to make prompt rulings on discovery ... failed to sanction OPM for its abusive and dilatory tactics and deliberately avoided making evidentiary rulings on discovery." These all relate to matters that are obviously within the sound discretion of the ALJ. Although certain documentary evidence and other information requested by Curtin apparently was not received until shortly before the hearing, Curtin has not shown how he was prejudiced by those rulings or how the alleged delay might have affected the outcome.
The ALJ in all of his discovery rulings was faced with difficult decisions and was required to weigh Curtin's right to the requested information, the relevance of voluminous amounts of material requested and the legitimacy of the government's claims of privilege. It is apparent from the record that the ALJ's rulings reflected an awareness and a balancing of the interests of the parties and an effort to assure a fair and expeditious hearing of the case.
Curtin also argues that it was an abuse of discretion not to qualify Leon Silverman as an expert. However, the ALJ stated that "I do not find that Mr. Silverman's qualifications--training, education and experience--are sufficient to qualify him as an expert in the field of validation of applicants for the position of federal administrative law judge." The ALJ thus considered the pertinent factors in determining whether to qualify Mr. Silverman as an expert and we are not convinced that his ruling was wrong. Further, the ALJ, after reviewing Mr. Silverman's proferred deposition testimony, noted that, even if he were so qualified, it would not have affected his findings regarding the ALJ examination or the ultimate outcome of the case. It is apparent, therefore, that the ALJ's ruling, even if held to be incorrect, was not harmful error.
We have considered the other arguments put forth by Curtin regarding the discovery and evidentiary issues and, under all the circumstances, we are persuaded that Curtin has failed to demonstrate that the ALJ abused his discretion in the rulings he made. Moreover, he has not carried his heavy burden of showing that the alleged procedural errors were prejudicial and could have changed the outcome of the proceedings below. See Cornelius v. Nutt, 472 U.S. at 658, 105 S.Ct. at 2888-2889; Smith v. United States Postal Serv., 789 F.2d at 1545.
Accordingly, the decision of the MSPB is
AFFIRMED.
Curtin's brief states:
The evidence of record establishes that the written demonstration portion, personal interview portion and personal reference inquiry portion of the examination are professionally and scientifically developed, are content valid (in psychology terms, a content valid examination is one that samples the kinds of achievements required in the performance of the job), highly structured, and greatly improved and refined versions of the garden variety written demonstration, personal interview and personal reference inquiry tests which had been used in the past.
The board also noted that "OPM considered alternative testing strategies including objective testing, but decided that other methods were not appropriate for testing candidates for the position of administrative law judge." The board then found that "[w]hile an objective test may be quite appropriate when selection is at the entry level of a profession ... its merits are somewhat diminished in testing professionals at more advanced stages of their careers."