ABE R. BROTSPIES, THOMAS A. ENNIS, SALLY WICK, SABATINO ADDONIZIO, WILLIAM EUBANKS, MONA McDONALD, HERBERT LART, HARRY SMITH, MILDRED FOX AND STANLEY ZACH, PLAINTIFFS-APPELLANTS,
v.
DEPARTMENT OF CIVIL SERVICE OF THE STATE OF NEW JERSEY, DEFENDANT-RESPONDENT.
Superior Court of New Jersey, Appellate Division.
Decided April 4, 1961.*493 Before Judges GOLDMANN, FOLEY and KILKENNY.
Mr. Sidney Krieger appeared for plaintiffs-appellants.
Mr. William L. Boyan, Deputy Attorney General, appeared for defendant-respondent (Mr. David D. Furman, Attorney General, attorney).
GOLDMANN, S.J.A.D.
Plaintiffs, ten in number, have appealed from a decision of the Department of Civil Service declaring that the examination for Supervisor of Case Work, Essex County (PC 106) was held in accordance with the provisions of the Civil Service rules and regulations and found to be in good order, and declaring that those *494 candidates, among them plaintiffs, who failed to achieve a passing grade in the written test would not be permitted to participate in the oral test for the position.
Thereafter plaintiffs moved for leave to enjoin the Department from certifying to the employing authority a list of persons eligible for appointment to the position, and to obtain an order making the examination and the answer sheets part of the record to be filed with the Appellate Division pursuant to R.R. 4:88-8. The Department opposed both demands.
Plaintiffs then made a second motion for leave to present additional evidence to show how the questions used in the examination were prepared and the correct answers established, and to show that the answers considered correct by respondent Department were, in effect, not the correct answers. (The examination was one which offered a multiple choice answer for each question.) This motion, too, was opposed.
Eventually this court entered an order directing that plaintiffs be given the opportunity to examine the questions on the examination they took, at which time they were to state the numbers of those questions they considered improperly marked by respondent and to indicate which answer they believed to be correct with regard to each of them. The application contained in their second motion was denied.
The parties consented that a single judge sitting on behalf of the Appellate Division, conduct a hearing at which plaintiffs would specify the questions they found objectionable, and the reasons why the answers considered correct by the Department were in fact incorrect, and their answers correct. Such a hearing was held before me, at which plaintiffs Brotspies, Addonizio, Eubanks and Zach were present, as well as the Department's Director of Examinations, Boyd, and Technical Assistant of Examinations Mankovich. At that time plaintiffs stated that they objected to 10 of the 110 questions marked and the correctness of the Department's *495 answers. I was provided with a copy of the ten questions, together with the answers the Department considered correct and those which plaintiffs, on their part and by common agreement, considered correct. As to the last mentioned, it may be noted that in some instances the answers of some of the plaintiffs did not agree with the one which they now jointly agree is the right one as they view the matter.
The hearing stems from the Department's objection to making the examination questions and answers a matter of public record, open to inspection and copying by any member of the public. Publication of the questions and correct answers from any examination would, in its opinion, destroy valuable interests of the Department and impede its effective administration of the merit system.
The reason is found in the fact that individual questions are used on various examinations conducted by respondent. The Department keeps a history of each item and analyzes its value in the light of two factors. The first is the percentage of testees who answer the item correctly. If everyone answers correctly or, on the other hand, answers incorrectly, the item is considered as making no contribution. The second factor is the ratio of those who passed and who answered correctly, to the total number of those who answered correctly. The ideal value for this ratio is 1.00. Should the ratio be 0.00, the item is considered to be not only a failure, but a negative factor in discriminating between the qualified and the unqualified. From the point of view of the Department, the grading of answers to such an item would tend to give higher scores to persons determined to be unqualified as a result of the entire examination.
The history of items so recorded gives the Department a valuable source for questions of whose validity it may be confident. Frequently an item will prove to be worthwhile for several positions but of no value in determining qualifications for other positions.
Since the Department may desire to use many of the items if a test is held for a position of Supervisor of Case *496 Work in a county other than Essex, here involved, or for examinations for other positions, then once a particular set of questions and correct answers is published, all of the value resulting from the empirical evidence so carefully collated by the Department is destroyed.
At the hearing it developed that of the ten items which are the subject of plaintiffs' protest, all but one were prepared by the Division of State Merit Systems in the U.S. Department of Health, Education and Welfare. These, along with others, appear in test booklets distributed to the civil service systems of the several states. The records of the federal agency fail to indicate that any of the questions and answers have at any other time been challenged by candidates in any jurisdiction.
The preparation of the test materials furnished by the Division of State Merit Systems is a federal-state cooperative effort, involving both subject-matter and test specialists. These materials are reviewed at various stages of their development by both psychologists and boards of specialists in the appropriate fields. The Division's "Statement of Policies and Procedures for Test Services to the States," a copy of which is in evidence, describes the process in some detail:
"After the drafting and technical editing of the materials and prior to their issuance, they are reviewed in Washington from the standpoint of practicality and validity by a group of subject matter specialists who are usually Federal employees. Whenever possible assigned State agency specialists participate in this review. The steps in test construction are as follows: (1) concept outlines are developed cooperatively covering the areas of job knowledges, skills, and abilities to be sampled by written tests; (2) a subject matter specialist writes multiple-choice examination questions in draft form, on assigned concepts; (3) an examination specialist edits them; (4) committees of representative personnel, drawn from the Federal bureaus and offices, answer the questions as though they were candidates for jobs, then they write their comments on each question; (5) the comments are collated on a master sheet which shows what each committee member chose as the proper answer, what criticisms he may have on the item, what editorial changes he may have suggested, what he thinks about the appropriateness or inappropriateness of the question, and for what level *497 of difficulty he thinks it is suitable; (6) a specialist in examinations then meets with those who shared in the process of construction or editing, to discuss around a conference table the questions on which there is not complete agreement; (7) the questions surviving are subsequently made up into test booklets, according to subject and difficulty, which are then combined into appropriate examinations upon receipt of requests from the State merit systems."
The Statement of Policies and Procedures makes clear that the items supplied by the Federal Government on state examinations must be kept confidential "for the benefit of other States to which we may make the same materials available."
Plaintiffs invite the court to engage in a critical supervisory review of the composition of the questions propounded in the Supervisor of Case Work test, and to determine whether the Department's answers, or the answers upon which they have agreed, are the correct ones. The court is not disposed to exercise an appellate review in such a case. Involved is an administrative function "delegated most liberally to the authorized examiners of the Department by the Legislature." The courts will intervene to nullify such an examination only where "it is affirmatively shown to have been manifestly corrupt, arbitrary, capricious or conspicuously unreasonable." Artaserse v. Department of Civil Service, 37 N.J. Super. 98, 105 (App. Div. 1955), and cases there cited.
As was said in Flanagan v. Civil Service Department, 29 N.J. 1, 12 (1959), in another context:
"* * * `[c]ourts should let administrative boards and officers work out their problems with as little judicial interference as possible. They may decide a particular question wrong but it is their question. [They are] vested with a high discretion, and its abuse must appear very clearly before the courts will interfere.' Maxwell v. Civil Service Commission, 169 Cal. 336, 146 P. 869 (Sup. Ct. 1915). If there is any fair argument in support of the course taken or any reasonable grounds for difference of opinion among intelligent and conscientious officials, the decision is conclusively legislative, and will not be disturbed unless patently corrupt, arbitrary or illegal. Doubts held by the court as to the wisdom *498 of the administrator's decision do not alter the case. [Citing cases] * * *"
And see Mitchell v. McKevitt, 128 Cal. App. 458, 17 P.2d 789, 791 (D. Ct. App. 1932).
Kaplan, in his work on The Law of Civil Service (1958), observes (pages 158-9) that courts have generally been circumspect in reviewing ratings of examination papers and have confined their reviews solely to cases where there has been a clear showing that the competitive principle has been violated, or that the ratings are so arbitrary and unreasonable as to be indefensible. Such, for example, would be the case where the key answers adopted by the civil service commission were utterly erroneous and contrary to overwhelmingly acceptable standards.
"* * * If the question obviously called for an answer of `black,' and the commission persisted in viewing the correct answer as `white,' the court would set aside the commission's rating as an abuse of its authority. Where, however, the question appears to be plausible and the answer could be any shade of gray, the determination of what might be a reasonably correct answer is left in the discretion or judgment of the commission. The court will not weigh the judgment of so-called `outside experts' against the judgment of the technicians or specialists used by the commission as to the validity of the ratings. So long as there is reasonable basis for the technician's or specialist's determination, the findings of those vested by law to exercise judgment and discretion in weighing the propriety of answers to propounded questions in examinations is conclusive. A challenge to the propriety of the answers will not be indulged by judicial review."
We have carefully reviewed the questions and the answers considered correct by the respondent Department, as well as those claimed to be correct by plaintiffs' consensus. We perceive no reason for judicial intervention. Plaintiffs failed to establish that the questions were not fair and clearly put, or that they did not explore an area of knowledge and information appropriate to the Supervisor of Case Work position. The questions emanate from an entirely responsible and expert source, the Division of *499 State Merit Systems of the federal Department of Health, Education and Welfare, and are the result of careful preparation and pretesting. The answers considered correct by that Division, as well as by respondent Department which has adopted them, have not been shown to be unreasonable or implausible. We are persuaded that respondent exercised a reasonable administrative judgment in finding its answers more appropriate than those for which plaintiffs contend.
It might incidentally be noted that even were plaintiffs' agreed-upon answers accepted as correct, and those of the Department not correct, only one of the ten plaintiffs would score over 77, considered a passing grade for the examination. (110 questions were finally counted; 70% was used as the passing percentage, thus establishing 77 as the lowest passing grade.)
Accordingly, the court will accept the Department's questions and answers as correct. The appeal may proceed to a determination of any other questions raised by appellants in challenging the Department's decision that the examination for Supervisor of Case Work was in all respects in accordance with Civil Service rules and regulations and found to be in good order, and declaring plaintiffs to have failed to obtain a passing grade in the written test so as not to be eligible to take the oral examination.