*140Judge JACOBS concurs in the opinion of the court and in a separate concurring opinion.
WALKER, Circuit Judge:Gene Giaeumbo appeals from the order of the United States District Court for the Southern District of New York (David N. Edelstein, District Judge), United States v. Int’l Bhd. of Teamsters, No. 88 Civ. 4486 (S.D.N.Y. July 28, 1997) (“Giaeumbo III”), affirming the May 15, 1997 decision of the Independent Review Board (“IRB”), an investigative and disciplinary body established under the consent decree entered against the International Brotherhood of Teamsters (“IBT”) in United States v. Int’l Bhd. of Teamsters, 88 Civ. 4486 (S.D.N.Y. Mar. 14, 1989) (“Consent Decree”). The IRB had found that Giaeumbo violated the Consent Decree and recommended that he be barred for life from union membership and prohibited from receiving any direct or indirect payments from the IBT or its affiliates, except money accrued and benefits from employers resulting from collective bargaining agreements. The district court also upheld IRB member Lacey’s determination not to recuse himself. Giaeumbo now appeals.
This appeal is one in a long series of cases arising under the Consent Decree that resulted from a 1989 settlement of an action between the United States and the IBT brought under the Racketeer Influenced and Corrupt Organizations Act (“RICO”), 18 U.S.C. § 1964. We have summarized the history of the RICO action in several prior opinions, see, e.g., United States v. IBT (“Simpson”), 120 F.3d 341, 343 n. 1 (2d Cir.1997); United States v. IBT (“1991 Election Rules”), 931 F.2d 177, 180-81 (2d Cir.1991), and the district court has presented a detailed background of the facts leading up to this case, see United States v. IBT, 951 F.Supp. 1113, 1116-28 (S.D.N.Y.1997) (“Giacumbo II”). We assume familiarity with the foregoing renditions and will only briefly recapture their salient features.
As part of its efforts to root out organized crime from the IBT, the United States filed a RICO action against the IBT, its General Executive Board, the eighteen board members, the Commission of La Cosa Nostra and individual members and associates of La Cosa Nostra. See 1991 Election Rules, 931 F.2d at 180. The government’s allegations against the IBT, the IBT Board and the Board members were settled by the Consent Decree, which instituted sweeping reforms of the IBT’s electoral and disciplinary processes, including the creation of the IRB to determine IBT disciplinary matters. See United States v. IBT (“IRB Appointee”), 12 F.3d 360, 362 (2d Cir.1993). The IRB consists of three members: one chosen by the Attorney General of the United States, one by the IBT, and a third chosen by the first two IRB designees or in the absence of their agreement, by the district court. The current IRB members are Grant Crandall, Frederick B. Lacey, and William H. Webster. The IRB has broad investigatory powers and the power to hold hearings in certain circumstances. If the IRB uncovers what it believes to be impermissible conduct within the IBT, it recommends that the IBT proceed against the alleged wrongdoers. The outcome of IBT disciplinary proceedings are subject to review by the IRB and, if the IBT fails to take appropriate measures, the IRB may act in its stead. IRB determinations, in turn, are reviewable by the district court.
Since this appeal raises questions in two separate areas, the IRB’s sanctions against Giaeumbo and IRB member Lacey’s refusal to recuse himself, each will be dealt with in turn.
I. The Sanctions Against Giaeumbo
Background
On March 20, 1995, the IRB issued an investigative report recommending that three charges be brought against Giaeumbo, who was at the time an IBT Vice President and former President and principal officer of Local 843. The first alleged that Giaeumbo violated the IBT Constitution and brought reproach upon the IBT by
allowing] employees of Always Travel, including a friend of [his], to join Local 843 without negotiating any collective bargain*141ing agreement on their behalf, but in an effort to assist their employer to obtain the IBT travel contract and to allow such members to be eligible to vote for [him] in the 1992 Local 843 election....
The second alleged that Giacumbo brought reproach upon the IBT by “engaging] in a pattern of violating financial control provisions in the Local Bylaws, the IBT Secretary Treasurer’s Manual and the IBT Constitution” inter alia by using a Union credit card for personal expenses, by using a single signature on several checks rather than the required dual signature, and by purchasing computer equipment without prior approval. The third charge alleged that Giacumbo violated the IBT constitution by “failing] to return to the IBT $1,600 in monthly car allowances [he] received between February 1992 and May 1992 when the Local was also paying [his] car expenses thereby breaching [his] fiduciary duties and embezzling and converting $1,600 of IBT funds to [his] own use.... ”
The IRB heard the charges against Gia-cumbo on May 24, 1995, and on October 31, 1995, issued an opinion and order finding Giacumbo guilty as charged. The IRB ordered a sanction of a six month suspension from IBT membership to run concurrent with a six month suspension from his position in the union and a $1,600 fine (“Sanction I”). The IRB submitted its decision to the district court for review pursuant to the Rules and Procedures for Operation of the Independent Review Board for the IBT (“IRB Rules”). Without considering the merits, the district court noted the charges and summarily remanded to the IRB for reconsideration of the sanctions in light of the severity of the charges (“Remand I”). United States v. IBT, 910 F.Supp. 139 (S.D.N.Y.1996) (“Giacumbo I”).
The IRB conducted a supplemental hearing on March 21, 1996. It found that Gia-cumbo had violated his suspension by attending a Local meeting for the nomination of delegates to the 1996 IBT International Convention and by allowing himself to be nominated. The IRB also found that because Gicaumbo failed to pay the $1,600 fine, .he had shown disrespect for the sanctions and that the supplemental hearing “confirmed that [Giacumbo] sees himself as above the rules.” Opinion and Decision of the IRB at 3 (May 1,1996).
In light of Remand I’s directive to reconsider the sanctions and the additional findings made during the supplemental hearing, the IRB imposed sanctions of a three year suspension from IBT membership, prohibition from holding any position in the IBT or related entity during that period, and reaffirmed the $1,600 fine (“Sanction II”). The IRB again submitted its decision for review by the district court.
This time, the district court reviewed the merits and affirmed the IRB’s findings and conclusions except for the general finding that Giacumbo “sees himself as above the rules.” But the district court found the sanctions still too lenient and on January 9, 1997, again remanded the sanctions for reconsideration (“Remand II”). See Giacumbo II, 951 F.Supp. at 1160. Remand II was accompanied by the district court’s observation that “this Court deems it proper for the IRB to contemplate the wisdom of ever permitting Giacumbo to hold a position of influence within the IBT or any IBT-affiliated entity, given his documented history of misconduct and his startling arrogance of power.” Id.
Following Remand II, the IRB issued an opinion and decision on May 15, 1997, based on the findings made during the initial and supplemental hearing, a letter dated February 3, 1997 from the Chief Investigator and information solicited from Giacumbo’s attorney. The IRB found that, based on his “presence in Philadelphia during the IBT convention in July 1996 when he met with at least one Teamster official” and “his ongoing contacts with an IBT official,” Giacumbo “displayed his continuing disregard for the discipline imposed on him by participating in union affairs and the union political processes.” The IRB found that Giacumbo still had failed to pay the $1,600 fine, and that all of these violations, together with the prior violation of permitting himself to be nominated for IBT election, amounted to disrespect of the IRB and flaunting of the Consent Decree. These findings, in the wake of the district court’s earlier statement that the *142IRB “contemplate the wisdom of ever permitting Giacumbo to hold a position of influence within the IBT or any IBT-affiliated entity” resulted in the IRB’s final sanction. The IRB ordered Giacumbo’s permanent suspension from IBT membership, and barred any IBT-affiliated entity from paying Gia-cumbo any benefits. This time the district court issued an order affirming the IRB’s sanctions. See Giacumbo III. This appeal followed.
Discussion
In affirming past decisions of the district court reviewing IRB determinations, we have found it unnecessary to determine the appropriate standard for review since we would affirm under “any reasonable standard of review.” See, e.g., Simpson, 120 F.3d at 346 (citing United States v. IBT (“DiGirlamo”), 19 F.3d 816 819-20 (2d Cir.1994)); United States v. IBT (“Adelstein”), 998 F.2d 120, 124 (2d Cir.1993) (citing United States v. IBT (“Friedman & Hughes"), 905 F.2d 610, 617 (2d Cir.1990)); United States v. IBT (“Cimino”), 964 F.2d 1308, 1311 (2d Cir.1992) (same). Since this case does not admit of an easy affirmance “under any reasonable standard,” we must now determine the appropriate standard of review.
While we have acknowledged that the district court in many respects is in a unique position to administer a consent decree and interpret its terms, see United States v. Local 359, United Seafood Workers, 55 F.3d 64, 68-69 (2d Cir.1995), and have adverted to the possibility that “the district court’s decisions implementing the Consent Decree are entitled to the same deference as those of the Administrator,” see Friedman & Hughes, 905 F.2d at 616, upon closer consideration, we now believe that the district court should not be accorded any greater deference in its review of IRB applications than it would be accorded when it reviews agency actions under the Administrative Procedure Act (“APA”). Both the Consent Decree and the IRB Rules speak to the question of the standard of review by the district court: “In reviewing actions of the IRB, this Court shall apply the same standard of review applicable to review of final federal agency action under the [APA].” IRB Rules, Art. O; Consent Decree, Art. K (same). See also Simpson, 120 F.3d at 346.
Nothing in the IRB rules mandates heightened deference by the Court of Appeals to the district court. The rules govern, inter alia, the relationship between the district court and the IRB, but they do not speak to the Court of Appeals. In none of the numerous eases arising from the Consent Decree have we held that the rules limit our review of the district court’s review of IRB sanctions. The district court is in no better position than are we to review IRB decisions. “[T]he district court [is required] to treat decisions of the [IRB] with ‘great deference’,” DiGirlamo, 19 F.3d at 820; Friedman & Hughes, 905 F.2d at 616-17, and is not entitled to make independent factual findings, but is limited to the record before it. See SEC v. Chenery Corp., 332 U.S. 194, 196, 67 S.Ct. 1575, 91 L.Ed. 1995 (1947) (“Chenery II”). We believe that to accord the district court any heightened deference, on top of the heightened deference that court owes to the IRB, would serve no useful purpose and would conflict with the model of review of agency action which the Consent Decree and IRB Rules Article O contemplate. Accordingly, we will review the district court’s review of the IRB using the standards that would apply if the district court were reviewing the determinations of an administrative agency.
When considering a district court’s review of an agency action “we give no deference to the lower court [but] review de novo whether the agency action satisfies the standards of the APA. In other words, [like the district court,] we review the record to determine whether the agency considered the relevant factors and acted within its discretion.” Aqurius Marine Co. v. Pena, 64 F.3d 82, 87 (2d Cir.1995) (internal quotes and citations omitted). In fact, without expressly stating so, we have already applied this standard. In United States v. IBT (“Wilson"), 978 F.2d 68, 74 (2d Cir.1992), we placed ourselves in the shoes of the district court and effectively exercised de novo review when we reinstated the administrator’s choice of sanction, rather than remanding to the district court, after *143concluding that the district court erroneously substituted its chosen sanctions for those of the administrator.
What then are the standards for our review of IRB determined sanctions? Our review, like that of the district court, must be of a narrow scope, because this is an area where the IRB has been given wide discretion. See Wilson, 978 F.2d at 73. “The APA permits us to set aside the agency action only if it is ‘arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law.’ 5 U.S.C. § 706(2)(A). Although narrow, appellate review of an administrative record must nonetheless be careful, thorough and probing.” Ward v. Brown, 22 F.3d 516, 521 (2d Cir.1994).
We review the IRB’s findings of facts for “substantial evidence” on the whole record. See DiGirlamo, 19 F.3d at 820; Cimino, 964 F.2d at 1311. The substantial evidence test is deferential. “[T]he possibility of drawing two inconsistent conclusions from the evidence does not prevent an administrative agency’s finding from being supported by substantial evidence.” Illinois Cent R.R. v. Norfolk & W. Ry., 385 U.S. 57, 69, 87 S.Ct. 255, 17 L.Ed.2d 162 (1966) (quoting Consola v. Federal Maritime Comm’n, 383 U.S. 607, 620, 86 S.Ct. 1018, 16 L.Ed.2d 131 (1966)). See also Simpson, 120 F.3d at 346-47. Substantial evidence “is more than a mere scintilla [and] means such relevant evidence as a reasonable mind might accept as adequate to support a conclusion.” Chrysler Corp. v. U.S. Envtl. Protection Agency, 631 F.2d 865, 890 (D.C.Cir.1980) (internal quotation and citation omitted); see also Simpson, 120 F.3d at 346 (applying same standard).
Assuming that the agency’s findings of fact are supported by “substantial evidence,” inferences based on those findings are discretionary and can only be disturbed if they are “arbitrary and capricious.” When reviewing inferences, we are “oblige[d] ... to guard against an agencyf] drawing inferences that are ‘arbitrary’ in relation to the facts found, no matter how substantial may be the support for those facts.” Midtec Paper Corp. v. United States, 857 F.2d 1487, 1498 (D.C.Cir.1988).
When reviewing sanctions, our inquiry is limited to “whether the [IRB] made ‘an allowable judgment in [its] choice of the remedy.’ ” Wilson, 978 F.2d at 73 (quoting Butz v. Glover Livestock Comm’n Co., 411 U.S. 182, 189, 93 S.Ct. 1455, 36 L.Ed.2d 142 (1973)). We may only set aside the IRB’s choice of sanctions where the sanctions are “unwarranted in law” or “without justification in fact.” Butz, 411 U.S. at 186, 188, 93 S.Ct. 1455.
Imposing a sanction based on an improper understanding of the law is always an abuse of discretion, see Mendenhall v. NTSB, 92 F.3d 871, 874 (9th Cir.1996); Jaimez-Revolla v. Bell, 598 F.2d 243, 246 (D.C.Cir.1979)(per curiam), rendering the sanction arbitrary and caprieiouá and therefore impermissible. See SEC v. Chenery Corp. 318 U.S. 80, 94, 63 S.Ct. 454, 87 L.Ed. 626 (1943)(“Chenery I”)(“[A]n order may not stand if the agency has misconceived the law.”); Chenery II, 332 U.S. at 196, 67 S.Ct. 1575 (“If those grounds [invoked by an agency] are inadequate or improper, the court is powerless to affirm the administrative action-”). This is true even though the administrative agency, or in this case the IRB, permissibly could have imposed that very same sanction if it had done so. Compare Chenery I, 318 U.S. at 95, 63 S.Ct. 454 (remanding to Commission for further proceedings consistent with opinion) with Chenery II, 332 U.S. at 196, 209, 67 S.Ct. 1575 (accepting an identical result based on the SEC’s properly recast rationale). Said a bit differently, if a sanction is imposed as a result of an exercise of discretion that is impermissibly limited as a result of a mistaken view by the agency as to the scope of its discretion or an impermissible narrowing of the agency’s discretion by the district court, there has been a misunderstanding of law and the sanction must be vacated as arbitrary and capricious.
Examining the whole record, we uphold the IRB’s findings of fact upheld by the district court in Remand II for substantially the reasons stated by the district court. *144However, the inference that Giacumbo violated his suspension based on his “presence in Philadelphia during the IBT convention in July 1996 when he met with at least one Teamster official” and “his ongoing contacts with an IBT official” is arbitrary and capricious and must be vacated. Giacumbo was under no prohibition from associating with IBT members or officers or from being in Philadelphia during the IBT convention. The record does not establish that Giacumbo gained access to meetings open only to IBT officers or members, or otherwise acted or spoke with the apparent authority of an IBT officer or member.
Finally, we conclude that the final sanctions imposed on Giacumbo by the IRB cannot stand. Giaeumbo’s lifetime ban from IBT membership and office did not occur in a vacuum, but resulted from the district court’s repeated remand orders that effectively, if not explicitly, required the IRB to impose ever stricter sanctions. The district court buttressed its successive remands with a series of arguments that do not withstand scrutiny.
Remand I did not consider the merits of the case, but was based purely on the notion that “the sanctions imposed by the IRB are not commensurate with the severity of the charges levelled against Giacumbo.” Giacumbo I, 910 F.Supp. 139 (S.D.N.Y.1996). This was an improper basis for overturning the IRB’s choice of sanctions. The court must sustain the IRB’s determination “unless it finds the penalty ‘unwarranted in law1 or ‘without justification in fact.’ ” Wilson, 978 F.2d at 73 (quoting Butz, 411 U.S. at 185-86, 93 S.Ct. 1455), in other words, after a merits review. The IRB’s choice of sanction is not only predicated on the charges proven, but also on the reproach brought upon the IBT. We have no doubt that it was within the discretion of the IRB, upon considering the events leading to the charges and the harm caused to the IBT, to impose a six month suspension as a sanction even upon these charges.
When it rejected Sanction II, a three year membership suspension and $1,600 fine, and remanded for reconsideration, the district court did review the merits. But its decision, embodied in Remand II, was based upon comparisons with other cases, leading the district court to conclude that a three year suspension was so lenient as to be arbitrary and capricious. Uneven application of sanctions does not normally render the sanction imposed in a particular case arbitrary and capricious. See Butz, 411 U.S. at 187, 93 S.Ct. 1455; Wilson, 978 F.2d at 73-74 (finding that is was error for the district court to set aside a five year sanction based on reasoning that it was much milder than the lifetime suspension imposed in comparable cases, absent a finding that the sanction was arbitrary and capricious).
At the supplemental hearing, after reviewing the whole record, the IRB concluded that “when viewed on a relative scale the absolute consequence of the misconduct on the membership is not as great as perhaps in some other cases_” We cannot fault this conclusion, given that the $1,600 in double car reimbursements Giacumbo was found to have embezzled did not result from Giacumbo circumventing any financial controls. When he did circumvent financial controls, no unjust enrichment resulted except for the admittedly small gain resulting from his purchase of a computer game book.
The choice of appropriate sanctions for impermissibly permitting Allways to join the union was also peculiarly within the purview of the IRB. Increasing union membership is usually a laudable goal. We think the IRB should have wide discretion in choosing if and how to censure actions that normally benefit the IBT, but which are improperly motivated.
The instant case is therefore readily distinguishable from the cases cited by the district court. Cf. United States v. IBT (“Ligurotis”), 814 F.Supp. 1165 (S.D.N.Y.1993) (union official permanently barred for hiring convicted felons, possessing a handgun on the Local’s premisses, granting himself an unauthorized loan of $76,250, embezzling $77,000, and engaging in a pattern of conduct fostering corruption); United States v. IBT (“Caldwell”), 831 F.Supp. 278 (S.D.N.Y.1993) (union official permanently barred from IBT office for embezzling $3,231.25 in union funds *145by using union funds to pay legal expenses for two IBT members refusing to answer Independent Investigator’s questions despite receiving a letter warning him not to do so); United States v. IBT (“Morris & McNeil”), 777 F.Supp. 1123, 1124-25 (S.D.N.Y.1991) (upholding five year suspension of IBT officials based on their use of a variety of devices, including back-dated petitions to amend the local bylaws and falsified minutes to fraudulently grant themselves a $60,000 raise); United States v. IBT (“Perrucci”), No. 88 Civ. 4486 (S.D.N.Y., Dec. 11, 1996) (office manager agreed to resign from any IBT or Local office, 10 year membership suspension, and lifetime ban from holding office in IBT or IBT affiliates based on a charge that she embezzled $10,000 by misusing credit card and issuing unauthorized checks to herself).
In fact, if we compare the circumstances and sanctions in this case with those in Morris & McNeil, 777 F.Supp. at 1124-25 and Wilson, 978 F.2d at 71 (officials suspended from position for five years embezzling $5,626 in local funds by circumventing financial safeguards), it is evident that there was nothing arbitrary and capricious about Giaeumbo’s six month suspension, much less the three year suspension. Those cases imposed five year suspensions from any IBT or Local office due to embezzlement effectuated by circumventing financial controls and in amounts considerably greater than in this case. Giacumbo’s six month suspension as a union official was shorter, but he was also suspended from IBT membership, which prevented him from running for office in the upcoming election, and ordered to repay the $1,600. We have been given no reason to doubt that the IRB imposed its first sanctions after considering the harm to the IRB, the facts surrounding the violations, the character of the charges and the appropriate type of sanction in deciding how to deal with the violations. As we have said, the district court erred in vacating the first sanctions without a merits review. It was also error for the district court to substitute its judgment for that of the IRB following its merits determination of the three year membership suspension of Sanction II. Finally, the district court plainly exceeded its appellate role on the second remand when it suggested sanctions to the IRB.
The net result of all that occurred is that the progression of district court reviews misled the IRB into believing that it had to impose sanctions within a much narrower and stricter range than in fact it was permitted and obliged to consider. Inexorably, this led to a final sanction that was based on a mistake of law and thus was arbitrary and capricious.
It seems evident that the district court was trying to accomplish indirectly what we previously found impermissible in Wilson where the district court directly imposed a lifetime ban. The district court strongly suggested what it believed an appropriate sanction should be: “this Court questions Giacumbo’s fitness to ever again hold a position of trust, influence, or authority within the IBT, its Local Unions, or any other IBT-affiliated entity.” Giacumbo II, 951 F.Supp. at 1160.
In Wilson, after vacating the sanctions imposed by the district court, we were able to reinstate the original sanctions. We cannot do so here, however, because the IRB is permitted to consider conduct subsequent to the imposition of sanctions. Cf. Wilson, 978 F.2d at 74. We therefore vacate the IRB’s second and third sanctions and remand to the district court with instruction to remand to the IRB for it to consider whether the first sanction should stand or be modified due to any subsequent misconduct by Gia-cumbo in violation of the IRB’s October 31, 1995 order.
II. The Recusal Request
Background
Nine days before the first IRB Hearing on May 24, 1995, Time Magazine published an article critical of IRB member Lacey entitled “A Teamsters Tempest: Was a retired judge too soft in his handling of an investigation of labor boss Ron Carey?” Richard Behar, A Teamsters Tempest, Time, May 15, 1995, at 45. The article stated that “Time has obtained a private letter written by [Judge] Lacey in April 1994, amid his investigation of Carey, that raises questions about his impar*146tiality.” In the letter to Thomas Puedo, a court-appointed trustee of a Teamsters local, Lacey reminded Puccio—who had threatened to publicize information regarding Carey’s mafia ties—that
I told you that I thought you ... ought to have in mind what would happen if you brought Carey down in that there were ‘old guard’ teamsters throughout the country that were hoping that Carey would be eliminated as a candidate in 1996 so that the clock could be turned back to what it was when I first came on the scene as Independent Administrator.
Id. The IRB later acknowledged that Gia-cumbo, during his IBT Vice Presidency following his election on the Carey ticket, was a source of information that Carey had ties to organized crime, see Independent Review Board Report of Investigation of General President Ronald Carey, 41, 44 (July 11, 1994), and Giacumbo had been quoted in an earlier Time Magazine article as saying: “ ‘I believe [Carey has] been dancing with more than one partner—the Federal Government and the underworld-just like [former Teamster boss] Jackie Presser.’ ” Edward Barnes & Richard Behar, Rich Man, Poor Man, Time, Apr. 11,1994, at 42.
On May 16, 1995, Giacumbo wrote a letter to John Cronin, the administrator of the IRB. He brought to Cronin’s attention the May 15 Time magazine article, asserted that Lacey was biased in Giacumbo’s case, and requested his recusal. When Lacey refused to do so, Giacumbo moved in the district court for a hearing on the question. The district court denied that application on May 22, 1996, finding that Lacey was not involved in the events underlying the charges against Giacumbo. In so finding, the district court applied the “involved officer or member” standard set forth in the IBT Constitution, art. XIX, § 1(a), and expressly declined to adopt either the “appearance of partiality” standard applicable to federal judges, see 28 U.S.C. § 455(a), or the “evident partiality” standard used in arbitration hearings, see 9 U.S.C. § 10(a)(2). See Giacumbo II, 951 F.Supp. at 1129-30.
Discussion
Whether the correct recusal standard was employed is a question of law which we review de novo. See Morelite Constr. Corp. v. New York City Dist. Council Carpenters Benefit Funds, 748 F.2d 79, 82-84 (2d Cir.1984). IRB members are not United States Judicial Officers and are therefore not subject to the “appearance of partiality” test set forth in 28 U.S.C. § 455(a) which requires recusal by a judge “in any proceeding in which his impartiality might reasonably be questioned.” Instead, the governing recusal standard is determined by the IRB Rules, which establish that “[t]he hearings shall be conducted under these Rules and the rules and procedures generally applicable to labor arbitration hearings.” IRB Rules, Art. J.6. In labor arbitration hearings, the “evident partiality” standard applies. See Morelite, 748 F.2d at 82-84 (“evident partiality” standard applied to arbitrator in a proceeding to enforce a collective bargaining agreement); Local 814, IBT v. J & B Sys. Installers & Moving, 878 F.2d 38, 40 (2d Cir.1989) (per curiam) (standard applied to arbitrator in grievance proceeding). This is the same standard that applies to arbitrators generally under 9 U.S.C. § 10(a)(2), allowing federal courts to vacate an arbitration award “[w]here there was evident partiality ... in the arbitrators, or either of them.” Thus, we believe that the standard by which Lacey’s refusal to recuse must be determined is “evident partiality” and that the district court therefore erred in using the more lenient “involved officer or member” standard found in art. XIX 1(a) of the IBT Constitution.
The “evident partiality” standard is less strict than the “appearance of partiality” standard applicable to judges. See Local 814, 878 F.2d at 40; Morelite, 748 F.2d at 83. “The test for an appearance of partiality is ... whether an objective, disinterested observer fully informed of the facts underlying the grounds on which recusal was sought would entertain a significant doubt that justice would be done in the case.” DeLuca v. Long Island Lighting Co., 862 F.2d 427, 428-29 (2d Cir.1988) (internal quotation and citation omitted). Before recusal is required under the “evident partiality” test, a party *147must demonstrate that a “reasonable person would have to conclude that [the] arbitrator was partial to one party to the arbitration.” Morelite, 748 F.2d at 84. Partiality need not be actually proven to be the basis for a reasonable conclusion of partiality, but it may not be based simply on speculation either. See Local 814, 878 F.2d at 41. An integral part of the reasonable person test is an assessment of the facts established by the parly alleging bias. See In Re Drexel Burnham Lambert, Inc., 861 F.2d 1307, 1313 (2d Cir.1988); United States v. District Council of New York City, 941 F.Supp. 349, 358 (S.D.N.Y.1996).
As noted above, the facts pertaining to the recusal motion in this case were assessed by the district court under the incorrect “involved officer or member” standard and found not to warrant Lacey’s recusal under that standard. Whether the district court would have reached the same conclusion under the correct “evident partiality” standard remains an open question. As we have indicated, the test we endorse does not depend upon proof of actual partiality. Under the “evident partiality” standard, the bar the movant must clear is somewhat lower than actual partiality. The facts in United States v. IBT (“Simpson”), 120 F.3d 341, 347-48 (2d Cir.1997) are not dissimilar from those presented here; however, in Simpson we referenced the applicable standard as one of “bias” which may be equated to actual partiality. We did not review the conduct in question in that case under the more precise formulation of evident partiality. We believe that the allegations are such that the appropriate course is for us to remand to the district court for reconsideration of the recu-sal motion under the “evident partiality” standard. On remand, we expect the district court to consider Simpson as now modified by our formulation of the appropriate recusal standard.
Conclusion
The order of the district court is vacated and the case remanded for further proceedings consistent with this opinion. Each party shall bear its own costs.