Defendant-Appellants U.S. Bancorp and U.S. Bank Long Term Disability Plan, (the “Plan”), successors in interest to First Bank Systems, Inc. and First Bank Systems Long-Term Disability Plan, respectively, appeal from the district court’s grant of summary judgment in favor of Plaintiff-Appellee Denise Rekstad on her claim for disability benefits under the Employee Retirement Income Security Act of 1974 (“ERISA”), 29 U.S.C. §§ 1001-1461. Our jurisdiction arises under 28 U.S.C. § 1291, and we reverse and remand.
Background
Ms. Rekstad worked for U.S. Bancorp1 as a loan originator and took part in the Plan, which it administered and insured. After an ankle injury in November 1993, she could not work for a period of time and therefore received short-term and eventually long-term disability benefits pursuant to the Plan. In April 1995, Ms. Rekstad’s physician, Dale Kaiser, M.D., determined that Ms. Rekstad’s ankle had healed sufficiently so that she could return to work. He in turn informed ITT Hartford (“ITT”) — -U.S. Bancorp’s agent with respect to the administration, management and payment of the benefits described in the Plan — that she was medically cleared.
In June 1995, U.S. Bancorp offered Ms. Rekstad her previous position of employment. In response, she informed U.S. Bancorp that, on June 9, 1995, she was involved in an automobile accident. She explained that injuries she suffered therefrom rendered her totally disabled and that she needed to continue her long-term disability benefits under the Plan. Ms. Rekstad continued to receive long-term disability benefits until February 1996 when ITT sent her a letter explaining that she no longer met the Plan’s definition of “total disability.” Aplt.App. at 1331. Ms. Rekstad administratively appealed that decision and was again denied further benefits.
In April 1996, U.S. Bancorp sold its Colorado mortgage offices to Knutson Mortgage (“Knutson”). In May 1996, after Ms. Rekstad exhausted her administra*1117tive appeals, U.S. Bancorp terminated her employment because she did not return to work. U.S. Bancorp did advise her, however, that Knutson had agreed to offer continued employment to all U.S. Bancorp loan originators and that she should therefore immediately contact Knutson. Ms. Rekstad contacted Knutson and represented that she desired to commence her employment as soon as possible as a loan originator, presenting herself as ready and willing to work. Knutson refused to hire her in any capacity.
Ms. Rekstad’s post-accident medical condition has been extensively evaluated and treated. She has undergone several neuropsychological evaluations, and she continues to see a physician regarding her physical condition — namely in regards to chronic pain she allegedly continues to suffer as a result of the accident.
In June 1996, Dr. Thomas Bennett, Ph. D., a psychologist under whose care Ms. Rekstad was receiving neuropsychological treatment, released Ms. Rekstad for part-time work. This was the first time she had ever been released to do any type of work since her automobile accident. In a report documenting the release, Dr. Bennett explained that Ms. Rekstad would have residual deficits with fatigue, problems with memory and attention/coneen-tration, and confusion with financial aspects.
Ms. Rekstad subsequently sought and obtained employment three different times. Beginning in August 1996, she worked as a full-time straight-commissioned loan originator with the Norwest Corporation (“Norwest”). Ms. Rekstad was terminated by Norwest in February 1997, but soon thereafter, she began working for Cherry Creek Mortgage (“Cherry Creek”). She was terminated from that position in May 1997 but was again quickly employed by Chase Manhattan Mortgage (“Chase”). While working at Chase from May 1997 to July 1998, Ms. Rekstad earned $58,395.79. During this period, she also enrolled in Regis University’s masters of non-profit management program.
In July 1998, Ms. Rekstad took a leave of absence from Chase. Her supervisor at Chase informed her that her performance was unsatisfactory but that Chase proposed placing her on disability instead of terminating her. Ms. Rekstad was approved for and received short-term and then long-term disability benefits through Chase’s long-term disability carrier, Liberty Mutual. Liberty Mutual has been issuing disability benefits to Ms. Rekstad since January 12, 1999, based on a disability date of July 16,1998.
On January 20, 2000, Ms. Rekstad filed a claim for social security benefits, alleging that she became disabled on July 20, 1998. Though she claimed she became disabled as of that date, she explained that her disability was the result of her June 9, 1995 automobile accident. On June 29, 2001, the Social Security Administration (“SSA”) found her disabled. The Administrative Law Judge (“ALJ”) determined that she is unable to perform any work on a sustained basis, including sedentary unskilled work. See ApltApp. at 1758. The ALJ also concluded that Ms. Rekstad has not engaged in substantial gainful activity since July 10, 1998, and that she has been under a disability since July 10,1998.2 See id.
Ms. Rekstad filed the current civil action on June 24, 1997, seeking, inter alia, a judicial review of U.S. Bancorp’s denial of disability benefits under ERISA. On March 10, 1999, the district court conclud*1118ed that U.S. Bancorp’s denial of Ms. Rek-stad’s benefits was arbitrary and capricious.3 The district court then remanded to U.S. Bancorp the issue of Ms. Rekstad’s eligibility for disability benefits after January 31, 1996. U.S. Bancorp appealed from the remand order, but this court found that it was not an appealable final decision. See Rekstad v. First Bank Sys., Inc., 238 F.3d 1259, 1261-62 (10th Cir.2001). As such, pursuant to the remand order, U.S. Bancorp was required to ascertain Ms. Rekstad’s eligibility for long-term disability benefits after January 31,1996.
In accordance with the remand order, Ms. Rekstad served her claim for long-term disability benefits on U.S. Bancorp. In it, she made available her post-accident employment information and all of her medical records — including those of her treating physicians whom concluded Ms. Rekstad was totally disabled due to her physical and cognitive impairments. Upon receipt, U.S. Bancorp charged a benefit claim subcommittee (the “Committee”) with the responsibility of dealing with Ms. Rekstad’s claim. In turn, the Committee asked ITT to analyze whether Ms. Rek-stad ceased to be disabled at any time on or after January 31, 1996 for “any occupation.”
ITT utilized the Medical Advisory Group (“MAG”)4 to review and analyze the medical findings associated with Ms. Rekstad’s claims. ITT asked Todd Lyon, M.D., a practicing physician and member of MAG, to determine whether Ms. Rekstad had the capacity to work after January 31, 1996. Martin Zelman, M.D., a psychiatrist and another member of MAG, was asked by ITT to review Ms. Rekstad’s medical file regarding her neuropsychological condition to determine if she was capable of working. Neither physician treated or examined Ms. Rekstad, but based on their review of her medical records determined, in essence, that the medical evidence submitted by Ms. Rekstad in support of her disability was mostly subjective and inconsistent with the objective medical findings and that such objective medical findings do not support limitations that would preclude her from performing her job or any job due to a physical or cognitive impairment.
In July 2001, ITT submitted a report to the Committee concluding that Ms. Rek-stad was not disabled beyond January 31, 1996. ITT stated that this conclusion was based upon (1) Ms. Rekstad’s employment history with Norwest, Cherry Creek, and Chase; (2) her representations that she was not disabled; (3) her attendance at educational courses while working at Chase; (4) the SSA’s decision that Ms. Rekstad was disabled as of July 20, 1998; and (5) the conclusions of Drs. Lyon and Zelman. See Aplt.App. at 1425.
On August 2, 2001, the Committee met to review ITT’s determination that Ms. Rekstad’s claim should be denied. Based on this review, the Committee asked ITT to clarify a couple issues, which it did to the Committee’s satisfaction. On August 15, 2001, U.S. Bancorp denied Ms. Rek-stad’s long-term disability benefits claim.
On October 15, 2001, Ms. Rekstad administratively appealed U.S. Bancorp’s decision. In so doing, she supplied further documentation to U.S. Bancorp, including additional medical records from her treating physicians and affidavits from herself, *1119her husband, and her sister regarding her post-accident employment. The Committee sent the additional information submitted by Ms. Rekstad to ITT for its review. ITT in turn requested that Dr. Lyon review this information as well. Dr. Lyon submitted four addenda to his initial report after reviewing the additional information provided by Ms. Rekstad. In each, he concluded by explaining that despite the new information, he found that his previously rendered opinion was unchanged. However, neither Dr. Lyon nor ITT considered the affidavits submitted by Ms. Rekstad and her relatives because the information was not provided by medical professionals.
On January 20, 2002, ITT upheld its initial determination that Ms. Rekstad was not entitled to long-term disability benefits under the Plan. The Committee reviewed ITT’s determination and Dr. Lyon’s supplemental comments. On February 12, 2002, U.S. Bancorp agreed with ITT, upholding its earlier decision.
Following U.S. Bancorp’s denial of Ms. Rekstad’s claim for disability benefits, she filed a motion to reopen the ease, which the district court granted. Thereafter, Ms. Rekstad filed a motion for summary judgment claiming that U.S. Bancorp’s decision to deny her long-term disability benefits after January 31, 1996 was arbitrary and capricious. U.S. Bancorp filed a combined response and motion for summary judgment. On April 21, 2004, the district court agreed with Ms. Rekstad, granting her motion and denying U.S. Bancorp’s. On December 10, 2004, the district court ruled on the calculation of damages, finding that the benefits owed Ms. Rekstad under the Plan should not be reduced by the disability benefits she received from Liberty Mutual. This appeal followed.
Discussion
Appellants forward three arguments on appeal. They contend that the district court erred in (1) finding that U.S. Ban-corp, as administrator of the Plan, operated under a conflict of interest; (2) failing to properly apply the arbitrary and capricious standard of review; and (3) deciding not to reduce Ms. Rekstad’s award of benefits by the amount of disability payments she received from Liberty Mutual.
We review a grant of summary judgment de novo, applying the same legal standard used by the district court. Charter Canyon, 153 F.3d at 1135. Summary judgment is appropriate “if the pleadings, depositions, answers to interrogatories, and admissions on file, together with the affidavits, if any, show that there is no genuine issue of material fact and that the moving party is entitled to a judgment as a matter of law.” Fed.R.Civ.P. 56(c); Kimber v. Thiokol Corp., 196 F.3d 1092, 1097 (10th Cir.1999).
A court reviewing a challenge to a denial of employee benefits under 29 U.S.C. § 1132(a)(1)(B) applies an “arbitrary and capricious” standard to a plan administrator’s actions where, as here, the plan grants the administrator discretionary authority to determine eligibility for benefits or to construe the plan’s terms. See Firestone Tire & Rubber Co. v. Bruch, 489 U.S. 101, 115, 109 S.Ct. 948, 103 L.Ed.2d 80 (1989). The district court’s determination of whether a plan administrator’s decision is arbitrary and capricious is a legal conclusion subject to de novo review. Caldwell v. Life Ins. Co. of N. Am., 287 F.3d 1276, 1282 (10th Cir.2002).
Indicia of an arbitrary and capricious decision include, inter alia, lack of substantial evidence. Id. Substantial evidence is such evidence that a reasonable mind might accept as adequate to support *1120the conclusion reached by the decision maker. Id. It requires “more than a scintilla but less than a preponderance.” Sandoval v. Aetna Life & Cas. Ins. Co., 967 F.2d 377, 382 (10th Cir.1992) (internal quotations omitted). “In determining whether the evidence in support of the administrator’s decision is substantial, we must take into account whatever in the record fairly detracts from its weight.” Caldwell, 287 F.3d at 1282 (internal quotations and alterations omitted). Moreover, “substantiality of the evidence is based upon the record as a whole.” Id.
Though our standard of review always remains arbitrary and capricious, Fought v. UNUM Life Ins. Co. of Am., 379 F.3d 997, 1004 (10th Cir.2004) (per curium), the amount of deference given to a plan administrator’s decision decreases in certain circumstances. It is lessened in varying degrees if there exists a conflict of interest between the administrator’s duty to act in the interest of the plan participant and the administrator’s self interest or loyalty to his employer. See Firestone, 489 U.S. at 115, 109 S.Ct. 948; Fought, 379 F.3d at 1004-05. When there exists such a conflict of interest, we undertake a “sliding scale” analysis, “decreasing] the level of deference given to the conflicted administrator’s decision in proportion to the seriousness of the conflict.” Fought, 379 F.3d at 1004 (quoting Chambers v. Family Health Plan Corp., 100 F.3d 818, 825 (10th Cir.1996)).
The district court held that U.S. Bancorp’s decision was entitled to some deference, but that deference would be lessened to the degree necessary to neutralize any untoward influence resulting from the conflict it found. Relying on Kirnber, Appellants contend that this holding was in error. We need not address this issue because after reviewing U.S. Bancorp’s determination under the fully deferential arbitrary and capricious standard, we find that its decision to deny benefits was arbitrary and capricious. We emphasize, however, that we neither determine whether Ms. Rekstad is totally disabled, nor whether U.S. Bancorp’s decision was incorrect, rather we conclude only that it denied Ms. Rekstad’s benefits in an arbitrary and capricious manner. Central to that holding is our determination that U.S. Bancorp failed to examine a material portion of the relevant evidence.
In deciding whether Ms. Rekstad was totally disabled under the terms of the Plan, ITT considered evidence of her post-accident employment and education. Playing a significant role, ITT’s consideration of this evidence touched on several of the stated bases for its July 2001 determination that she was not so disabled: her post-accident employment history, her representations that she was willing and able to work, her attendance at educational courses while working at Chase, and Dr. Zelman’s report which indicates that his conclusion was based, in part, on Ms. Rek-stad’s success in her post-accident employment. Relying on ITT’s recommendation, U.S. Bancorp decided to deny benefits.
In her administrative appeal, Ms. Rekstad and two of her relatives submitted affidavits to U.S. Bancorp explaining that though she was employed three times subsequent to her accident and did enroll in some post-graduate courses, she was unable to maintain her job with each employer or complete her course work because of manifest difficulties associated with her physical and cognitive impairments. U.S. Bancorp forwarded those affidavits to ITT for its consideration. In ITT’s January 2002 re-determination of whether Ms. Rekstad was disabled under the Plan, it stated that the affidavits “were not considered in making our determination.” Aplt.App. at 1411. U.S. Bancorp requested ITT to clarify that statement. *1121ITT explained that the “personal affidavits were reviewed but not deemed pertinent to the disability determination, as these individuals are no [sic] medical professionals.” Id. at 1613. U.S. Bancorp took no further action on the issue.
It was arbitrary for U.S. Bancorp to make its decision to deny disability benefits without it giving full and fair consideration to the affidavits submitted by Ms. Rekstad and her relatives. As noted, evidence of Ms. Rekstad’s post-accident employment and education played a significant role in ITT’s determination that Ms. Rekstad is not totally disabled. But its consideration of this evidence was imper-missibly one-sided. While Ms. Rekstad’s ability to pursue and attain employment and attend post-graduate level courses may support a determination that she is not totally disabled, her inability to hold that employment or finish her course work because of problems associated with her physical and cognitive impairments may support a contrary conclusion. See Wilcott v. Matlack, Inc., 64 F.3d 1458, 1460-61 (10th Cir.1995). Evidence regarding the latter need not have been furnished by a medical professional to warrant its consideration. Because U.S. Bancorp relied upon ITT’s determination, wherein ITT did not consider this evidence, its decision was arbitrary. Cf. Sandoval, 967 F.2d at 380 (an administrator’s decision is not arbitrary and capricious for a failure to consider evidence not before it).
We have previously confronted a scenario where a plan administrator failed to gather and examine relevant evidence. In Caldwell, we accorded less deference to the plan administrator’s decision, but we went on to analyze whether it was nevertheless supported by substantial evidence, concluding that it was not. See Caldwell, 287 F.3d at 1285-86 (court’s de novo review of the record supported conclusion that claimant was disabled and the plan administrator’s failure to gather and examine relevant evidence buttressed the court’s determination that the plan administrator had 'acted arbitrarily and capriciously).
Under the facts of this case, however, we believe that a remand to U.S. Ban-corp is the more appropriate course of action. This is not a case where it is so clear-cut that it was unreasonable for U.S. Bancorp to deny Ms. Rekstad benefits. But because ITT’s one-sided consideration of Ms. Rekstad’s post-accident employment and education permeated U.S. Ban-corp’s decision, we are unable to determine the substantiality of the evidence supporting it. See Gaither v. Aetna Life Ins. Co., 388 F.3d 759, 773 n. 5 (10th Cir.2004) (concluding that merely reviewing a plan administrator’s decision under reduced deference per Caldwell is inappropriate where the failure to investigate left inadequate grounds for the court to determine whether it could still be within the bounds of reason); see also Quinn v. Blue Cross & Blue Shield Ass’n., 161 F.3d 472, 477 (7th Cir.1998) (“[T]he proper remedy in an ERISA case, as well as a conventional case, is to remand for further findings or explanations, unless it is ‘so clear cut that it would be unreasonable for the plan administrator to deny the application for benefits on any ground.’ ”) (quoting Gallo v. Amoco Corp., 102 F.3d 918, 923 (7th Cir.1996)). Moreover, we are not convinced that U.S. Bancorp would arrive at its previous conclusion once full and thorough consideration is given to all relevant evidence. And we will not substitute our judgment for that of U.S. Bancorp. See Quinn, 161 F.3d at 478; Miller v. United Welfare Fund, 72 F.3d 1066, 1073-74 (2d Cir.1995). We therefore remand to the district court with instructions that the case be returned to U.S. Bancorp for reconsideration.
*1122Determining that remand is proper, we need not reach the last issue presented: whether the district court erred in deciding not to reduce Ms. Rekstad’s award of benefits by the amount of disability payments she received from Liberty Mutual.
REVERSED and REMANDED.
. At many times in this case's factual chronology, First Bank was the actual entity involved. But for ease of understanding, this opinion refers only to U.S. Bancorp, unless otherwise noted.
. There is no indication in the record as to the basis for the discrepancy between Ms. Rekstad's claimed disability date of July 20, 1998 and the ALJ determination that she was disabled as of July 10, 1998. This difference is, however, immaterial to our discussion.
. The district court originally granted summary judgment in favor of Ms. Rekstad on July 30, 1998. In so doing, it applied a de-novo standard of review to U.S. Bancorp’s decision to deny benefits, a standard of review this court found to be improper in Charter Canyon Treatment Ctr. v. Pool Co., 153 F.3d 1132 (10th Cir.1998). As such, the district court granted a motion for reconsideration and reviewed the denial of benefits under an "arbitrary and capricious” standard.
. MAG is a vendor of ITT.