OPINION OF THE COURT
SLOVITER, Circuit Judge.In this action, appellant, Daniel P. Cotter, seeks review of the decision of the Secretary of Health and Human Services denying Cotter’s concurrent applications for disability benefits under Title II of the Social Security Act as amended, 42 U.S.C. §§ 416(i), 423 (1976), and for Supplemental Security Income Benefits under Title XVI of the Act as amended, 42 U.S.C. § 1381a (1976). Benefits were denied initially, and again on reconsideration. After the subsequent hearing before an Administrative *702Law Judge, at which Cotter was represented by counsel, the ALJ ruled that Cotter was not disabled within the meaning of the Act and thus denied him benefits.1 The decision of the ALJ was approved by the Appeals Council. See 20 C.F.R. §§ 404.951, 404.957 (1980). Cotter then filed suit in district court pursuant to sections 205(g) and 1631(c)(3) of the Act, 42 U.S.C. §§ 405(g), 1383(c)(3) (1976), seeking review of the Secretary’s decision. Both parties moved for summary judgment, and the district court granted judgment in favor of the Secretary. For the reasons stated below we will vacate that judgment and remand the matter to the agency.
FACTS
Cotter was 57 years old on June 21, 1978, the alleged date of the disability. He had an eighth grade education, completed a welding course, and worked for at least the past 15 years as a welder of heavy equipment. He last worked on June 20, 1978, several days following an accident at work. In the work activity report filed by Mr. Cotter in connection with his disability claim, he stated that his last job as a welder of heavy equipment in the coal stripping industry required sitting eight hours a day, occasional lifting of from 21 to 50 pounds, carrying a 50 pound box 50 feet to a truck, and frequent exposure to fumes and dust. (A-134). At the hearing, Cotter also testified that this job required sometimes dragging 100 pound tanks. (A-47-48). In the report filed about his earlier employment, Cotter indicated that from 1970 to 1976 he had welded road construction equipment which required frequently lifting up to 50 pounds and occasionally up to 100 pounds, and carrying these weights approximately 10 feet. (A-136).
Cotter claimed disability for a variety of reasons, including a heart condition.2 The medical evidence presented to the ALJ concerning this claim is conflicting. Cotter has a history of frequent premature ventricular contractions (PVC)3 (bigeminy) and takes medication for the condition. On June 18, 1978 Cotter was seen in the emergency room at Clearfield Hospital for double vision resulting from a work accident. Subsequently, an ophthalmologist noted an irregular heartbeat and advised Cotter to see Dr. Baltazor Corcino, an internist and Cotter’s physician since July 1975, who was associated with Clearfield Hospital. An electrocardiogram showed frequent premature ventricular contractions with a bigeminal pattern, and Cotter was hospitalized at Clearfield Hospital from June 28 to June 30, 1978 for treatment. Cotter was advised to resume use of medication for his heart *703condition, and he was discharged from the hospital.
Dr. Corcino’s report, dated June 30, 1978, diagnosed Cotter as having arteriosclerotic heart disease with bigeminy (A-150). In a subsequent report dated September 28, 1978, Dr. Corcino stated that he had last seen Cotter on September 9, 1978; he reaffirmed his diagnosis of arteriosclerotic heart disease “with PVCs controlled.” Dr. Corcino noted that there was recurrent cardiac arrythmia but that none had occurred since the last visit. He reported dyspnea4 on fast walking and exertion but apparently was uncertain whether it was attributable to a cardiac condition. He also reported a two to three year history of chest pain consisting of a mild fluttering feeling occurring on and off even at rest. Current status showed regular sinus rhythm (A-163). On March 8,1979 and March 28,1979 Dr. Corcino reported his opinion that Cotter was unable to work; on the latter date Dr. Corcino reported: “Although [Cotter] feels okay, I don’t think he is able to go back to his usual work.” (A-216).
Several months after his hospitalization, Cotter was referred to Dr. William Kimber, a specialist with the Department of Cardiovascular Medicine at Geisinger Medical Center. After an initial series of tests performed on October 9,1978, Kimber reported his “impression” of “Ventricular premature beats — rule out associated cardiac disease.” (A-167). Further tests were scheduled and on October 25, 1978 a treadmill exercise ECG was administered. Following examination of the results of this test, Kimber reported that although Cotter experienced no chest pain or ST segment changes of significance, he did have a bigeminal rhythm during the warm-up period and short runs of ventricular tachycardia, a significant rhythm disturbance of the heart,5 during the test. (A-168). Kimber recommended that Cotter “not return to a vigorous and physically demanding occupation because of the ventricular tachycardia noted coming on with exercise.” (A-168). In a letter dated March 6, 1979, Kimber reviewed his previous findings and reiterated his concern that, due to the evidence of ventricular tachycardia coming on with physical stress, Cotter should not “return to any type of work that would entail physical labor.” (A-213).
In sharp contrast to Drs. Corcino’s and Kimber’s diagnosis and recommendations were the conclusions of Dr. Tito Trinidad, an internist, who performed a consultative examination in October/November 1978. In a letter dated November 6, 1978. Dr. Trinidad reported that Cotter’s heart was “Regular with occasional premature ventricular contractions. No murmur.” Dr. Trinidad concluded that Cotter’s prognosis was good and that he should be able to go back to work without any sequelae. These conclusions were apparently based on the results of a series of ECGs, including a two-step exercise ECG, taken on October 18, 1978. In a physical capacity evaluation, dated October 16, 1978, Dr. Trinidad estimated that Cotter had the ability to frequently lift from 21 to 50 pounds, occasionally lift from 51 to 100 pounds, and to frequently bend, squat, crawl and climb. (A-170). Dr. Trinidad’s report does not indicate the clinical tests or observations, if any, on which this evaluation is based.
ALJ REPORT
The ALJ denied Cotter’s application, finding that although Cotter had an “occasional premature ventricular heart beat” he still had “the physical capacity to perform his past customary work as a welder.” (A-29). The following portion of the decision entitled “Evaluation of the Evidence” represents the entire discussion of the evidence relating to Cotter’s heart condition:
*704Claimant was examined at the Geisinger Medical Clinic and it was noted that the claimant had experienced a ventricular occasional premature heart beat.. . . The examinations made at the Geisinger Medical Clinic indicated that a treadmill test found the sinus was regular and test results were normal and the claimant was asymptomatic. Associated heart disease was ruled out at the clinic. Physical examination conducted by Dr. Trinidad, the premature ventricular contractions were noted and in spite of the contractions the doctor gave a good prognosis and indicated that the claimant should be able to go back to work without any sequelae.... Physical capacities evaluation made at the same time, indicated that the claimant had the physical capacity to ... frequently lift up to 50 pounds and occasionally lift up to 100 pounds . . . and was able to frequently bend, squat, crawl and climb. (A-27).
The ALJ has a duty to hear and evaluate all relevant evidence in order to determine whether an applicant is entitled to disability benefits.6 The ALJ’s decision must be in writing and contain findings of fact and a statement of reasons in support thereof. 20 C.F.R. § 404.939 (1980).
We note that in his somewhat abbreviated discussion of the evidence relating to the heart condition, the ALJ makes no mention of any medical findings or opinions supporting Cotter’s claim. Dr. Corcino’s opinion that Cotter could not return to work is entitled to substantial weight because he is Cotter’s treating physician. Bastien v. Califano, 572 F.2d 908, 912 (2d Cir. 1978). It is not even mentioned by the ALJ. Perhaps more significant, the ALJ does not mention the findings of Dr. Kimber, a specialist in cardiovascular medicine, that Cotter had runs of ventricular tachycardia brought on by physical stress; that this condition is a significant rhythm disturbance of the heart which he believed was of more concern than premature ventricular contractions, the only arrythymia mentioned by the ALJ; and that because of the tachycardia Cotter should not “return to any type of work that would entail physical labor.” (A-213).
Further, the discussion by the ALJ indicates some confusion over the medical evidence. The ALJ stated that the treadmill test conducted at Geisinger indicated “that the sinus was regular and test results were normal.” In fact, Dr. Kimber’s evaluation of the treadmill test cannot fairly be characterized as showing normal results. In addition the ALJ stated that “Associated heart disease was ruled out at the clinic.” It is true that this was Dr. Kimber’s “impression” after his initial examination on October 9, 1978. However, the ALJ failed to mention that this report was written before Dr. Kimber had the benefit of the treadmill ECG’s, which resulted in Dr. Kimber’s subsequent findings and conclusions discussed above.
Apparently, the ALJ based his ultimate conclusion that Mr. Cotter could perform his prior job on Dr. Trinidad’s report and evaluation. Significantly, the ALJ gave no reason for implicitly rejecting the obviously probative and significant but conflicting findings and conclusions of Drs. Kimber and Corcino, which the ALJ failed to discuss.
DISCUSSION
Findings of fact by the Secretary must be accepted as conclusive by a reviewing court if supported by substantial evidence. Substantial evidence is such relevant evidence as a reasoning mind might accept as adequate to support a conclusion. Lewis v. Califano, 616 F.2d 73, 76 (3d Cir. 1980); 42 U.S.C. § 405(g).
There are cogent reasons why an administrative decision should be accompanied by a clear and satisfactory explication of the basis on which it rests. Chief among *705them is the need for the appellate court to perform its statutory function of judicial review. A statement of reasons or findings also helps to avoid judicial usurpation of administrative functions, assures more careful administrative consideration, and helps the parties plan their cases for judicial review. See K. Davis, 2 Administrative Law Treatise § 16.05 (1958). It is significant that both the Administrative Procedure Act governing administrative adjudications generally and regulations applicable to decisions of ALJs in disability matters require that the administrative law judge specify the reasons or basis for the decision. See 5 U.S.C. § 557(c) (1976); 20 C.F.R. § 404.939 (1980).
In consideration of our statutory obligation to review administrative decisions on disability claims, in Baerga v. Richardson, 500 F.2d 309, 312 (3d Cir. 1974), cert. denied, 420 U.S. 931, 95 S.Ct. 1133, 43 L.Ed.2d 403 (1975), we suggested the following standard for a hearing examiner’s opinion:
In our view an examiner’s findings should be as comprehensive and analytical as feasible and, where appropriate, should include a statement of subordinate factual foundations on which ultimate factual conclusions are based, so that a reviewing court may know the basis for the decision. This is necessary so that the court may properly exercise its responsibility under 42 U.S.C. § 405(g) to determine if the Secretary’s decision is supported by substantial evidence.
That suggestion was repeated in Kephart v. Richardson, 505 F.2d 1085, 1089-90 (3d Cir. 1974), and Gober v. Matthews, 574 F.2d 772, 776 (3d Cir. 1978).7 Finally, in Hargenrader v. Califano, 575 F.2d 434 (3d Cir. 1978), the suggestions made in Baerga were, as the dissent in Hargenrader recognized, elevated to a holding that required the hearing examiner to include subsidiary findings to support the ultimate findings. Id. at 438. Recently, the responsibilities set out in Baerga were referred to approvingly in Dobrowolsky v. Califano, 606 F.2d 403, 409 (3d Cir. 1979).
It is important to recognize that our requirement in this regard is not designed in any way to derogate from the ALJ’s responsibility under the statute to make the relevant findings of fact and “decisions as to the rights of any individual applying for” benefits. 42 U.S.C. § 405(b) (1976). We are also cognizant that when the medical testimony or conclusions are conflicting, the ALJ is not only entitled but required to choose between them. We cannot expect that this choice by the ALJ, in the exercise of his or her statutory responsibility, will be accompanied by a medical or scientific analysis which would be far beyond the capability of a non-scientist.
We interpret our prior language and holding in light of our statutory function of judicial review. In this regard we need from the AU not only an expression of the evidence s/he considered which supports the result, but also some indication of the evidence which was rejected. In the absence of such an indication, the reviewing court cannot tell if significant probative evidence was not credited or simply ignored. As we stated in Dobrowolsky v. Califano, 606 F.2d 403 (3d Cir. 1979),
unless the Secretary has analyzed all evidence and has sufficiently explained the weight he has given to obviously probative exhibits, to say that his decision is supported by substantial evidence approaches an abdication of the court’s “duty to scrutinize the record as a whole to determine whether the conclusions reached are rational.”
*706Id. at 407, quoting Gober v. Matthews, 574 F.2d at 776.8
We find Judge Garth’s objection to our reiteration of the need for articulation of reasons by administrative law judges somewhat puzzling. Although he voices concern that we are departing from the congressional standard of review, he agrees that the ALJ must make findings on the basis of “all the evidence in the record” and “[explain] both the evidence supporting his findings and the reasons for his decision.” Typescript op. of Judge Garth’s opinion at 711 (emphasis added). We acknowledge that the statute does not expressly place this obligation on the ALJ. It is, as we noted in our prior cases, one fairly inferable from our statutory obligation of review. Itds difficult to separate the obligation to explain why certain evidence has been accepted from the obligation to explain why other significant probative evidence has been rejected. To state the issue simplistically but clearly, if the record contained the evidence of six medical experts, one of whom supported the claimant and five of whom did not, it would be of little assistance to our review function were the ALJ merely to state that s/he credited the one supporting expert because that evidence adequately demonstrated disability, but failed to either mention or explain why the evidence of the other five experts was rejected. In that instance, we would not know whether the evidence of the five experts was rejected because the ALJ found it lacking in credibility, irrelevant, or marred by some other defect.
Thus, Judge Garth trifurcates what is essentially a uniform obligation of the ALJ, which is to provide an adequate basis so that the reviewing court can determine whether the administrative decision is based on substantial evidence. “Substantial evidence” can only be considered as supporting evidence in relationship to all the other evidence in the record.8 9
This court has recognized that there is a particularly acute need for some explanation by the AU when s/he has rejected relevant evidence or when there is conflicting probative evidence in the record. We have emphasized our concern in a long line of cases. Thus in Kennedy v. Richardson, 454 F.2d 376 (3d Cir. 1972), we vacated and remanded the decision of the ALJ because it failed to afford an explanation why the ALJ rejected medical evidence that supported the claimant which was inconsistent with other medical evidence and the ALJ’s findings. In Hargenrader v. Califano, supra, we reversed and remanded the decision of the hearing examiner because he had failed to address significant items of evidence which were in direct conflict with his findings. In Schaaf v. Matthews, 574 F.2d 157 (3d Cir. 1978), we held that it was error for an ALJ to reject uncontradicted medical evidence without a clear statement of the reasons for doing so. See also Smith v. Califano, 637 F.2d 968 (3d Cir. 1981) (Adams, J., concurring and dissenting); Gachette v. Weinberger, 551 F.2d 39 (3d Cir. 1977); Walker v. Mathews, 546 F.2d 814 (9th Cir. 1976); Rosario v. Harris, 492 F.Supp. 414 (D.N.J.1980).
Since it is apparent that the ALJ cannot reject evidence for no reason or for the wrong reason, King v. Califano, 615 F.2d 1018 (4th Cir. 1980), an explanation *707from the ALJ of the reason why probative evidence has been rejected is required so that a reviewing court can determine whether the reasons for rejection were improper. For example, in Gober v. Matthews, supra, we held that the ALJ had rejected medical testimony for improper reasons, a holding we could not have made without knowing the basis of the rejection. We also note with interest that the Fourth Circuit requires that the Secretary must consider all the evidence and explain on the record the reasons for his findings, including reasons for rejecting evidence in support of the claim. See King v. Califano, 615 F.2d at 1020; Myers v. Califano, 611 F.2d 980, 983 (4th Cir. 1980); Stawls v. Califano, 596 F.2d 1209, 1213 (4th Cir. 1979).
Turning to the decision of the ALJ in this case, we note there was expert medical testimony that was probative and supportive of Cotter’s claim which conflicted with the medical testimony accepted by the ALJ. The ALJ’s failure to explain his implicit rejection of this evidence or even to acknowledge its presence10 was error. Moreover, as noted above, the ALJ appears to have misunderstood some of Dr. Kimber’s findings and this misunderstanding may have affected his decision.
In addition, the ALJ’s conclusion that Cotter’s impairment of premature ventricular contractions did not prevent the performance of his past relevant work appears to be based on an erroneous construction of the nature of weight lifting entailed by Cotter’s work. The ALJ stated that Cotter’s work was “more of a skilled nature” and not one “involving physical endurance and excessive stamina requirements.”11 The ALJ apparently concluded that Cotter’s response on one work activity report that he occasionally carried from 21 to 50 pounds referred solely to Cotter’s carrying a 50 pound box of welding rods from a truck to the working station. Based on this interpretation of the work report, the ALJ apparently rejected as incredible Cotter’s otherwise uncontradicted testimony that welding involved heavy lifting, at times requiring the dragging of 100 pound tanks.12 The ALJ’s interpretation of the work activity report finds no support in the record. Moreover, the ALJ did not mention Cotter’s work activity report concerning his next to last job, also as a welder of heavy equipment, in which he indicated that he frequently carried from 21 to 50 pounds and occasionally carried from 51 to 100 pounds. The physical demands of previous welding jobs are relevant to the issue of whether Cotter has any impairment which prevents his engaging in past relevant work. 20 C.F.R. § 404.1503(e) (1980). Cotter’s response on the activity report regarding the demands of his prior employment also corroborates his testimony that welding involved heavy lifting. Since the ALJ did not consider all the relevant record evidence, and, more significantly, misconstrued the evidence considered, his conclusion that Cotter’s impairment did not prevent the performance of his past relevant work, which is based on the ALJ’s understanding of the physical demands of welding, must be reconsidered.
For the foregoing reasons, we believe that this matter should be returned to the ALJ for reconsideration based on the record heretofore established. The ALJ may take additional evidence as well, if he deems it necessary in his discretion. See *708Hargenrader v. Califano, 575 F.2d at 438. The burden is, of course, on the claimant to demonstrate by medical evidence that he is unable to return to his former occupation. Dobrowolsky v. Califano, 606 F.2d at 406. Thus, on remand the ALJ should reconsider whether Cotter’s evidence is sufficient to support a conclusion that Cotter’s heart condition, either alone or in conjunction with his mental impairment, precludes Cotter from performing his past relevant work.
For the foregoing reasons the judgment of the district court will be vacated and the matter remanded to the Secretary for further proceedings consistent with this opinion.
. Title II of the Act defines a disability as an
inability to engage in any substantial gainful activity by reason of any medically determinable physical or mental impairment which . . . can be expected to last for a continuous period of not less than 12 months....
42 U.S.C. § 423(d)(1)(A) (1976). The Act further provides that
an individual ... shall be determined to be under a disability only if his physical or mental impairment or impairments are of such severity that he is not only unable to do his previous work but cannot, considering his age, education, and work experience, engage in any other kind of substantial gainful work which exists in the national economy.. ..
42 U.S.C. § 423(d)(2)(A). Title XVI defines disability in nearly identical terms. See 42 U.S.C. § 1382c(a)(3).
. Cotter’s application for benefits also alleged disability due to diplopia (double vision) resulting from a work accident. The ALJ found that the condition had subsided shortly after the accident and thus could not be the basis of a disability claim. Appellant does not contest this portion of the ALJ’s ruling.
Appellant also claimed that he is disabled due to a mental impairment. The ALJ concluded that although Cotter suffered from depressive neurosis the condition was not sufficiently severe to be considered disabling. We cannot say that there is not substantial evidence to support this finding.
. A premature ventricular contraction is a “contraction of the ventricles (lower chambers of the heart) occurring sooner than it should in the timetable of the heart action.” 2 J. Schmidt, Attorneys’ Dictionary of Medicine P-207 (1980). “Although such beats are not uncommonly encountered in patients with hearts not found to be the seat of a structural abnormality that can be documented, they are more frequently noted where cardiac damage is detectable.” 1C R. Gray, Attorneys’ Textbook of Medicine ¶ 30.95 (1980).
. Dyspnea is “ [shortness of breath, labored breathing. In most cases dyspnea is due to heart disease; in some instances it is caused by kidney disease.” 1 J. Schmidt, supra note 3, at D 88.
. Tachycardia is an abnormally rapid beating of the heart. 3 J. Schmidt, supra note 3, at T 3. It is a significant rhythm imbalance and is of more concern that a finding of premature ventricular contractions. See 1C R. Gray, supra note 3, at ¶ 30.95.
. 20 C.F.R. § 404.1503(a) (1980) provides in part:
In the determination of whether or not an impairment in a particular case constitutes a disability . . ., consideration is given to all the pertinent facts of that case, (emphasis added)
See also 20 C.F.R. § 404.1502.
. In an analogous context the Supreme Court stated,
[CJourts cannot exercise their duty of review unless they are advised of the considerations underlying the action under review. . . . [T]he orderly functioning of the process of review requires that the grounds upon which the administrative agency acted be clearly disclosed and adequately sustained.
SEC v. Chenery Corp., 318 U.S. 80, 94, 63 S.Ct. 454, 462, 87 L.Ed. 626 (1943). See also Environmental Defense Fund, Inc. v. Ruckleshaus, 439 F.2d 584 (D.C. Cir. 1971).
. In Dobrowolsky we also stated:
This Court has repeatedly emphasized that the special nature of proceedings for disability benefits dictates extra care on the part of the agency in developing an administrative record and in explicitly weighing all evidence. (emphasis added)
. This very fact was recognized by Judge Garth in NLRB v. New York-Keansburg-Long Branch Bus Co., 578 F.2d 472, 478 n.15 (3d Cir. 1978), where, in fulfilling the same statutory obligation to ascertain if the record was supported by “substantial evidence,” Judge Garth found that the ALJ had not “consider[ed] all relevant factors and sufficiently explain[ed] his resolutions.” (emphasis deleted). To support his conclusion, Judge Garth noted: “[C]rucial undisputed documentary evidence contained in the record . . . cannot be reconciled with, and completely undermines, the testimony credited by the ALJ . . . Yet the ALJ makes little reference to this evidence in his opinion and in no way attempts to harmonize the documentary evidence with the testimonial evidence.” (emphasis added).
. Under the circumstances of this case we do not consider the ALJ’s bare recital of the boilerplate language that he “carefully considered all the testimony ... and the exhibits ...” to be sufficient.
. The ALJ did not make a specific finding of fact to this effect. Rather the statement appears in the portion of the decision entitled “Evaluation of the Evidence.” (A-29).
. Cotter submitted to the district court a work activity questionnaire completed by Cotter’s last employer, which indicated that his welding job required lifting up to 100 pounds. See Appendix A to Appellant’s Reply Brief. This report was not submitted to the ALJ and thus is not a part of the administrative record for purposes of reviewing the ALJ’s decision, either here or in the district court. Of course, on remand the ALJ can, in his discretion, reopen the record, so that this evidence can be considered.