Opinion for the court filed by Circuit Judge SILBERMAN.
Dissenting opinion filed by Chief Judge MIKVA.
SILBERMAN, Circuit Judge:The Secretary of Health and Human Services denied appellant hospitals’ claims for reimbursement under the Medicare program, and they sought review of the Secretary’s decision in the district court. Appellants claimed that in limiting them to the lower reimbursement rates allowed to rural hospitals, the Secretary ignored appellants’ “urban” character and arbitrarily and capriciously refused to grant them an exception from the classification scheme. The district court dismissed appellants’ complaint for failure to state a claim under Fed.R.Civ.P. 12(b)(6). On appeal, the hospitals contend that in ruling on the Secretary’s 12(b)(6) motion the district court improperly looked beyond the face of the complaint to the Secretary’s statements in the Federal Register and decided the merits of the case. Such a decision, they argue, should have been left to summary judgment. The district court may, however, examine matters of public record in ruling on a Rule 12(b)(6) motion, and when a district court is reviewing agency action — sitting as an appellate tribunal — the legal questions raised by a 12(b)(6) motion and a motion for summary judgment are *1223the same. We therefore affirm the dismissal.
I.
Under the Prospective Payment System implemented in 1983, the Medicare program does not reimburse hospitals for their actual costs. Instead, the Department of Health and Human Services (HHS) pays hospitals fixed rates based on average costs that are calculated according to both the type of service provided and the hospital’s geographic location. See 42 U.S.C. § 1395ww(d) (1983). The Medicare Act charges the Secretary of HHS with calculating both national and regional average costs for various procedures and specifies that the costs be computed separately for urban and rural areas. See id. § 1395ww(d)(2). A hospital classified as part of an urban area will receive higher Medicare reimbursements because the hospital’s payment rate will be based on the higher average costs of urban hospitals.
Appellants are all hospitals in counties classified as rural for purposes of Medicare reimbursement. The statute defines an urban area as “an area within a Metropolitan Statistical Area [ (MSA) ] (as defined by the Office of Management and Budget) [ (OMB) ] or within such similar area as the Secretary has recognized under subsection (a) of this section by regulation.” Id. § 1395ww(d)(2)(D). Under OMB regulations, a county adjacent to an MSA is included within the MSA if a specified percentage of its residents commute to the urban center. See 50 Fed.Reg. 35,685 (1985). Appellants do not qualify as urban under that provision, however, because they are located in counties that border on not one, but two or more MSAs. Their counties have the requisite percentage of commuters to neighboring MSAs, but because the commuters travel to multiple MSAs rather than a single one, those counties are classified as rural.1
As noted, the OMB system is not required by the terms of the statute. The Secretary may define urban areas “similar” to, but not the same as, OMB’s MSAs, see 42 U.S.C. § 1395ww(d)(2), and the statute further states that the Secretary “shall provide ... for such other exceptions and adjustments to such [prospective] payment amounts ... as the Secretary deems appropriate.” Id. § 1395ww(d)(5)(C)(iii) (1983). The Secretary has, for the most part, declined to exercise the power to modify the OMB classification scheme. See 42 C.F.R. §§ 412.62(f), 412.63(b) (1984). In notice and comment proceedings in 1984, 1985, and 1986, the Secretary considered suggestions to depart from the OMB system to take into account the anomalous commuting patterns of counties such as those in which appellant hospitals are located. The Secretary concluded that the existing system provided the best method currently available for making the urban/rural classification and that there were no other “generally accepted criteria” that could objectively distinguish urban from rural hospitals. See 49 Fed.Reg. 234, 254 (1984); see also 50 Fed.Reg. 35,646, 35,684-85 (1985).
For the cost years from 1984 to 1988, appellants claimed reimbursement as urban hospitals despite their ineligibility under the then-existing regulations. After their claims for reimbursement were denied, appellants sought review in the district court pursuant to 42 U.S.C. § 1395oo(f)(l). Appellants asserted that in adhering to OMB’s MSA classification system, the Secretary had refused to consider material that had been submitted on the record in earlier notice and comment proceedings and that showed appellants’ urban character. The hospitals charged that the Secretary had arbitrarily and capriciously refused to formulate exceptions to the OMB scheme to take into account the anomalies in the commuting patterns of appellants’ counties.
The Secretary moved to dismiss, both for lack of subject matter jurisdiction and for *1224failure to state a claim. According to the Secretary, during the years in question the Act committed any decision to deviate from the OMB scheme to the agency’s unreviewable discretion. The district court disagreed with the Secretary as to jurisdiction, but the court dismissed appellants’ complaint for failure to state a claim under Fed.R.Civ.P. 12(b)(6). See Marshall County Health Care Auth. v. Sullivan, Civ. Act. No. 91-0208, Mem. Op. at 5 (D.D.C. Nov. 12, 1991).
II.
Before the district court, appellants argued that the Secretary had illegally (arbitrarily and capriciously) declined to grant an exception to the regulatory scheme for hospitals in counties adjacent to multiple MSAs. Yet the “exception” they called for is a generic modification of the definition of urban areas, a modification which they subsequently obtained by legislative amendment. Of course, as appellants note, the Secretary had, and has, the power to redefine urban areas by regulation — as well as to grant exceptions. Their request could be satisfied, it would seem, by either procedure. And a refusal to engage in rulemaking is, of course, reviewable under the Administrative Procedure Act (APA). See, e.g., American Horse Protection Ass ’n, Inc. v. Lyng, 812 F.2d 1, 4-5 (D.C.Cir.1987); Natural Resources Defense Council v. SEC, 606 F.2d 1031, 1047 (D.C.Cir.1979). But since appellants emphasize the Secretary’s refusal to grant an exception, the government reasserts in this court a jurisdictional defense — that under the APA the Secretary’s decision not to grant an exception is unreviewable. The government claims that the statutory language “as the Secretary deems appropriate” satisfies the APA’s exception to judicial review for agency action “[that] is committed to agency discretion by law.” 5 U.S.C. § 701(a)(2).
We agree with the government that the language and structure of the statute suggest a broad delegation of discretion to the Secretary either to modify the regulation or to grant an exception. We disagree, however, that the latter determination is completely unreviewable. But see Columbia-Greene Medical Center, Inc. v. Sullivan, 758 F.Supp. 821, 824 (N.D.N.Y.1991). The government relies on Webster v. Doe, 486 U.S. 592, 108 S.Ct. 2047, 100 L.Ed.2d 632 (1988), in which the Supreme Court held that a similarly worded statute that authorized the Director of the CIA to discharge an employee whenever “he shall deem [it] necessary or advisable in the interests of the United States,” 50 U.S.C. § 403(c) (emphasis added), granted unreviewable discretion. Id. at 600, 108 S.Ct. at 2052. But the government, in our view, puts too much emphasis on the word “deem.” Judicial review is barred under 5 U.S.C. § 701(a)(2) “in those rare instances where ‘statutes are drawn in such broad terms that in a given case there is no law to apply.' ” Citizens to Preserve Overton Park v. Volpe, 401 U.S. 402, 410, 91 S.Ct. 814, 820, 28 L.Ed.2d 136 (1971) (citation omitted). In Webster, the lack of reasonable legal standards stemmed from the phrase “advisable in the interests of the United States,” which provides no judicially cognizable norm. The statute, after all, dealt with national security — an area in which the judiciary almost invariably defers to the executive branch. As the Webster Court observed, a deferential attitude on the part of Congress, as well, permeated “the overall structure of the NSA [National Security Act].” Webster, 486 U.S. at 600, 108 S.Ct. at 2052.
It is certainly not true, by contrast, that the Medicare statute reflects that degree of fulsome congressional deference to the executive. With respect to the particular provisions relevant to this case, moreover, Congress has provided a rather specific norm — the OMB model — to guide the Secretary’s judgment concerning the definition of urban areas. The Secretary may vary the definition, but we surely can hypothesize forms of regulatory amendments that could be thought unreasonable in light of the statute. Were the Secretary arbitrarily to grant an exception for some hospitals and not for others identically situated, one could expect a successful challenge. To be sure, a refusal to grant any changes in the definition, either by way of modification of *1225the regulation or through an exception, presents a less vulnerable target for challenge under the APA. We have often held that the scope of judicial review of a refusal to engage in rulemaking, see, e.g., American Horse Protection Ass’n, Inc. v. Lyng, 812 F.2d 1, 4-5 (D.C.Cir.1987); WWHT, Inc. v. FCC, 656 F.2d 807, 817-18 (D.C.Cir.1981), or a refusal to grant waivers or exceptions, see, e.g., City of Angels Broadcasting, Inc. v. FCC, 745 F.2d 656, 663 (D.C.Cir.1984); Thomas Radio Co. v. FCC, 716 F.2d 921, 924 (D.C.Cir.1983); WAIT Radio v. FCC, 459 F.2d 1203, 1207 (D.C.Cir.), cert. denied, 409 U.S. 1027, 93 S.Ct. 461, 34 L.Ed.2d 321 (1972), is quite narrow. Extremely narrow review is not, however, conceptually equivalent to — although it may often turn out in practical effect to be the same as — no review at all.2
In sum, the statute outlines a system that groups together hospitals facing similar operating costs to generate average figures that reflect the costs faced by hospitals in different markets. It is conceivable that a petitioner could show that due to peculiarities in the hospital industry, some of the divisions drawn by OMB between urban and rural areas bear a perverse relationship to the cost differentials of hospital markets. In that case, the statutory scheme might provide a sufficient standard for a reviewing court to conclude that the Secretary had acted arbitrarily and capriciously by refusing to modify the classification system — either by altering the regulation or by granting an exception. Accordingly, we think that the decision whether to provide exceptions under § 1395ww(d)(5)(C)(iii) is not committed to agency discretion by law and that the district court properly determined that it had jurisdiction over the case.
III.
As best we understand appellants’ underlying position, it is that in 1986 when the Secretary, in a record proceeding, considered whether to modify the regulation in the direction appellants sought, the Secretary both unreasonably denied the modification and on the same basis rejected appellants’ request for an exception.3 The Secretary acknowledged that “[ijntuitively, the approach suggested by the commentators has merit and thus warranted our consideration,” 51 Fed.Reg. 31,454, 31,470 (1986). But the Secretary rejected appellants’ request based in part on hospital wage data that distinguished “urban” and “rural” counties’ costs in a fashion unfavorable to appellants’ contentions. Id. Appellants wish to go to trial, preceded by discovery, to show that the Secretary’s wage studies were “flawed” or unfair and that they were “of questionable value” to justify refusal of an exception.
Appellants misunderstand the role the district court plays when it reviews agency action. The district court sits as an appellate tribunal, not as a court authorized to determine in a trial-type proceeding whether the Secretary’s study was factually flawed. Appellants imply that their claims that the Secretary acted arbitrarily and capriciously, or refused to consider material in the record, are similar to factual allegations — allegations they are entitled to “prove” as if the complaint alleged that the Secretary, driving her car negligently, had run into one of their hospitals. Based on this misunderstanding, appellants contend that the district court erred in dismissing their complaint under Rule 12(b)(6) because the court actually considered whether their “allegations” were true and thus improperly determined the “merits” of the dispute. Such a conclusion, according to appellants, might be appropriate on a motion for summary judgment, but certainly not on a motion to dismiss — the latter is meant to test only the sufficiency of the complaint.
*1226Appellants, however, overlook the character of the questions before the district court when an agency action is challenged. The entire case on review is a question of law, and only a question of law. And because a court can fully resolve any purely legal question on a motion to dismiss, there is no inherent barrier to reaching the merits at the 12(b)(6) stage. Cf. Neitzke v. Williams, 490 U.S. 319, 327, 109 S.Ct. 1827, 1832, 104 L.Ed.2d 338 (1989) (noting that a party may lose on a 12(b)(6) motion whether his claim “is based on an outlandish legal theory or a close but ultimately unavailing one”). Appellants’ complaint, properly read, actually presents no factual allegations, but rather only arguments about the legal conclusion to be drawn about the agency action. As a result, the sufficiency of the complaint is the question on the merits, and there is no real distinction in this context between the question presented on a 12(b)(6) motion and a motion for summary judgment.4
The same basic misunderstanding underlies appellants’ claim that by examining the Secretary’s published responses to comments in the rulemaking proceeding, the court exceeded its latitude in ruling on a 12(b)(6) motion. Of course, when a district court is not sitting as an appellate court and the district judge looks outside the complaint to factual matters, he or she must convert a motion to dismiss into a motion for summary judgment, see, e.g., Gordon v. National Youth Work Alliance, 675 F.2d 356, 360 (D.C.Cir.1982). Similarly, when agency action is challenged on grounds that could call for an evidentia-ry hearing, such as certain allegations of unconstitutional behavior, see, e.g., Telecommunications of Key West, Inc. v. United States, 757 F.2d 1330, 1334 (D.C.Cir.1985), the same rule applies.5 But the agency record was not used here to test any factual allegations in the complaint. In this context, it is no different from an agency record or district court opinion before the court of appeals for review. Thus, the district court can consult the record to answer the legal question before the court — in this case whether the agency adhered to the standards of decisionmaking required by the APA.6
As for appellants’ procedural claim — that they were entitled to discovery and to a trial to test the validity of the Secretary’s study or the reasonableness of the Secretary’s reliance on it — appellants are simply wrong. They were entitled to argue, just as they could on direct review of agency action in the court of appeals, that the Secretary arbitrarily did not consider material in the record, see, e.g., Motor Vehicles Mfrs. Ass’n v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 52, 103 S.Ct. 2856, 2871, 77 L.Ed.2d 443 (1983), or unreasonably relied on other material in the record that was logically flawed. Challengers to agency action are not, however, ordinarily entitled to augment the agency’s record with either discovery or testimony presented in the district court. See, e.g., Florida Power & Light Co. v. Lorion, 470 U.S. 729, *1227743-44, 105 S.Ct. 1598, 1606-07, 84 L.Ed.2d 643 (1985); Camp v. Pitts, 411 U.S. 138, 142-43, 93 S.Ct. 1241, 1244, 36 L.Ed.2d 106 (1973). Even on those rare occasions when that is appropriate, the district court is not engaged in ordinary fact-finding, but instead is filling in gaps in the record to determine what the agency actually did. See Citizens to Preserve Overton Park v. Volpe, 401 U.S. 402, 420, 91 S.Ct. 814, 825, 28 L.Ed.2d 136 (1971).
Appellants did not present any real argument before the district court to show that the Secretary’s determination not to grant them an exception (really to modify generally the regulation) was arbitrary and capricious. They stood on their “right” to proceed through to discovery and to trial to establish that the Secretary’s decision was illegal. Unfortunately for appellants, they had no such right, and the district court correctly so perceived their situation. It seems rather obvious to us, however, that appellants lost nothing by their mistaken posture. Given the extremely narrow scope of review any court is entitled to exercise over the Secretary’s determination not to modify the regulation or to grant exceptions, it does not appear that there is any merit to their claim that the Secretary violated the law.
sjc % * * * fc
Accordingly, the judgment of the district court is affirmed.
. Under a 1988 revision of the Act, counties adjacent to more than one MSA can have their commuting to multiple MSAs aggregated and, if their total commuting to adjacent MSAs meets the threshold defined by OMB, may be considered a part of the adjacent MSA to which they have the highest commuting level. See Medicare Catastrophic Coverage Act of 1988, § 411(b)(4), Pub.L. No. 100-360, 102 Stat. 683, 770 (1988).
. And the use of the mandatory “shall," in the statute ("the Secretary shall provide ... for such exceptions ... as he deems appropriate”) might be thought to add at least some obligation to consider exceptions. See, e.g., Armstrong v. Bush, 924 F.2d 282, 295-96 (D.C.Cir.1991).
. Appellants claim that prior to 1986 the Secretary merely refused to change the regulation without sufficient "analytical basis.”
.Our dissenting colleague does not disagree with our premise that the issue before the district court is a question of law. He is concerned, however, that on a motion to dismiss the plaintiff might not have as adequate an opportunity to brief and argue the administrative law issues as it would have on summary judgment. We see no reason why that would be so; a district judge can just as easily call for briefing (memoranda) and argument of such issues whether it is under the auspices of a 12(b)(6) or Rule 56 motion. In this case, the government filed a memorandum with its motion, but the appellants declined to engage and claimed instead that they had a right to discovery. Nor does our colleague assert that the outcome of this appeal should be altered, so we assume that the dissent is really a concurrence.
. It is probably the better practice for a district court always to convert to summary judgment so as to avoid the question presented in this case. But even in our earlier opinion in Telecommunications of Key West, we recognized that if we agreed with the district judge on the law when we reviewed a 12(b)(6) dismissal, a remand to convert to a Rule 56 motion would be futile. See Tele-Communications of Key West, 757 F.2d at 1334.
. In any event, as matters of public record, statements in the Federal Register can be examined on 12(b)(6) review. See Mack v. South Bay Beer Distributors, Inc., 798 F.2d 1279, 1282 (9th Cir.1986) (citing Interstate Natural Gas Co. v. Southern California Gas Co., 209 F.2d 380, 385 (9th Cir.1953)); Phillips v. Bureau of Prisons, 591 F.2d 966, 969 (D.C.Cir.1979).