This petition for review challenges, on various grounds, civil assessments and penalties imposed upon a doctor by the Department of Health and Human Services (“HHS”) for submitting fraudulent Medicare claims. Dr. Timothy L. Stern petitions for review of the March 16,1993, decision of the HHS Departmental Appeals Board finding him liable under the Civil Monetary Penalties Law (“CMPL”), 42 U.S.C. § 1320a-7a (1988 & Supp. Ill 1991). The Board imposed “assessments” of $70,648 and “penalties” of *149$345,000. We conclude that the penalties were imposed in violation of a plea agreement that concluded criminal charges against Dr. Stern. We therefore modify the agency’s decision and affirm it, as modified.
Background
Dr. Stern practiced, as a sole practitioner, at the Rochester Pain Clinic from 1980 to 1986. He collected payment from his patients at the time services were rendered and then submitted claims on behalf of the patients to Blue Shield of Western New York (“BSWNY’), the then-designated Medicare carrier for the region. BSWNY and the Inspector General (“IG”) of HHS decided to seek a criminal prosecution after their investigations indicated that Dr. Stern was performing acupuncture but claiming his services as nerve blocks and “office visits.”
The criminal proceedings against Stem. In January 1986, a federal grand jury returned a 19-count indictment against Stern, charging him with mail fraud, filing false claims, intimidation of grand jury witnesses, mailing threatening communications, and conspiracy, and, in a separate indictment, with 23 counts of unlawful distribution of codeine to himself. On July 15, 1987, pursuant to a plea agreement, the details of which are set forth below, Dr. Stern pleaded guilty to one count of mail fraud (encompassing seven false Medicare claims submitted by Dr. Stem), 18 U.S.C. § 1341 (1988), and one count of unlawful distribution of codeine, 21 U.S.C. § 841(a)(1) (1988).
On September 24,1987, Dr. Stern received concurrent five-year suspended sentences and five years of probation. He was also ordered to pay restitution to patients in an amount not exceeding $70,000, to issue a personal apology to a grand jury witness whom he had threatened, and to continue in a drug rehabilitation program and psychiatric care. As a condition of his plea agreement, Dr. Stern also surrendered his license to practice medicine in New York.
Under 42 U.S.C. §§ 1320a-7(a)(l) and 1320a-7(c)(3)(B), the Secretary of HHS is required to exclude a person convicted of a criminal offense related to service under Medicare from participation in Medicare or any state health care program for a minimum of five years. After HHS notified Stern in April of 1988 that he was being excluded for twenty years, Stern petitioned for review of his exclusion pursuant to 42 U.S.C. § 1320a-7(f).
The HHS Office of Hearings and Appeals in Syracuse (Joseph G. Medicis, ALJ) reduced Stern’s exclusion to the mandatory minimum five-year period. ALJ Medicis found that there was a “legitimate difference of opinion” as to whether Stem’s services were reimbursable. The HHS Appeals Council denied the IG’s request for review, concluding “that there is no basis for changing the Administrative Law Judge’s decision.”
Proceedings under the CMPL. Shortly after ALJ Medicis issued his decision, the IG issued a Notice of Determination, informing Stern that the IG was initiating proceedings under the CMPL seeking an assessment of $70,648, civil monetary penalties of $425,250, and a 30-year exclusion from participation in Medicare and state health care programs. In the proceedings that followed, the HHS Departmental Appeals Board, Civil Remedies Division (Charles E. Stratton, ALJ) and the Appellate Division found that the “Respondent submitted claims to Medicare for services which he knew, should have known or had reason to know were not provided as claimed.” Such a finding of scienter, which appears to be at variance with the finding of ALJ Medicis, is a prerequisite for liability under the CMPL, see 42 U.S.C. § 1320a-7a(a)(l)(A). ALJ Stratton originally imposed assessments of $45,000 and penalties of $140,000. After the HHS Appellate Division rejected as irrelevant or unsubstantiated the ALJ’s consideration of all allegedly mitigating factors, ALJ Stratton on remand imposed assessments of $70,648, the full amount sought by the IG, and penalties of $345,000; the Appellate Division affirmed. Dr. Stern seeks review of this final determination by HHS.
Discussion
Petitioner presents a substantial argument that the imposition of penalties based *150on a finding of scienter is barred by the collateral estoppel effect of the prior administrative determination that there was a “legitimate difference of opinion” whether Stern’s services were reimbursable. However, we need not resolve that contention because the penalties are precluded in any event by the plea agreement that concluded the criminal prosecution.
The plea agreement provided that Stern would plead guilty to one count of mail fraud and one count of distributing a controlled substance, and would surrender his license to practice medicine. The agreement also specified that restitution might be ordered in an amount no less than $5,000 and no more than $110,000, the amounts to depend upon verifiable claims of patients. In a sentence critical to this petition for review, the agreement further provided: “Claims of the Medicare program will be determined in a separate civil proceeding and will not be part of the restitution ordered by the Court.” This provision was given further meaning during the plea allocution. The Assistant United States Attorney informed the defendant and the Court, “An additional one hundred ninety thousand dollars which the Medicare program alleges they lost in this will be handled in a separate civil proceeding and will not be part of the disposition of this case.” The Court also discussed with the defendant what claims remained to be pursued: “And that there are disputed claims in the amount of one hundred ninety thousand dollars which involve a separate civil case and that is not involved in this plea agreement. Do you understand that?” Stem replied, “I do understand.” There was no mention of any remaining dispute concerning civil penalties.
Two weeks later, the prosecutor wrote to the IG, recommending that the Government pursue a claim for civil penalties instead of the $190,000 of civil assessments for actual losses that had been discussed at the plea allocution. This letter fully confirms Stern’s understanding that only the claim for actual losses remained open: “Timothy L. Stern understands that Medicare will initiate civil action against him to recover a loss of approximately $190,000.” The prosecutor urged the proposed switch to penalties as “more appropriate” and “much easier” for HHS. Ultimately, the amount of the losses, as determined in the administrative proceedings, was $70,648, the amount of assessments sought by the IG.
Courts enforce what “the defendant reasonably understood” the plea agreement to mean when the guilty plea was entered, United States v. Corsentino, 685 F.2d 48, 51 (2d Cir.1982); accord, e.g., United States v. Nelson, 837 F.2d 1519, 1521 (11th Cir.), cert. denied, 488 U.S. 829, 109 S.Ct. 82, 102 L.Ed.2d 58 (1988); In re Arnett, 804 F.2d 1200, 1203 (11th Cir.1986); United States v. Crusco, 536 F.2d 21, 27 (3d Cir.1976); Johnson v. Beto, 466 F.2d 478, 480 (5th Cir.1972). A “rigidly literal approach in the construction of language” of a plea agreement is not appropriate, see United States v. Bowler, 585 F.2d 851, 854 (7th Cir.1978) (citing United States v. Brown, 500 F.2d 375 (4th Cir.1974)), and oral agreements can amplify ambiguous terms, see Arnett, 804 F.2d at 1202.
In the pending case, the plea agreement left “[c]laims of the Medicare program” open for determination in a later “civil proceeding,” and the defendant understood those claims to be actual losses alleged to total no more than $190,000. That understanding was articulated by the District Judge and shared by the prosecutor until he decided that it would be “much easier” to pursue a claim for penalties. Under these circumstances, the collection of penalties in the amount of $345,000 was barred by the plea agreement, and these penalties must be stricken from the agency’s decision. As modified by the elimination of the $345,000 in penalties, the decision requiring payment of $70,648 in assessments is affirmed.