OPINION EN BANC
BOUDIN, Chief Judge.Raymond Ellsworth appeals from the district court’s judgment denying his petition under 28 U.S.C. § 2254 (2000) for a writ of habeas corpus. Between 1988 and 1992, Ellsworth worked at the Spaulding Youth Center (“Spaulding”), a residential school and treatment facility for children with emotional, behavioral, and neurological impairments, in Northfield, New Hampshire. Ellsworth was a “cottage teacher” at Colcord Cottage, a dormitory facility at Spaulding for boys between the ages of six and twelve. His job included supervising the boys at meals and on field trips, meeting with several students on a weekly basis, and sleeping in the staff room at the cottage approximately once a week.
In November 1991, an eleven-year-old boy named Matthew was transferred from another treatment facility to Spaulding’s Colcord Cottage. In November 1992, Matthew accused Ellsworth of sexual abuse. In January 1995, Ellsworth was tried in New Hampshire state court on four counts of aggravated felonious sexual assault and eight counts of felonious sexual abuse, relating to three different alleged episodes of abuse. The primary evidence at trial was the conflicting testimony of Ellsworth and Matthew.
Matthew testified that Ellsworth sexually abused him on three separate occasions. He claimed that the first occurred on a bicycle trip near Spaulding in the summer of 1992. Matthew stated that Ellsworth lured him into the woods by claiming that he heard a noise and then pulled down his own pants and told Matthew to touch and put his mouth on Ellsworth’s penis. Matthew said that Ellsworth’s penis became erect when Matthew did this and that Ells-worth also took off Matthew’s pants and touched his penis. Matthew claims that Ellsworth threatened to hurt him if he told anyone about the incident.
According to Matthew, the second incident took place at a swimming pond near Spaulding when Matthew went swimming *3with Ellsworth and another Spaulding resident named Stephen. Matthew claimed that the three were in the pond and that Ellsworth asked Stephen to swim away. Matthew recalled that no one else was at the pond and that Ellsworth removed both of their bathing suits, touched Matthew’s penis and buttocks, and told him to put his mouth on Ellsworth’s penis. Matthew said that Ellsworth again told him not to tell anyone about the incident.
The third and final incident allegedly occurred when Matthew returned early to Spaulding from a weekend home-visit and Ellsworth was the only staff member on duty. Matthew stated that while he was putting on his pajamas, Ellsworth entered his room and touched him. He also asked Matthew to put his mouth on Ellsworth’s penis.
Ellsworth denied all three incidents. He claimed that he never took a bicycle ride with Matthew without others present, and that he did not draw Matthew into the woods. He also stated that he did go swimming with Matthew and Stephen, but that nothing sexual occurred. Ellsworth denied that he ever abused Matthew at the cottage.
Little direct or circumstantial evidence supported either side’s version of events. Another cottage resident testified that Matthew had previously told him that Ells-worth had molested him in the afternoon but mentioned nothing concerning the evening abuse about which Matthew testified at trial. Stephen, the boy who accompanied Matthew and Ellsworth on the swimming trip, also testified and stated that he went on two such trips with Matthew, but that the pond had been crowded on both occasions and that he had been within twenty feet of Matthew and Ellsworth and did not notice anything unusual. Ells-worth offered evidence, described below, seeking to cast doubt on Matthew’s credibility but the evidence was not allowed.
The jury convicted Ellsworth on two counts of aggravated felonious sexual assault and five counts of felonious sexual assault. He was sentenced to 18to 37 years in prison, with an additional 14 to 28 years in prison deferred. The New Hampshire Supreme Court affirmed Ellsworth’s conviction. New Hampshire v. Ellsworth, 142 N.H. 710, 709 A.2d 768 (1998). Ells-worth then petitioned for habeas relief in the federal district court on three grounds: first, that the prosecution violated Brady v. Maryland, 373 U.S. 83, 83 S.Ct. 1194, 10 L.Ed.2d 215 (1963), by failing to turn over exculpatory evidence; second, that the trial court violated the confrontation clause, U.S. Const. amend. VI, by refusing to permit him to cross-examine Matthew as to sexual abuse by a babysitter when he was three; and third, that the trial court had similarly erred in excluding testimony from a counselor at the Pine Haven School, where Matthew lived after he left Spaulding, that Matthew falsely accused boys there of peeking at him in the shower and at the toilet and of stealing his toys.
The district court rejected all three constitutional claims and denied the petition. Ellsworth appealed, and a divided panel of this court reversed the district court on all three grounds. Concerned with the prece-dential effect of certain of the panel’s rulings, the en banc court granted rehearing sua sponte and withdrew the panel opinion (as is customary when rehearing en banc is granted). We now resolve the merits, concluding that only one of Ellsworth’s three constitutional claims is substantial and that as to it further proceedings are required.
Appellate review of the district court’s denial of habeas relief is de novo, Almanzar v. Maloney, 281 F.3d 300, 303 (1st Cir.2002), cert. denied, 537 U.S. 817, *4123 S.Ct. 86, 154 L.Ed.2d 22 (2002), but we accord deference to the state court as to issues it actually decided. 28 U.S.C. § 2254(d)(1) (2000). In this case, the state court addressed only Ellsworth’s claim that the trial court violated his rights under the confrontation clause by refusing to allow him to admit testimony from the counselor at Pine Haven. The state court did not address Ellsworth’s other two claims, and thus our review of those claims is de novo. Fortini v. Murphy, 257 F.3d 39, 47 (1st Cir.2001), cert. denied, 535 U.S. 1018, 122 S.Ct. 1609, 152 L.Ed.2d 623 (2002).1
We begin with the Brady claim. Brady requires the prosecution to provide a defendant access to exculpatory evidence that is in the prosecutor’s control. 373 U.S. at 87, 83 S.Ct. 1194; Conley v. United States, 323 F.3d 7, 14 (1st Cir.2003) (en banc). Withheld evidence warrants undoing a conviction only when “there is a reasonable probability that, had the evidence been disclosed to the defense, the result of the proceeding would have been different,” a “reasonable probability” here being one that is “sufficient to undermine confidence in the outcome.” United States v. Bagley, 473 U.S. 667, 682, 105 S.Ct. 3375, 87 L.Ed.2d 481 (1985).
Ellsworth claims that the state withheld evidence as to three matters but only one seems to us potentially powerful enough even arguably to meet the reasonable probability standard.2 This is an intake note written by Jan Smith, Director of the Program for Emotionally Disturbed Boys at Spaulding, indicating that Matthew alleged that he was abused by staff members at Hampstead Hospital, where he stayed prior to his stay at Spaulding, that Matthew expressed concern that he would be sexually assaulted at Spaulding, and that Spaulding would need to take special precautions to protect staff members from false accusations if Matthew enrolled.
The intake note is exculpatory; it suggests that Matthew had made false accusations of sexual abuse against caretakers in the past. But there are two problems: first, the note itself is inadmissible as double hearsay; someone at Hampstead Hospital told Jan Smith that Matthew’s accusations there were false and she then recorded this in her intake note. See New Hampshire v. Winders, 127 N.H. 471, 503 A.2d 798, 802 (1985).3 Second, New Hampshire only permits the defendant to introduce extrinsic evidence of a prior false allegation of sexual assault by the victim if the allegations are similar and the defendant can show that the prior allegations were “demonstrably false.” New Hampshire v. Gordon, 146 N.H. 258, 770 A.2d 702, 704 (2001); accord New *5Hampshire v. White, 145 N.H. 544, 765 A.2d 156, 159 (2000).
The circuits are split on whether a petitioner can have a viable Brady claim if the withheld evidence itself is inadmissible. Most circuits addressing the issue have said yes if the withheld evidence would have led directly to material admissible evidence.4 We have never squarely ruled on this question, but cf. United States v. Hemmer, 729 F.2d 10, 16 n. 3 (1st Cir.), cert. denied, 467 U.S. 1218, 104 S.Ct. 2666, 81 L.Ed.2d 371 (1984); United States v. Ranney, 719 F.2d 1183, 1190 (1st Cir.1983), yet given the policy underlying Brady, we think it plain that evidence itself inadmissible could be so promising a lead to strong exculpatory evidence that there could be no justification for withholding it. Wood v. Bartholomew, 516 U.S. 1, 6-8, 116 S.Ct. 7, 133 L.Ed.2d 1 (1995), implicitly assumes this is so.
Whether the intake note at issue in this case undermines confidence in the verdict is a more difficult question. The clear implication of Jan Smith’s note is that someone at Hampstead Hospital, where Matthew made the prior allegations, told her that the allegations were false. If the defense had known about the note before trial, it presumably could have traced those at Hampstead who could testify as to the circumstances of the allegations and the basis for believing them to be false. Whether the evidence would convincingly establish that Matthew had lied is hard to know but surely the episode most likely would have been investigated and the implication that someone at Hampstead believed it false is strong.
If strong evidence of a prior false accusation exists, it would be very powerful. The setting and type of alleged he are similar, so this evidence would be far more potent than a random unrelated episode of untruthfulness by Matthew. Some evidence of Matthew’s background, and possible instability, was admitted at Ellsworth’s trial. Coupled with such proof, evidence that Matthew had falsely accused caretakers before could easily have created the reasonable doubt necessary to acquit Ells-worth in what was otherwise largely a credibility contest. The lack of any significant corroborating evidence makes this case unusual and heightens the concern about any Brady violation.
But this case presents a further peculiarity. Ellsworth’s counsel now has the lead available to him and a basis for knowing how Jan Smith reached her conclusion. The district court stated that the intake note was prepared after an intake session at Spaulding when Matthew was referred there from Hampstead; and at this intake session, Jan Smith talked with Matthew and his mother, Matthew’s psychiatrist at Hampstead, a Hampstead social worker, and a Hampstead nursing supervisor. However, Ellsworth’s counsel has not explained whether these sources have been questioned; and, if so, whether this has led to admissible evidence or whether evidence once existed but is no longer available; or, if the lead has not been pursued, why not. Curiously, the state has not emphasized this gap.
Nevertheless, it would be very odd for us to require a new trial because of a wrongly withheld lead unless the lead would, or would likely, have led to valuable new evidence which was itself arguably admissible (the final decision is for the *6state court). However unlikely it might be after eight years, the state would be entitled to retry Ellsworth if the writ were granted; yet such a remedy would be incongruous unless evidence existed that might alter the result at such a trial. Ha-beas doctrine is flexible enough for us to condition a grant of the writ on the outcome of a further inquiry into where the lead, even though wrongly withheld, would have taken Ellsworth. Cf. Manko v. United States, 87 F.3d 50, 55 (2d Cir.1996); Stewart v. Coalter, 48 F.3d 610, 617 (1st Cir.), cert. denied, 516 U.S. 853, 116 S.Ct. 153, 133 L.Ed.2d 97 (1995).
If there exists admissible evidence that Matthew made demonstrably false accusations at Hampstead Hospital under similar circumstances and Ellsworth was not otherwise aware of these allegations at the time of the first trial, a new trial is required. If admissible evidence of false accusations never existed, the writ should be denied. If it did or may well have existed but has been lost because of the Brady violation and the ensuing delay in discovery of this fact, Ellsworth may have a good claim to a new trial, cf. California v. Trombetta, 467 U.S. 479, 486-88, 104 S.Ct. 2528, 81 L.Ed.2d 413 (1984), but we need not definitively resolve this issue. It is difficult to foresee every possible outcome of a further inquiry and — apart from a clear-cut decision that the evidence does exist or never existed — the district court must exercise its own judgment in the first instance.
In his supplemental brief, Ellsworth argues for the first time that the state standard for the admission of such evidence— that prior false accusations be not only false but “demonstrably” so — is itself too demanding and therefore unconstitutional. For this proposition, he cites some state court authority but no Supreme Court or federal circuit case in point. As we explain below, the confrontation clause objection is pretty well limited to extreme cases where the state restriction is patently unreasonable, so Ellsworth’s newest claim is an uphill struggle; but we need not resolve the issue now.
The initial question on remand is whether Ellsworth can produce non-hearsay evidence that Matthew previously made false accusations in pertinently similar circumstances. If he cannot do so, the constitutional issue is moot; this is equally so if he produces evidence that Matthew did make “demonstrably false” accusations. If and when Ellsworth produces admissible evidence of prior accusations that were probably false but not “demonstrably false,” he is free to make his new constitutional argument in the district court (subject to whatever other limitations — e.g., Teague v. Lane, 489 U.S. 288, 109 S.Ct. 1060, 103 L.Ed.2d 334 (1989) — might otherwise apply)-
One other issue may be presented on remand. If the state continues to contest the issue, the district court on remand should also decide whether Ellsworth had independent knowledge of the intake note at the time of trial. In general, “[e]vidence is not suppressed if the defendant either knew, or should have known of the essential facts permitting him to take advantage of any exculpatory evidence.” United States v. LeRoy, 687 F.2d 610, 618 (2d Cir.1982), cert. denied, 459 U.S. 1174, 103 S.Ct. 823, 74 L.Ed.2d 1019 (1983). The district court did not resolve the issue because it resolved the Brady claim on a different ground.5
*7The “should have known” standard refers to trial preparation; and whether or not Ellsworth was careless in his perusal of the file while a cottage teacher does not matter. - But if he in fact knew of the note at the time of his trial and failed to pursue the lead, then his Brady claim might well be barred. The state has identified nothing in the record that shows affirmatively that Ellsworth did have timely knowledge of the note, but it does point to an Ells-worth affidavit filed in connection with the habeas petition that — in listing information new to Ellsworth — omits the note or its contents. The omission may or may not be significant; but we cannot dismiss the state’s claim of knowledge out of hand.
Finally, we reach Ellsworth’s claims under the confrontation clause. Ellsworth first argues that the trial court should have allowed him to introduce evidence that Matthew was sexually abused when he was three years old. Ellsworth claims that this evidence would have been relevant as a possible explanation for why . Matthew could describe sexual acts in such detail, thus making more plausible the possibility that Matthew was fabricating his claim against Ellsworth. The district court held that the exclusion of this evidence did violate Ellsworth’s rights under the confrontation clause,, but that the error was harmless.
Harmless or not, we do not think that Ellsworth has shown a violation of the confrontation clause at all. “The Confrontation Clause lies obscurely behind ... claims of evidentiary error because, in a few extreme cases, the Supreme Court has invoked it to overturn state court restrictions on cross-examination or impeachment. However, such a challenge is tenable only where the restriction is manifestly unreasonable or overbroad.” United States v. Gomes, 177 F.3d 76, 81-82 (1st Cir.), cert. denied, 528 U.S. 911, 120 S.Ct. 260, 145 L.Ed.2d 218 (1999). Cf. Chambers v. Mississippi, 410 U.S. 284, 93 S.Ct. 1038, 35 L.Ed.2d 297 (1973) (overturning the defendant’s murder conviction where the court excluded evidence that another person had repeatedly confessed to the murder).
Ellsworth cannot make such a showing here. Courts routinely exclude evidence regarding the sexual history of an alleged victim of sexual abuse. See, e.g., Fed.R.Evid. 412. These restrictions are designed to give sexual assault victims “heightened protection against surprise, harassment, and unnecessary invasions of privacy.” Michigan v. Lucas, 500 U.S. 145, 150, 111 S.Ct. 1743, 114 L.Ed.2d 205 (1991). The evidence of abuse when Matthew was three is far from highly probative as to his sexual knowledge at age twelve. Compare United States v. Begay, 937 F.2d 515, 520-24 (10th Cir.1991).
The earlier abuse was obviously remote in time and different in circumstances and the prosecution never argued that the alleged abuse was Matthew’s only source of sexual knowledge. And, even if the jury did not know on its own that a twelve-year-old had likely acquired some general knowledge of sex, Matthew himself testified at trial that he participated in group discussions at Spaulding regarding topics such as “good and bad touching,” showing an alternate basis for at least some of his sexual knowledge. Finally, Ellsworth himself indicated in his testimony that Matthew had been abused in the past.
Our .result here is similar to the Tenth Circuit’s decision in United States v. Pow*8ell, 226 F.3d 1181 (10th Cir.2000), cert. denied, 531 U.S. 1166, 121 S.Ct. 1128, 148 L.Ed.2d 995 (2001). There, the defendant was accused of sexually assaulting a thirteen-year-old girl. At trial, the defendant sought to introduce evidence regarding the victim’s allegedly flirtatious behavior in the past, but the district court ruled that the evidence was inadmissible under Fed.R.Evid. 403 and 412. Id. at 1197. On appeal, the Tenth Circuit rejected the defendant’s confrontation clause claim, explaining “the relevance of [the victim’s] alleged conduct to rebut any inference the jury may have drawn regarding her sexual naivety is too attenuated....” Id. at 1199.
Perhaps under state law the trial judge should have admitted the episode as to what happened when Matthew was three; quite likely many judges would have admitted it. But trial judges are constantly making on-the-spot judgments as to whether evidence, although formally relevant, is too remote, likely to lead to unnecessary excursions, or partly or wholly duplicative — the range of considerations embraced in the federal courts by Rule 403’s balancing test. Fed.R.Evid. 403. Close calls are common and, right or wrong, do not thereby become constitutional violations. Delaware v. Van Arsdall, 475 U.S. 673, 678-79, 106 S.Ct. 1431, 89 L.Ed.2d 674 (1986).
Ellsworth’s remaining claim is that the trial court violated the confrontation clause by refusing to allow him to offer testimony from a counselor at the Pine Haven School, the facility where Matthew lived after he left Spaulding, that Matthew falsely accused boys at the school he attended after Spaulding of peeking at him in the shower and at the toilet and of stealing his toys. Here Ellsworth has a further hurdle because the state court considered the constitutional claim and rejected it, so under the statute we can only reverse if the state court decision was contrary to, or involved an unreasonable application of, clearly established federal law as determined by the Supreme Court. 28 U.S.C. § 2254(d)(1).
Quite apart from the higher standard, there is nothing unusual about limiting extrinsic evidence of lies told by a witness on other occasions; under the Federal Rules of Evidence, exclusion of such evidence is the usual rule and even cross-examination as to such lies is limited. The theory, simple enough, is that evidence about lies not directly relevant to the episode at hand could carry courts into an endless parade of distracting, time-consuming inquiries. In this instance, a lie about toy stealing or peeping at a different time and location from the alleged sexual abuse by Ellsworth is classic “collateral” evidence regularly excluded in federal criminal trials. See Fed.R.Evid. 608(b).
The judgment of the district court is vacated and the case is remanded for further proceedings consistent with this opinion.
. The state argues that Fortini is undermined by Early v. Packer, 537 U.S. 3, 123 S.Ct. 362, 365, 154 L.Ed.2d 263 (2002). We do not think this is so — Early said only that deference does not depend on the state court citing federal case law — but Ellsworth's other claims fail even under a de novo standard and therefore fail a fortiori under a more deferential one.
. The other two were (first) a critical incident report written by someone at Spaulding indicating that Matthew had been having dreams regarding his prior sexual victimization and (second) Spaulding records indicating that the staff at Spaulding was trying to get the case against the person who had previously sexually abused Matthew re-opened.
. The note might be admissible as a business record of Spaulding if it recorded Smith's own knowledge of a prior false accusation, N.H. R. Evid. 803(6); but a note recording what someone outside the hospital told her is not within the business records exception when offered for the truth of the report, N.H. R. Evid. 805. See generally Ricciardi v. Children’s Hosp. Med. Ctr., 811 F.2d 18 (1st Cir.1987); Johnson v. Lutz, 253 N.Y. 124, 170 N.E. 517 (1930).
. See, e.g., United States v. Gil, 297 F.3d 93, 104 (2d Cir.2002); Bradley v. Nagle, 212 F.3d 559, 567 (11th Cir.2000), cert. denied, 531 U.S. 1128, 121 S.Ct. 886, 148 L.Ed.2d 794 (2001); United States v. Phillip, 948 F.2d 241, 249 (6th Cir.1991). But see Hoke v. Netherland, 92 F.3d 1350, 1356 n. 3 (4th Cir.1996).
. The district court held that the intake note was not material because Ellsworth knew that Matthew's parents feared that he had made false allegations in the past and because the district court thought that the prosecution’s case was quite strong. In our view, evidence *7that Matthew had in fact falsely accused other caretakers of sexual abuse is much more potent and, as to the strength of the case against Ellsworth, it appears to us to have rested largely on Matthew's otherwise uncorroborated allegations of abuse.