Defendant Sidney Osman appeals from the judgment of the United States District Court for the Southern District of New York convicting him, after a jury trial, of the sole count against him, conspiracy to distribute and possess with intent to distribute one hundred grams or more of phencyclidine (“PCP”), in violation of 21 U.S.C. §§ 812, 841(a)(1), 841(b)(1)(A) and 846. Osman argues that the admission of testimony from the case agent “interpreting” telephone calls, in which the agent had not participated, but had reviewed, was error and that the evidence was insufficient to support a conviction. In this opinion, we address Osman’s appeal. For the reasons set forth below, the judgment of conviction as to Osman is vacated and the case is remanded.
All of Osman’s co-defendants pled guilty. One, Mark Reneau, appeals from a judgment convicting him following his plea of guilty to both counts of a two-count information charging him with both the same conspiracy and a substantive PCP offense involving 5.4 kilograms of a substance containing PCP. In a summary order filed today, we resolve the appeal by Reneau and affirm his conviction.
BACKGROUND
At Osman’s trial, the Government established the existence of a substantial conspiracy to distribute large quantities of PCP. The PCP was manufactured in California and then transported tó New York City by Mark and Elaine (Betsy) Reneau and distributed in New York by Raymond Grinage, the lead defendant, to local dealers. Osman was charged with being one of those local dealers. Elaine Reneau, a cooperating witness, described the overall conspiracy and the role of Grinage and other individuals; including her husband Mark Reneau, in the conspiracy. Although Elaine Reneau testified that she knew Osman from high school in Belize, that she had seen him on a few occasions over several years on the street or at a social club in upper Manhattan that Belize-ans frequented and that she had seen Gri-nage in Osman’s presence, she did not implicate Osman in the conspiracy nor did she testify to any involvement by Osman in the drug trade.
The only other witness was the case agent, Scott Seeley-Hacker, a special agent with the Drug Enforcement Agency (“DEA”), who' had supervised the wiretap of Raymond Grinage’s cellular phone. The Government elicited at great length Agent Seeley-Hacker’s background, his training and his experience in narcotics investigations. In particular, the Government elicited his experience in listening to drug traffickers speak on the telephone. He testified that, based upon his training and experience, he was familiar with the manner in which drug dealers speak, a manner that “is very cautious and often coded.”
Received into evidence were some 2000 wiretapped calls from Grinage’s cellular *748phone. The agent testified that he had listened to all of them; some pertinent calls he listened to “upwards of 20 plus times.” Based upon his “investigation” and his understanding of the calls, he described in detail the Grinage organization and the roles of Grinage’s supplier Rodney Thomas and customers, including Sherry Cannon, Michael Garvin and Edgar Arroyo. In his discussion of the calls between Grinage and individuals other than Osman, the agent did not identify any mention of Osman.
Only approximately thirteen of the 2000 calls were played to the jury. The agent interpreted two calls played to the jury from Garvin and Cannon asking, respectively, for “about two” and “one and a half’ as requests for quantities, by ounce, of PCP. The agent had previously testified that ounce quantities of PCP are diluted and resold for personal use as individual “dips.” The Government also played a call between Elaine Reneau and Grinage, which, according to the agent, related to PCP distribution. The agent described his understanding of another call played to the jury as reflecting an arrangement between Arroyo and Grinage at which Arroyo would pick up drugs. According to the agent, on the 2000 calls, there was only one explicit reference to PCP and that was made by Grinage’s girlfriend. The agent further testified that the participants were not using code.
Only three of the 2000 calls to Grinage were from Osman, and each was played to the jury in its entirety. The agent was permitted to interpret each of these calls to the jury, as he had interpreted the others. In the first call, made by Osman to Grinage on July 21, 2001, they had a dispute over money owed by Osman to Grinage in which Grinage cursed and threatened Osman, and Osman assured Grinage he would bring the money. The agent testified that, “Based on my knowledge of Raynard Grinage and his operation and the context of this call, I believe that this is a call where money is owed him over a drug debt, and he’s demanding his money to be paid.” The agent specifically identified the debt as being over a PCP deal. His basis for this conclusion was that Grinage had made other calls seeking drug money from other people. The second call, which the Government referred to in summation as the “key” to its case against Osman, occurred on August 7, 2001. Cannon tells Grinage she is calling him on Osman’s phone and that Osman wants to speak to him. After saying that he doesn’t want to get himself in any “situation,” Osman tells Grinage, “I need something bad, bad, bad,” and “I need about nearly four.” Grinage responds, “I’m not up with the bullshit.” Osman says, “No way, you, you, you look ... I’m going to pass there. I’m not gonna chat with you right now. I am coming around there.” Grinage responds, “All right.” After playing the tape, the prosecutor went through the conversation line by line, asking the agent his understanding of each sentence. In sum, the agent testified that Osman did not want to be in a situation where he owed Grinage money, as reflected in the earlier conversation, but that he badly needed four ounces of PCP; when Grinage rebuffed him, it was because of his failure to pay a prior drug debt and unwillingness to do a PCP deal with Osman without getting his money at the same time. When Osman said he would come to Gri-nage in person, the agent testified they were confirming to meet in person at the end of the call. The third call occurred about 12 minutes later; in it Osman indicates he dialed the wrong number and did not mean to call Grinage.
On cross-examination, the agent adhered to his interpretations, while acknowledging them to be “subjective.” The agent also *749told the jury that his interpretations were “based on my knowledge of the entire investigation.” In response to a question whether he assumed that Osman’s conversation with Grinage was about drugs because of his knowledge regarding Gri-nage’s activities, the agent said “yes,” even though he had no personal knowledge at that time that Osman was a drug dealer. The agent also believed that Osman was with another person who wanted to buy drugs based upon the “totality of the investigation,” and, he said, that conclusion was “consistent with other observations I made about other drug purchases that occurred during this investigation.” He agreed that he had not made any observations of Osman.
Also in evidence were telephone records establishing that numerous additional, but unrecorded, telephone calls were made from telephones associated with Osman to Grinage over the period January to August 2001, within the dates of the charged conspiracy, and that, upon his arrest, Osman denied knowing Grinage. Finally, the plea allocutions of co-defendants Grinage, Cannon, Michael Garvin and Edgar Arroyo were admitted into evidence with appropriate limiting instructions that they were to be used only to establish the conspiracy and what the individuals themselves, and not Osman, had done. However, the taped conversations, other than Osman’s, which the Government played to the jury were those between Grinage and Elaine Reneau, Cannon, Garvin and Arroyo, and it was the conversations of those individuals, who, the Government emphasized, had pled guilty to conspiracy, which the Government compared to the conversations of Osman. The Government also argued that Osman must have known he was dealing with drug dealers because both Cannon and Grinage had admitted that they were.
DISCUSSION
I. Standard of Review
On appeal, Osman contends that the district court erroneously admitted Agent Seeley-Hacker’s testimony and that the error was not harmless. We review a district court’s decision to admit evidence for abuse of discretion. United States v. Garcia, 291 F.3d 127, 139-40 (2d Cir.2002).
II. Lay Opinion Testimony
Rule 701 of the Federal Rules of Evidence provides that, where a witness is not testifying as an expert, “the witness’ testimony in the form of opinions or inferences is limited to those opinions or inferences which are (a) rationally based on the perception of the witness, [and] (b) helpful to a clear understanding of the witness’ testimony or the determination of a fact in issue.” The burden is on the party wishing to introduce lay opinion testimony to establish the proper foundation. Garcia, 291 F.3d at 140. We conclude that the Government has not met its burden of establishing the foundational requirement of Rule 701(b), which “is designed to provide ‘assurance[ ] against the admission of opinions which would merely tell the jury what result to reach.’ ” United States v. Rea, 958 F.2d 1206, 1215 (2d Cir.1992) (quoting Fed.R.Evid. 704 Advisory Committee Notes on 1972 Proposed Rules).
III. Agent Seeley-Hacker’s Testimony
The defendant argues that the agent’s testimony interpreting the meaning of the telephone conversations improperly allowed the case agent to provide the jury with expert testimony based on hearsay information which he had obtained in the course of the investigation, and usurped the function of the jury, in violation of the principles set forth in this court’s recent decision in United States v. *750Dukagjini, 326 F.3d 45 (2d Cir.2003), which was decided after Osman’s trial.
The Government disavows that the case agent was either offered as, or testified as, an expert. It expressly argues that he did not testify to coded language and did not base his testimony on specialized knowledge. Rather, it argues, the case agent’s testimony was based on his review of all of the recorded telephone conversations and was proper as lay opinion testimony under Rule 701 of the Federal Rules of Evidence. It further argues that, in any event, the concerns expressed in Dukagjini are not implicated here. Specifically, the Government argues that it was nearly impossible for the Government to play all the calls or the jury to listen to them all during deliberations. “Accordingly, it was helpful for the jury to have the benefit of the testimony of an individual who reviewed every call” and could assist the jury by placing the Osman call in context and highlighting its similarities to other calls in furtherance of the conspiracy.
This argument fundamentally misunderstands Rule 701(b). Were it to be accepted, there would be no need for the trial jury to review personally any evidence at all. The jurors could be “helped” by a summary witness for the Government, who could not only tell them what was in the evidence but tell them what inferences to draw from it. That is not the point of lay opinion evidence. Indeed, in its summation, the Government seemed to acknowledge that lay opinion was unnecessary, when the prosecutor told the jury, “[Y]ou can make your own interpretations, ladies and gentlemen. You can use your own common sense to know that these were PCP calls.” Cf. United States v. Cruz, 981 F.2d 659, 662 (2d Cir.1992) (noting the Government’s “telling slips of the tongue” when the prosecutor stated in summation that the answer to a key question on which the Government had elicited expert testimony-which the court found was improperly admitted was based upon “common sense” and “logic[j”). The agent interpreted both the calls that the jury heard and the calls that the jury did not hear. In doing so, he usurped the function of the jury to decide what to infer from the content of the calls and in particular what to infer from the calls involving Osman as to Osman’s participation in the drug ring. Cf. Dukagjini, 326 F.3d at 54 (allowing a case agent who testifies as an expert to provide sweeping conclusions about a defendant’s activities “may come dangerously close to usurping the jury’s function”) (quoting United States v. Nersesian, 824 F.2d 1294, 1308 (2d Cir.1987)).
In addition, the Government’s claim that the agent’s testimony was based solely on his listening to the telephone conversations is not supported by the record. The agent testified to the jury that his interpretations were “based on my knowledge of the entire investigation.” Even though the Government did not seek to qualify him as an expert, he testified at great length about his background and expertise as a drug investigator and explained at length his role as case agent. Whether labeled as an expert or not, the risk that he was testifying based upon information not before the jury, including hearsay, or at the least, that the jury would think he had knowledge beyond what was before them, is clear. See Dukagjini, 326 F.3d at 53. Indeed, the Government in its rebuttal summation told the jury that “the agent has the background to make interpretations,” suggesting either expertise, for which he had not been qualified, or investigative information not before the jury. On oral argument of the appeal, the Government acknowledged that this statement was improper under Dukagjini. In that case, we addressed at length our concerns about a case agent testifying as an expert as to *751coded language and then going “beyond interpreting code words and summariz[ing] his beliefs about the defendant’s conduct based upon his knowledge of the case.” 326 F.3d at 53. Although in this case, unlike in Dukagjini, the case agent did not state explicitly that he was relying on his conversations with cooperating defendants, he did state that he was relying upon the “entirety” and the “totality” of the investigation, of which he was in charge.
In sum, the agent’s testimony as to his interpretations of the calls went beyond permissible lay opinion testimony under Rule 701(b) because, rather than being helpful to the jury, it usurped the jury’s function. Moreover, as in Dukagjini, supra, the agent was presented to the jury with an aura of expertise and authority which increased the risk that the jury would be swayed by his testimony, rather than rely on its own interpretation of the calls.
IV. Harmless Error Analysis
Although the evidence was legally sufficient to convict Osman, the error in admitting the case agent’s interpretations of the calls was not, as the Government argues, harmless. In the absence of a constitutional violation, the erroneous admission of evidence is subject to the harmless error test of Federal Rule of Criminal Procedure 52(a). See Kotteakos v. United States, 328 U.S. 750, 764-65, 66 S.Ct. 1239, 90 L.Ed. 1557 (1946) (a non-constitutional error is harmless where an appellate court has a fair assurance that the error did not substantially affect the verdict). Where the erroneously admitted evidence goes to the heart of the case against the defendant, and the other evidence against the defendant is weak, we cannot conclude that the evidence was unimportant or was not a substantial factor in the jury’s verdict. See Wray v. Johnson, 202 F.3d 515, 524-30 (2d Cir.2000); United States v. Forrester, 60 F.3d 52, 64-65 (2d Cir.1995).
The testimony of the cooperator and the plea allocutions of his co-defendants did not directly implicate Osman in the conspiracy. (After submission of the briefs but before oral argument of this appeal, the Supreme Court decided Crawford v. Washington, 541 U.S. 36, 124 S.Ct. 1354, 158 L.Ed.2d 177 (2004), which prescribed a limited rationale for the admission of prior testimonial statements. Crawford explicitly called into question cases, including cases in this Circuit, which had upheld the use of co-defendant plea allocutions for the purpose of establishing the existence of a conspiracy. Crawford 124 S.Ct. at 1372. On oral argument the Government conceded that, in this case, the plea allocu-tions would not have been admissible under Crawford but argued that, applying a harmless error analysis, there remained sufficient evidence to convict the defendant. Since, as discussed below, the trial error that was briefed was not harmless, we need not address here the implications of Crawford.)
The phone calls Osman made to Gri-nage, as interpreted by the agent, were the principal evidence implicating him. The central focus in the testimony was on the agent’s interpretations of the calls, which were, in the Government’s words, “the key” to the Government’s case. Although the prosecution did not expressly say that it was relying on the agent’s interpretations until the rebuttal summation, the main summation tracked those interpretations and asked the jury to accept them. In rebuttal, the prosecutor argued to the jury that “the agent has the background to make interpretations.” Since the calls required the drawing of inferences before they could be seen as inculpatory, the improper interpretations *752from the agent were central to the Government’s case.
In the absence of the agent’s interpretation of the July 21st call to mean that Grinage was demanding that money owed to him over a PCP debt be paid and that, in the August 7th call, Grinage did not refuse to sell drugs to Osman, but only refused to sell him drugs on consignment, the evidence that Osman’s conversations related to drug dealing was thin. There was no reference to PCP or drugs on the calls, nor was there any evidence that the calls were in code. There was also no evidence that in fact Grinage and Osman had met after the August 7th call or that they had exchanged anything.
The trial judge on more than one occasion characterized the evidence against Osman as “weak.” After brief deliberations, the jurors sent a note that they were hung and, only after a modified Allen charge, Allen v. United States, 164 U.S. 492, 17 S.Ct. 154, 41 L.Ed. 528 (1896), did they reach a verdict. This was not a case where the prosecution presented a “formidable array of admissible evidence” and where the agent’s inadmissible testimony was “merely corroborative and cumulative,” as in Dukagjini, 326 F.3d at 62. On the contrary, we conclude that the inadmissible evidence had a substantial effect on the jury’s verdict. This was a close case. Agent Seeley-Hacker’s testimony went to the crux of the Government’s case, see Hester v. BIC Corp., 225 F.3d 178, 185 (2d Cir.2000), and the jury may well have afforded unusual authority to the agent, who was presented as having expertise, as well as knowledge beyond that available to the jury. See Dukagjini, 326 F.3d at 53. The error was not harmless.
CONCLUSION
As to defendant Sidney Osman, for the foregoing reasons, the judgment of conviction is vacated and the matter is remanded for further proceedings. As to defendant Mark Reneau, for the reasons stated in the accompanying summary order, the judgment of conviction is affirmed.