OPINION OF THE COURT
SMITH, Circuit Judge.Robert Contini and Richard Patterson appeal from an order entered on November 21, 2003, in which the United States District Court for the Western District of Pennsylvania granted summary judgment in favor of defendants Bob Cranmer and Mike Dawida, both of whom were sued individually and in their role as commissioners on the Allegheny County Board of Commissioners. Contini and Patterson were employed for years on the Allegheny County public payroll, ascending to the positions of Park Manager and Recreation Manager, respectively. They were discharged from their county jobs on April 2, 1997, and later filed suit under 42 U.S.C. § 1983, asserting that they had been improperly terminated on account of their political affiliation in violation of the First Amendment. The District Court granted summary judgment in favor of Cranmer and Dawida, holding that Contini and Patterson failed to establish a prima facie case under our three-part test governing claims of discrimination on the basis of political affiliation. We will affirm the judgment of the District Court.
Because we write only for the parties, we restrict our discussion to those facts and legal principles necessary to resolution of this appeal. We exercise plenary review of the District Court’s decision to grant summary judgment. Assaf v. Fields, 178 F.3d 170, 171 (3d Cir.1999). In doing so, we must apply the same test that the District Court must apply. See Chipollini v. Spencer Gifts, Inc., 814 F.2d 893, 896 (3d Cir.1987) (en banc). Reviewing the record as a whole, we will “draw all reasonable inferences in favor of the non-moving party” and will not weigh the evidence or make credibility determinations. Reeves v. Sanderson Plumbing Products, Inc., 530 U.S. 133, 150, 120 S.Ct. 2097, 147 L.Ed.2d 105 (2000). If it appears that “there is no genuine issue as to any material fact” and the movant is entitled to judgment as a matter of law, we will affirm a grant of summary judgment. Fed. R.Civ.P. 56(c). When a moving party has carried its burden under Rule 56(c), the non-moving party “must do more than sim*188ply show that there is some metaphysical doubt as to the material facts.” Matsushita Electric Industrial Co. v. Zenith Radio Corp., 475 U.S. 574, 586, 106 S.Ct. 1348, 89 L.Ed.2d 538 (1986). The non-moving party “must come forward with ‘specific facts showing that there is a genuine issue for trial. ’ ” Id. at 587, 106 S.Ct. 1348 (quoting Fed.R.Civ.P. 56(e)). Where the record taken as a whole could not lead a rational trier of fact to find for the nonmoving party, there is no genuine issue for trial. Id.
We have adopted a three-part test for determining whether a public employee has established a prima facie case of discrimination based upon political affiliation. The employee must prove (1) that the employee works for a public agency in a position that does not require a political affiliation, (2) that the employee maintained an affiliation with a political party, and (3) that the employee’s political affiliation was a substantial or motivating factor in the adverse employment decision. See Robertson v. Fiore, 62 F.3d 596, 599 (3d Cir.1995) (citing Laskaris v. Thornburgh, 733 F.2d 260, 265 (3d Cir.1984)). Here, the defendants conceded that the first prong was satisfied. The District Court held that Contini and Patterson failed to establish the existence of a prima facie case under the second and third prongs of our three-part test, and thus granted summary judgment in favor of Cranmer and Dawida.1
On appeal, Contini and Patterson argue that the District Court erred by granting summary judgment, in that it invaded the province of the jury and overlooked the existence of a genuine dispute concerning issues of material fact. The appellants’ position lacks merit. The third prong of our three-part test for discrimination based upon political affiliation requires a public employee to show that his political affiliation was a substantial or motivating factor in the adverse employment decision. See Robertson, 62 F.3d at 599. It is well-settled that implicit in this third prong is a requirement that the plaintiff produce evidence to show that the defendant knew of the plaintiffs political persuasion. See Goodman v. Pennsylvania Turnpike Comm., 293 F.3d 655, 664 (3d Cir.2002) (citing Laskaris, 733 F.2d at 265). In Stephens v. Kerrigan, 122 F.3d 171, 177-80 (3d Cir.1997), we characterized the inquiry under the third prong as involving two parts — “knowledge” and “causation.”
With respect to the defendants’ knowledge of Contini and Patterson’s political affiliation, the District Court addressed each of the two plaintiffs separately. The District Court noted that Cranmer testified that he did not know Contini or know what Contini’s job was, and the District Court observed that Contini failed to submit any contrary evidence or to otherwise dispute this testimony. Indeed, Contini and Patterson did not even file a response to the defendants’ summary judgment motion below, but instead merely adopted the response filed by other plaintiffs in a related case that was pending before the District Court. There is no indication in the *189record that this response adopted by Contini and Patterson ever addressed the specific evidence submitted by Cranmer and Dawida in support of their motion for summary judgment in the present case, and on appeal Contini does not challenge the District Court’s finding that Cranmer’s testimony was unrebutted. While the District Court did not specifically address Dawida’s knowledge concerning Contini, Contini on appeal points to no evidence that would indicate that Dawida knew of Contini’s claimed political affiliation. Moreover, the record indicates that the chief witness relied upon by Contini and Patterson, an Allegheny County political activist named Patrick Risha, indicated that he learned of rumors that Contini was to be fired from Contini himself, and he never spoke with either Cranmer or Dawida concerning this issue.
As to Patterson, the District Court found that he had provided no evidence to indicate that either of the defendants knew anything at all concerning his claimed political affiliation. Cranmer testified in his deposition that he did not know who Patterson was or what Patterson did, and the District Court correctly noted that “Patterson failed to submit any evidence in response to defendants’ motion for summary judgment to dispute Cranmer’s testimony.” Nor does Patterson point to any such evidence on appeal. Similarly, Dawida testified in his deposition that he did not know Patterson, and the District Court noted that Dawida’s testimony was reinforced by Patterson himself, who admitted during his deposition that Dawida “wouldn’t have a clue” who he [Patterson] was.
The evidence submitted by the defendants in support of their motion for summary judgment supports the conclusion that the defendants did not know of any political affiliation that may have been held by either Contini or Patterson. Contini and Patterson failed to challenge or rebut this evidence in the proceedings before the District Court, and they likewise fail to do so on appeal. The District Court was therefore correct in entering summary judgment in favor of the defendants based upon the failure of Contini and Patterson to demonstrate a necessary element of their prima facie case. See Laskaris, 733 F.2d at 265 (entering judgment in favor of defendants, and emphasizing that “there is simply insufficient evidence to show that any of the defendants knew that [the plaintiffs] were Democrats”).
Moreover, even if Contini and Patterson could show that the defendants knew of their claimed political affiliations, the District Court still properly granted summary judgment in favor of the defendants based upon the failure of the plaintiffs to establish a prima facie case that their political affiliations were a “substantial or motivating factor” in their termination. The District Court observed that the defendants, in support of their motion for summary judgment, submitted extensive evidence demonstrating that the 1997 terminations of Contini and Patterson resulted from a reorganization effort in which various Allegheny County departments were consolidated, and redundant administrative positions were eliminated. The District Court relied upon a February 26,1997 memorandum from the County Manager to the County Commissioners, in which the County Manager discussed the upcoming consolidation of multiple departments into a newly-created Department of Public Works. This memorandum noted that the consolidation process had resulted in the discovery of “redundancies in the administrative functions,” and that as a result a number of redundant positions were to be eliminated. This memorandum included an attachment listing the positions to be eliminated within various departments, *190and identifying the individuals who currently held those positions. Contini and Patterson were among the employees listed.
The District Court also cited a March 31, 1997 memorandum, in which the County Manager presented the final reorganization plan, and noted that the consolidation of separate departments into the newly-created Department of Public Works had required a “significant staff cutback.” This memorandum indicated that the cutbacks had been implemented “by searching out duplicative or redundant functions and reducing them to needed levels.” Two days later, on April 2, 1997, Contini and Patterson were terminated. The District Court noted that the PAA-100 forms executed in connection with their terminations were signed by Cranmer and Dawida, and that the reason for termination was described as: “INVOLUNTARY TERMINATION DUE TO PUBLIC WORKS REORGANIZATION.”
The District Court properly held that in the absence of any countervailing evidence submitted by the plaintiffs, this evidence submitted by the defendants demonstrated that Contini and Patterson had been terminated as a result of Allegheny County’s organizational restructuring, rather than because of any political affiliation. The record on appeal does not include any indication that Contini and Patterson challenged the defendants’ reliance on these memoranda below. Instead, Contini and Patterson argue that deposition testimony from Patrick Risha, in which he speculated that Contini and Patterson were terminated because of “politics,” was sufficient to create a triable issue of material fact.
This argument is unavailing. Risha’s deposition transcript, included in the record on appeal, reveals that Risha explicitly and repeatedly disclaimed any actual knowledge concerning the reason Contini and Patterson were terminated. With respect to his knowledge concerning the motivation behind the termination of Contini and Patterson, Risha at various points testified that he was just offering “my own personal opinion,” that “I can just tell you what I assume,” that “I don’t have any proof on this,” that his opinion was based on “just talk through the grapevine,” and that he was relying upon “the rumor mill.” We have repeatedly emphasized that a plaintiff in a political affiliation discrimination case may not rely upon mere speculation and conelusory assertions to establish that a discharge was politically motivated, and thus Risha’s testimony does not undermine the basis for the District Court’s decision to grant summary judgment in favor of the defendants. See, e.g., Liotta v. Borough of Springdale, 985 F.2d 119, 122 (3d Cir.1993) (rejecting reliance on conelusory affidavits submitted by council members who opposed plaintiffs discharge, and stating that “an employee’s case must be based on more than speculation. Otherwise, there is a danger that disputes among public officials of every nature would be characterized as ‘political’ in an attempt to bring the action within the First Amendment framework”); Laskaris, 733 F.2d at 265-66 (affirming grant of summary judgment where evidence showed only “a vague aura of politically motivated patronage firings” and where a verdict for the plaintiffs “could only have been speculative”).
Perhaps recognizing the weakness of Risha’s testimony, Contini and Patterson also assert that while the record contains evidence that an organizational restructuring was planned, it contains no evidence that a restructuring actually occurred, and that the absence of such evidence could lead a jury to infer that the discharge of Contini and Patterson was politically motivated. Contrary to this unfounded assertion, we *191believe the District Court properly relied upon the various memoranda described above in concluding that Allegheny County implemented an organizational restructuring during the first half of 1997, and that Contini and Patterson were terminated in connection with this restructuring. Thus, in the absence of any contrary evidence from Contini and Patterson, we believe the District Court properly entered summary judgment in favor of the defendants on the ground that Contini and Patterson failed to establish a prima facie case in support of their political discrimination claim. The judgment of the District Court will be affirmed.
. The District Court's analysis focused in part upon whether Contini and Patterson had established that Cranmer and Dawida even knew who they were, and whether Cranmer and Dawida would have been aware of any political affiliation that either Contini or Patterson may have had. The District Court addressed this issue in the context of whether Contini and Patterson had presented sufficient evidence to establish their affiliation with a political party, as required by the second prong of our three-part test. However, as set forth below, we believe the District Court's analysis of the defendants' knowledge with respect to Contini and Patterson is more appropriately characterized as part of the inquiry under the third prong.