This is a consolidation of three appeals arising from two diversity actions brought in the Northern District of California. Such consolidations not infrequently induce appellate dispositions that resemble a vehicle made from mismatched spare parts. This disposition, unfortunately, did not escape this infirmity.
The first action was initiated on July 6, 1970 by appellee Telfon Communications Corporation (Telfon) for the purpose of terminating a franchise agreement with appellant Thomas W. Moore. Moore filed a counterclaim against Telfon and appellees William A. Anderson, Telfon’s president, and Bear, Stearns & Co. (Bear Stearns), a substantial stockholder and financier of Tel-fon. The counterclaim was dismissed with prejudice for failure to prosecute by a final order issued October 26, 1976. The order also denied Moore’s motion for change of venue under 28 U.S.C. § 1404(a), granted a motion to amend a cross-complaint filed by Moore against appellees Telfon, Anderson, Bear Stearns, Feldman, Waldman & Kline, Telfon’s former attorneys, and the three insurance companies, and simultaneously dismissed the cross-complaint with prejudice. Moore appeals this order, claiming that the district court’s disposition of the counterclaim, cross-complaint and motion for change of venue amounted to an abuse of discretion.
During the pendency of the counterclaim, appellee Broad, Khourie & Schulz moved to withdraw as Moore’s attorneys of record and requested the district court to fix fees and disbursements. The court granted the former motion, but denied the latter for lack of jurisdiction. Moore claims the district court erred by refusing to fix fees and disbursements.
The second action, filed by Moore against Telfon and Anderson, alleged violations of the California and Federal wiretapping and eavesdropping statutes as well as common law invasion of privacy. After a three week trial the case resulted in a jury verdict for Telfon and Anderson. Moore appeals, contending that there was insubstantial evidence to support the verdict, that a statement of defense counsel in final argument prejudiced the jury in favor of defendants, and that certain jury instructions were erroneous.
We decline to overturn the jury verdict in the privacy action. Also, we hold that the order denying the motion to fix attorney’s fees and disbursements was not an abuse of discretion. Furthermore, we find that the district court did not abuse its discretion in dismissing the counterclaim or in denying the motion for change of venue, and, accordingly, those portions of the October 26, 1976 order disposing of the counterclaim and venue motion are affirmed. However, for reasons hereinafter stated, we remand to the district court that portion of the order disposing of the cross-complaint.
I.
Statement of the Case.
Telfon operated a radio broadcasting correspondence school known as “Columbia School of Broadcasting.” Although its principal place of business was San Francisco, courses were marketed nationwide through franchised distributors. Moore, a resident of New York, contacted Anderson, the creator and founder of the school, for the purpose of obtaining a franchise in 1966.
Although Moore desired the New York franchise, he ultimately purchased the Hartford, Connecticut franchise. The New York franchise was beyond Moore’s means and Anderson intended to reserve that market for Columbia itself. Nevertheless, Moore extended his operations into the New York area, violating a Telfon agreement with Columbia Broadcasting Systems. An*963derson and Moore entered into negotiations over this and other problems arising from Moore’s activities in the course of which Moore threatened to institute a multi-mil-lion dollar antitrust suit if Anderson did not buy him out at an equal figure.
A. The Privacy Action.
On July 6, 1970, Telfon instituted an action to terminate the franchise agreement. Moore filed a counterclaim against Telfon, Anderson, and Bear Stearns, alleging antitrust violations, as well as breach of contract. Discovery disclosed that Anderson had recorded several conversations with Moore during the course of their business relationship. It had been a common practice for Anderson to record conversations with franchisees pertaining to statistical information and advertising leads. Anderson did not regard these conversations as confidential because the information was to be related to other officers and employees. During the period when Moore was threatening to institute lawsuits, Anderson recorded seven conversations without Moore’s consent for the purpose of documenting threats of extortion. On the basis of these conversations Moore sued Anderson and Telfon for common law invasion of privacy, violations of the California invasion of privacy statute (C.P.C. §§ 630-637.2), the Omnibus Crime Control and Safe Streets Act of 1968 (18 U.S.C. § 2510 et seq.). The jury found for defendants on July 21, 1975. Although Moore represents himself on this appeal, he was represented by counsel during the entire trial.
B. Attorney’s Fees.
On September 8, 1975, after Telfon was adjudicated bankrupt, Telfon’s original claim against Moore was dismissed with prejudice for failure to prosecute. Meanwhile, Moore, dissatisfied with his counsel’s conduct of the privacy action and the pending antitrust suit, discharged his counsel, Broad, Khourie & Schulz. Thereafter, on January 15, 1976, Broad, Khourie & Schulz moved to withdraw as attorneys of record in the antitrust case, requesting the court to fix fees and disbursements. Moore filed a countermotion disputing his liability for attorney’s fees. The court granted the motion to withdraw but declined to fix fees and disbursements.
C. The October 26, 1976 Order.
A final order, issued October 26, 1976, dismissed the antitrust counterclaim with prejudice for failure to prosecute. The counterclaim had been pending for six years. Moore’s entire prosecution thereof consisted of a Request For Production of Documents on April 14, 1972, and the taking of a deposition of an officer of Bear Stearns. He resisted defendant’s discovery efforts. Moore’s filing on August 2, 1976, a motion to continue the trial date and to grant an indeterminate extension of time precipitated the October 26, 1976 dismissal. Moore’s reasons for the delay were the pendency of the privacy action and the withdrawal of his attorneys of record, which he claimed left him without the resources or documents necessary to prosecute the counterclaim. The district court did not credit the excuse, stating that Moore appeared to have made a deliberate choice to prosecute another suit instead. It also denied Moore’s motion for change of venue, reasoning that “it came too late” and the “demonstration which was made to support [it] is not nearly adequate ... to justify a transfer.”
A third motion, which has been interchangeably designated as either a motion to amend the counterclaim or a cross-complaint, also was disposed of at hearings on September 22, 1976. The October 26, 1976 order accurately reflects the district court’s disposition of the motions to dismiss the counterclaim and for change of venue. It is by no means clear that this order accurately reflects the district court’s intended disposition of the cross-complaint or amendment to Moore’s counterclaim.
To understand the uncertainty it is necessary to relate certain pertinent events which occurred at the hearing which led to the October 26, 1976 order. At the hearing the district judge treated the cross-corn-*964plaint as a motion for leave to amend the counterclaim.1 This clearly was denied and the district judge informed Moore that he could file the claims alleged in the cross-complaint elsewhere.2 A proposed order, prepared at the request of the district judge by cross-defendant’s counsel, for the most part accurately represented the oral rulings. That order stated, however, that the “Cross-complaint is hereby Dismissed with Prejudice.” (C.T. 1176). The district judge corrected this misstatement by interlining the phrase and initialling it, thus conforming the order to his earlier ruling. (C.T. 1176). Perhaps because Moore had filed a petition for reconsideration of the rulings of September 22, this order did not become the final order. Rather, the district judge asked cross-defendant’s counsel to prepare another order, reflecting his rulings, but incorporating the denial of the motion for reconsideration. (C.T. 1172).
The final order as prepared originally read, the “Motion of the Cross-Complainant to amend the Cross-complaint is hereby Denied.” “Denied” was crossed out and “granted” was substituted. (C.T. 1183). The order then continues by granting a motion to dismiss the cross-complaint with prejudice for failure to state a claim. (C.T. 1183). However, the record does not indicate that such a motion was filed. The order concludes by reiterating that “if mov-ant, Thomas W. Moore, wishes, he is free to institute a new action by filing a complaint in a court of his choice.” (C.T. 1183).
The problem is fairly obvious. If the district judge meant to preserve the claim set forth in the cross-complaint to permit it to be pursued by Moore elsewhere, why did he dismiss the cross-complaint with prejudice? Conversely, if he intended to dismiss the cross-complaint with prejudice, why did he conclude the order by observing that Moore was “free to institute a new action by filing a complaint in a court of his choice”? This problem is not made easier by the inexplicability of granting a motion to amend the cross-complaint and simultaneously dismissing the cross-complaint with prejudice.
We shall discuss the alleged errors, first, with respect to the privacy action, next the attorney’s fees, and, finally, the October 26, 1976 order.
II.
The Privacy Action.
Moore, in his attack on the jury verdict in the privacy action, raises several issues. Although we have considered each of them, only a few merit discussion. While Moore does not expressly question • the sufficiency of the evidence in support of the jury’s findings, his objections considered as a whole indicate that is his intention. The scope of our authority to review jury findings is narrow and where the findings are supported by substantial evidence they may not be disturbed on appeal. Cf. Schepp v. Langmade, 416 F.2d 276, 278 (9th Cir. 1969). The verdict is so supported in this case.
*965The privacy action involved three theories of law. The first alleged violation of the California common law right to privacy. Implicit in the verdict is the finding that Anderson’s recordation of conversations with Moore was not an unauthorized or unwarranted intrusion into Moore’s personal affairs either (1) because there was no serious and unreasonable invasion of privacy or (2) because the defendants were permitted to record and disclose the conversations.
The second theory was California Penal Code sections 630-637.2. Section 632 prohibits recording, intentionally and without the consent of all parties, a confidential communication. The definition of “confidential communication” excludes communications made in “circumstances in which the parties to the communication may reasonably expect that the communication may be overheard or recorded.” Cal.Pen.C. § 632(c). Section 633.5 provides that nothing in § 632 shall be construed as prohibiting one party to a confidential communication from recording such communication for the purpose of obtaining evidence reasonably believed to relate to the commission of the crime of extortion. The verdict for Telfon and Anderson means that the jury alternatively found either that Moore impliedly consented, the communication was not confidential, or Anderson recorded the conversation for the purpose of obtaining evidence reasonably believed to relate to the crime of extortion.
The final theory of the privacy action is an alleged violation of 18 U.S.C. § 2511 which prohibits the willful interception of any wire or oral communication. However, for a person who is a party to the communication and not acting under color of law an interception is unlawful only if done for the purpose of committing a criminal, tortious or injurious act. 18 U.S.C. § 2511(2)(a)(d). The jury must have found that Anderson did not record the conversation for any such purpose.
After reviewing the record, we conclude that substantial evidence, i. e., “such relevant evidence as a reasonable mind might accept as adequate to support,” Swanson v. Levy, 509 F.2d 859, 861 (9th Cir. 1975), exists to support any of the above alternative findings of the jury.
Moore next argues that the trial judge erred in instructing the jury as to 18 U.S.C. § 2511(2)(a)(d), the exception to the blanket prohibition against willful interception applicable to a party to the communication not acting under color of law. As already mentioned, a person is not entitled to this exemption if his purpose was to commit a criminal, tortious or injurious act. Moore objects because the jury was instructed that in determining whether Anderson recorded with the purpose of committing an injurious act it could consider, inter alia, whether the conversations were recorded for the purpose of improperly terminating Moore’s franchise. Moore contends that if Anderson’s purpose was to terminate Moore’s franchise properly it also would amount to an injurious act.
Congress did not define the meaning of “injurious act.”3 While we acknowledge *966that the term embraces acts not easily classified as either “criminal” or “tortious,” we cannot believe that Congress intended it to be read to embrace every act which disadvantages the other party to this communication. Such a reading would nullify the exemption created by § 2511(2)(a)(d). Presumably there is some disadvantage in having any conversation intercepted in the absence of consent of all parties. Congress, we believe, intended to permit one party to record conversation with another when the recorder is acting “out of a legitimate desire to protect himself.”4
The Eighth Circuit, in Meredith v. Gavin, 446 F.2d 794 (8th Cir. 1971), after reasoning that “a perfectly legitimate act may often be injurious,” held that the conduct Congress intended to prohibit was the “interception by a party to the conversation with an intent to use that interception against the non-consenting party in some harmful way and in a manner in which the offending party had no right to proceed.” Id. at 799. It further decided the scope of harmful conduct within the meaning of “injurious act” was to be determined on an ad hoc basis. United States v. Phillips, 540 F.2d 319, 325 (8th Cir.), cert. denied, 429 U.S. 1000, 97 S.Ct. 530, 50 L.Ed.2d 611 (1976); Meredith, supra at 799. We align ourselves with these views. It follows that Congress did not intend to prohibit recording a conversation when its purpose was to preserve evidence of extortion directed against the recorder to be used later for the purpose of terminating a franchise agreement. The district court’s instructions to that effect were not erroneous.
Moore objects for the first time on appeal to several other jury instructions. These concern the allocation of the burdens of proof of elements of the California and Federal statutes, and section 633.5 of the California Penal Code. Moore is not now entitled to assign error to the giving of these instructions. Fed.R.Civ.P. 51. Nor may the “plain error” rule be invoked here to obtain review of instructions, given or refused, where the objection was not raised in trial court. See, Hargrave v. Wellman, 276 F.2d 948, 950 (9th Cir. 1960); Bertrand v. Southern Pacific Co., 282 F.2d 569, 572 (9th Cir. 1960), cert. denied, 365 U.S. 816, 81 S.Ct. 697, 5 L.Ed.2d 694 (1961); Monsma v. Central Mutual Ins. Co., 392 F.2d 49, 53 (9th Cir. 1968); Hays v. United Fireworks Mfg. Co., 420 F.2d 836, 841 (9th Cir. 1964). Moreover, after reviewing the challenged instructions we are convinced they were proper.
Finally, Moore argues that defense counsel’s terse reference in closing argument to Telfon’s bankrupt state constituted prejudicial error. Again no objection was made at trial; this statement, therefore, cannot now serve as grounds for reversal. United States v. Socony-Vacuum Oil Co., 310 U.S. 150, 239, 60 S.Ct. 811, 84 L.Ed. 1129 (1939); Herman v. Hess Oil Virgin Islands Corp., 524 F.2d 767, 770 (3d Cir. 1975). In any event, the reference did not constitute error. An isolated and casual remark does not constitute grounds for reversal, if not representative of the quality of the argument as a whole. Juaire v. Nardin, 395 F.2d 373, 378 (2d Cir.), cert. denied, 393 U.S. 938, 89 S.Ct. 302, 21 L.Ed.2d 274 (1968); see also United States v. Socony-Vacuum Oil Co., supra, 310 U.S. at 239-40, 60 S.Ct. 811. The trial judge, moreover, admonished the jury not to consider the wealth of the parties or Telfon’s bankruptcy (R.T. 35-38). Counsel’s remark in the light of this admonition did not so prejudice the jury as to render the refusal to grant a new trial “inconsistent with substantial justice.” Fed.R.Civ.P. 61.
We find Moore’s other assignments of error to be lacking in substance.
*967III.
Attorney’s Fees.
The district court declined to fix attorney’s fees and disbursements upon the withdrawal of Broad, Khourie and Schulz on the ground that it did not have “jurisdiction to entertain an entirely new lawsuit for fees and a counterclaim for malpractice.” After reviewing the record we find that this statement represents an exercise by the court of its discretion not to embark on what could only have been a new lawsuit. This refusal was not an abuse of discretion.
While there is an absence of Ninth Circuit authority on point, several other circuits have held that a district court has power to permit the substitution of attorneys and determine fees, disbursements and liens, as ancillary to the conduct of the principal litigation. Iowa v. Union Asphalt and Roadoils, Inc., 409 F.2d 1239, 1244 (8th Cir. 1969), affirming Iowa v. Union Asphalt and Roadoils, Inc., 281 F.Supp. 391 (S.D. Iowa 1968); National Equipment Rental Ltd. v. Mercury Typesetting Co., 323 F.2d 784, 786 (2d Cir. 1963); Demeulenaere v. Rockwell Mfg. Co., 275 F.2d 572, 574 (2d Cir. 1960); Doggett v. Deauville Corp., 148 F.2d 881, 883 (5th Cir. 1945). Whether to do so, however, is a matter within the trial judge’s discretion.
As a condition for substitution of attorneys, for example, the court may require the payment of fair compensation. Doggett, supra at 883; Mercury Typesetting, supra at 786; City of Hankinson, North Dakota v. Otter Tail Power Co., 294 F.Supp. 249, 252 (D.C.N.D.1969). In Doggett, supra, the Fifth Circuit, however, found the district court’s discretion “well-exercised” when it permitted the substitution of attorneys but refused to entertain a contractual dispute over compensation which it believed would unduly delay resolution of the principal dispute. The Second Circuit also has held that it was not- an abuse of discretion to remit the attorney to an independent action to collect his fee where the principal action, an antitrust suit, was already seven years old. Demeule-naere, supra at 5.72.
Likewise, here, the antitrust suit had been pending for six years when Broad, Khourie, and Schulz moved for leave to withdraw. The determination of fees would have been a time-consuming activity which would have subjected the opposing parties and the court to delay and vexation. Therefore the district court did not abus.e its discretion in refusing to hear the collateral dispute between the attorneys and Moore.
IV.
The October 26, 1976 Order.
The October 26 order, as previously pointed out, was issued to reflect the September 22 oral disposition of (1) Moore’s motion, designated interchangeably as either a cross-complaint or a petition to amend the counterclaim, and (2) his motion for change of venue and (3) cross-defendant’s motion to dismiss the counterclaim for failure to prosecute. The district court dismissed the counterclaim with prejudice pursuant to Fed.R.Civ.P. 41(b). Such a dismissal will not be overturned save for clear abuse of discretion. Anderson v. Air West, Inc., 542 F.2d 522, 524 (9th Cir. 1976). No abuse of discretion occurred.
The plaintiff is charged with the exercise of “reasonable diligence” in prosecuting the action. Anderson, supra at 524; States Steamship Co. v. Philippine Air Lines, 426 F.2d 803, 804 (9th Cir. 1970). During the six years from the filing of the counterclaim to its dismissal, Moore did little toward prosecuting the action and may even have impeded its progress by resisting defendant’s attempts at discovery. Failure to prosecute diligently alone justifies dismissal, even where actual prejudice to the defendant is not shown. Alexander v. Pacific Maritime Ass’n, 434 F.2d 281, 283 (9th Cir. 1970), cert. denied, 401 U.S. 1009, 91 S.Ct. 1254, 28 L.Ed.2d 545 (1971); Pearson v. Dennison, 353 F.2d 24, 28 (9th Cir. 1965). Although no showing of actual prejudice was made, prejudice is presumed from un*968reasonable delay. Anderson, supra at 524; States Steamship, supra at 804. Thus, the district court was justified in finding that the defendants were prejudiced by the lengthy delay “especially in a case of this kind, where a person’s recollections about conversations and conduct are very important . . . .” Moore did not attempt to rebpt the presumption of prejudice.
The denial of Moore’s motion for change of venue pursuant to 28 U.S.C. § 1404(a) also did not amount to an abuse of discretion. Generally, the district court must weigh the “convenience of the parties and witnesses, in the interest of justice” 28 U.S.C. § 1404(a) in determining whether the action should be transferred to another forum. In support of his motion, Moore cited his choice of forum as the plaintiff, and the fact that he and two witnesses reside in New York. These are factors to be considered by the district court. Gulf Oil Corp. v. Gilbert, 330 U.S. 501, 508, 67 S.Ct. 839, 91 L.Ed. 1055 (1947). The court justifiably found them outweighed by other, more compelling, considerations.
One of these was the duration of the pendency of the litigation prior to the motion to transfer. See, Kasey v. Molybdenum Corp. of America, 408 F.2d 16 (9th Cir. 1969). Moore did not seek a change of venue until six years after the initiation of the suit. Venue in the Northern District of California did not become inconvenient until Moore discharged his San Francisco-based attorneys. The district court also found the gravamen of the action to be in San Francisco; the franchise agreement was executed, and Telfon’s principal place of business was in San Francisco. The counter-defendants, as well as other witnesses, reside in San Francisco. Also, counsel and important documents are located in San Francisco. In light of these circumstances, the district court did not abuse its discretion by finding that the “demonstration which was made to support [the] motion was not nearly adequate to justify a transfer.”
Finally, we turn to the troublesome issue of the disposition of the cross-complaint by the district court. There is, as already indicated, considerable confusion surrounding its disposition which stems from the discrepancies between the treatment of the cross-complaint at the September 22 hearing and its disposition in the final order. At the hearing the district judge treated the cross-complaint as a motion to amend the counterclaim which for good reasons he then denied. He assured Moore that the claims therein could be pursued elsewhere. Nowhere in the record, other than in the October 26 order, does it appear clearly that the district judge intended to dispose of the cross-complaint in the manner that order indicates. We are uncertain what the district judge intended and under the circumstances of this case are hesitant to accept the order in this particular as written. Nor is there anything to be accomplished by attempting to guess what the district judge intended.
Accordingly, we remand for reconsideration of the cross-complaint. The district court is free to dismiss the cross-complaint without prejudice. Should the dismissal be with prejudice the district court shall indicate its reasons therefor.
Affirmed in part, Remanded in part.
. The focus on the issue of the statute of limitations most clearly evidences this treatment. Moore had alleged claims in the cross-complaint potentially barred by the statute. The district judge stated, “[i]f we allow this amendment under the rule it would relate back and it would overcome the bar of the statute.” (R.T. 84-85). He later comments “[t]he first question that arises is whether the amendment arises out of the same transaction or series of transactions which are the subject of the pending litigation.” (R.T. 87). After the cross-defendants argued that the amendment did not arise from the same transaction as the counterclaim the district judge asked if there are “any other reasons to deny leave to file an amendment?” (R.T. 88-89). In response, cross-defendants Bear, Stearns and Feldman, Waldman & Kline injected their contention that the cross-complaint fails to state a claim. Neither party so moved, however.
. Moore questioned whether the ruling prevented him from filing a new lawsuit. The district judge replied “No. You are not precluded from filing any new lawsuits. All I am saying is that you will not be permitted to amend this case, your counterclaim to add these claims, but you are free to file any new lawsuits that you wish.” (R.T. 110:6-10). This statement was repeated at the hearing, (R.T. 110:21-24), and in the October 12, 1976 order denying Moore’s petition for reconsideration of the September 22 rulings. (C.T. 1172).
. The statute originally exempted from its prohibition any interception of a wire or oral communication where one of the parties to the communication consented. 2 U.S.Code Cong. & Ad.News, 90th Cong., 2d Sess., p. 2182 (1968). Senator Hart objected to this exemption because it was too permissive, possibly allowing one party to monitor the communication for “insidious purposes such as blackmail, stealing business secrets, or other criminal or tortious acts in violation of Federal or State laws.” Id at 2236. Senators Hart and McClellan proposed an amendment to narrow the exemption, part of which was enacted as § 2511(2)(a)(d). Senator Hart cast the only light upon the meaning of “injurious act” stating, “We propose to prohibit a one party consent tap, except . for private persons who act in defensive fashion . . . [Whenever a private person acts in such situations with an unlawful motive he will violate the criminal provisions of title III and will also be subject to a civil suit. Such one party consent is also prohibited when the party acts in any way with an intent to injure the other party to the conversation in any other way. For example the secret consensual recording may be made for the purpose of blackmailing the other party, threatening him, or publicly embarrassing him . . 114 Cong.Rec. 14694 (May 23, 1968).
. Senator Hart, further describing the purpose of the amendment, noted that it did not prohibit “such recording in other situations when the party acts out of a legitimate desire to protect himself and his own conversations from later distortion' or other unlawful or injurious uses by the other party.” 114 Cong.Rec. 14694 (May 23, 1968).