Opinion of the Court
COX, Chief Judge:We granted review of this issue:
WHETHER THE MILITARY JUDGE ABUSED HIS DISCRETION BY DENYING PRODUCTION OF PROFESSOR DENBEAUX, A DEFENSE EXPERT CRITIC OF HANDWRITING ANALYSIS.
We hold that the military judge did not abuse his discretion in so ruling because he expressly kept the issue open for defense counsel to pursue after evidence had been heard; and defense counsel’s failure to raise the question for reconsideration waived any appellate complaint as to the initial ruling.1
Facts
The theme of the Government’s case was that appellant and an active-duty friend, Private Joseph M. Durocher, developed a scheme to “get rich quick.” Appellant planned and worked with Private Durocher (who was convicted of similar offenses prior to appellant’s court-martial and became the key witness against appellant) to set up allotments and transfer money from the bank accounts of unsuspecting military members into a bank account opened under a fictitious name in Liechtenstein. This account was later to be accessed by Private Durocher after he left active duty. The fraud was discovered when the victims were contacted by their respective banks about the requested transfer of funds, and each denied making such a request.
Prior to trial, the Government notified the defense that they intended to call Special Agent Richard A. Horton, Army Criminal Investigation Command (CID), as an expert in the field of handwriting analysis. Mr. Horton would link appellant to the scheme through his “expert opinion” that there were “strong indications” that appellant signed at least one of the forged letters entered into evidence on the forgery and larceny charges. Mr. Horton’s opinion reflected the strongest positive finding on the scale used by questioned document examiners.
In response, the defense made a motion requesting an expert witness to challenge the lack of scientific rationale used to support the conclusions of handwriting analysts in general and to challenge the reliability of this evidence in particular. In that motion, the defense stated that they expected that “an essential element of the government’s ease will be what purports to be a scientific determination by a CID handwriting expert that SPC [Specialist] Ruth ‘almost certainly’ wrote one of the signatures appearing on the bank application forms used to open a bank account at the Liechtensteinische Landes-bank under the fictitious name of ‘William Cooper.’ ” They also alleged that
the CID recovered a forged letter sent to the bank of PVT Durocher, bearing a purported signature of Durocher. The CID handwriting expert has opined, and will testify, that SPC Ruth can neither be identified nor eliminated as the writer of the signature, but that there are indications PVT Durocher did not write the signature.
*3The defense team specifically requested Professor Mark P. Denbeaux,2 a professor of law at Seton Hall Law School, as their expert witness to contradict the Government’s expert.3 Professor Denbeaux had co-authored a law review article4 based upon evaluation of research and experiments on the reliability of handwriting analysis. The authors of that article seriously questioned the accuracy and reliability of handwriting analysis and suggested that
a direct attack by motion in limine on such testimony ought to be successful, under either the so-called Frye[5] test or any other defensible standard. The power of the argument is obvious. No cases have ever examined, much less determined, whether these “experts” can do what they claim. Further, the tests that have been done do not support their claim. If handwriting identification testimony were to be proffered and treated as a case of first impression now, the proponent would clearly have the burden of proving the existence of the claimed skill, a burden that has yet to be met in any forum — legal, scholarly, or scientific.
137 U.Pa.L.Rev. 731, 771-72 (1989) (footnotes omitted); see United States v. Gipson, 24 MJ 246 (CMA 1987) (judge must make preliminary assessment whether reasoning or methodology underlying proffered expert testimony is scientifically valid); United States v. Houser, 36 MJ 392, 399 (CMA 1993) (It is “appropriate” to determine whether proposed “evidence embraces a new technique or theory and the potential rate of error, as well as the existence of any specialized literature and cases on the subject.”); see generally Imwinkelried, Coming to Grips with Scientific Research in Daubert’s “Brave New World”: The Courts’ Need to Appreciate the Evidentiary Differences between Validity and Proficiency Studies, 61 Brook.L.Rev. 1247 (1995).
The defense team made a motion to compel production of Professor Denbeaux as their witness for the trial on the merits, which was litigated and preliminarily denied prior to trial. RCM 703(d), Manual for Courts-Martial, United States (1995 ed.). However, the military judge specifically stated that he would be open to reconsideration of the request during trial, if circumstances supported so doing after the Government’s expert had testified. See Appendix. The defense never renewed their request.
Discussion
We review the military judge’s initial denial of the defense’s request for production of a witness for abuse of discretion. Houser, 36 MJ at 397; United States v. Tangpuz, 5 MJ 426, 429 (CMA 1978). The military judge’s decision should only be reversed if, “on the whole,” denial of the defense witness was improper. Tangpuz, supra at 429, citing United States v. Manos, 17 USCMA 10, 16, 37 CMR 274, 280 (1967). The reviewing court should not set aside a judicial action “unless it has a definite and firm conviction that the court below committed a clear error of judgment in the conclusion it reached upon a weighing of the relevant factors.” Houser, supra at 397, quoting Judge Magruder in The New York Law Journal at 4, col. 2 (March 1, 1962), quoted *4in Quote It II: A Dictionary of Memorable Legal Quotations 2 (1988). •
“The ... defense counsel ... shall have equal opportunity to obtain witnesses and other evidence in accordance with such regulations as the President may prescribe.” Art. 46, UCMJ, 10 USC § 846. “Each party is entitled to the production of any witness whose testimony on a matter in issue on the merits ... would be relevant and necessary.” RCM 703(b)(1). The equal right of the Government and the defense to obtain witnesses specifically includes the “equal opportunity to obtain expert witnesses.” Mil.R.Evid. 706(a), Manual, supra; Art. 46. While “the right of an individual accused to the attendance of witnesses on the merits of the case [is] not absolute[,] ... discretion [is] vested in the trial judge.” Tangpuz, supra at 430. However, this Court has reviewed the judge’s exercise of discretion “to assure to the greatest degree possible ... equal treatment for every litigant before the bar.” Tangpuz, supra at 430, quoting Coppedge v. United States, 369 U.S. 438, 446, 82 S.Ct. 917, 922, 8 L.Ed.2d 21 (1962).
This Court has never announced a bright-line rule as to when to require production of a defense-requested witness. The facts of each case are unique. However, over time we have recognized specific factors to be considered in deciding whether personal production of a witness is necessary. See Houser, 36 MJ at 399; United States v. Garries, 22 MJ 288, 291 (CMA 1986); Tangpuz, supra at 429. These include, but are not limited to: “the issues involved in the case and the importance of the requested witness as to those issues; whether the witness is desired on the merits or the sentencing portion of the trial; whether the witness’ testimony would be merely cumulative; and the availability of alternatives to the personal appearance of the witness, such as deposition, interrogatories or previous testimony.” Tangpuz, supra at 429. This list is not exhaustive, but provides a solid foundation for a judge’s ruling on which he or she may expand. However, failure of the judge to fully consider these factors in making the ruling will be considered in determining whether the judge abused his or her discretion. See Tangpuz and Houser, both supra. Our opinion in Houser requires that the following criteria be established for requiring production of an expert witness: “(A) the qualifications of the expert, Mil.R.Evid. 702; (B) the subject matter of the expert testimony, Mil.R.Evid. 702; (C) the basis for the expert testimony, Mil.R.Evid. 703; (D) the legal relevance of the evidence, Mil.R.Evid. 401 and 402; (E) the reliability of the evidence, United States v. Gipson, 24 MJ 246 (CMA 1987), and Mil.R.Evid. 401; and (F) whether the ‘probative value’ of the testimony outweighs other considerations, Mil.R.Evid. 403.” 36 MJ at 397.
Here, although the judge did not articulate his ruling in terms of these factors, he ultimately addressed many of these factors in a roundabout manner through a question and answer session with defense counsel. See Appendix. Further, the defense never fully litigated its motion to produce a defense expert witness on the basis that handwriting analysis is not generally reliable, to attempt to show that the government expert’s testimony was inadmissible. United States v. Gipson, supra; see United States v. Bur-nette, 29 MJ 473 (CMA 1990).6 Therefore, the only other purpose for producing Professor Denbeaux was to offer testimony that would go to the appropriate weight that the members should give to the government expert’s testimony.
The record is clear that the military judge deferred the finality of his ruling until the point in the trial where the materiality and necessity of the witness for this purpose would become crystal clear. It is not always possible to determine the necessity of a defense expert witness until after the government expert has testified. Yet, after that expert testified, the defense chose not to renew the motion to produce Professor Denbeaux.
*5In fact, even though the military judge had explained alternatives to the production of Professor Denbeaux, such as use of his article as a learned treatise (Mil.R.Evid.803(18)) and use of the article for cross-examination (Mil.R.Evid.803(18) and Mil.R.Evid. 705), defense counsel did not avail himself of these measures either.
Regardless, defense counsel’s request was expressly open to reconsideration. Therefore, under these circumstances, we find no abuse of discretion in the military judge’s preliminary ruling. See 42 MJ 730, 733-34; United States v. Garries, supra; cf. United States v. Velasquez, 64 F.3d 844 (3d Cir. 1995).
Appellant aptly cites Velasquez in support of his argument. In Velasquez, the Court of Appeals for the Third Circuit held that failure to allow Professor Denbeaux to testify as an expert witness in that case was reversible error. Judge Roth specifically stated:
We find that sufficient evidence exists to show that the Professor had “good grounds” for his rejection of handwriting analysis.
[T]he Professor’s proffered testimony was highly relevant to the reliability of [the government expert’s] testimony. His criticisms of the field of handwriting analysis generally, as well as [the government expert’s] analysis in this case, would have assisted the jury in determining the proper weight to accord [the government expert’s] testimony. His testimony “fits” the facts of the case because his opinion, criticizing handwriting analysis and [the government expert’s] conclusions, connect to the issue of whether Defendant’s continuing criminal enterprise involved at least five other people.
Thus, in light of the liberal standard of admissibility of Rule 702, Professor Den-beaux’s testimony should have been admitted. Moreover, because his testimony bore on the critical issue of [the government expert’s] identification of the persons who were required to have participated in Velasquez’s “continuing criminal enterprise,” his testimony might very well have affected the jury’s verdict ... We cannot conclude that the district court’s decision to exclude that evidence was harmless error.
64 F.3d at 851-52.
In contrast, the military judge made an initial ruling that a sufficient showing was not made to produce Professor Denbeaux prior to trial, but he added that he would reconsider his ruling at a later time during the trial, if the defense would care to renew its motion. Unlike in Velasquez, here the military judge did not exclude the testimony of a witness who was already standing outside the courtroom door. In other words, under the approach taken by the military judge, had the defense really wanted Professor Denbeaux after the government expert testified, it simply meant that a brief break in the court-martial may have been necessitated.
The decision of the United States Army Court of Criminal Appeals is affirmed.
Judges SULLIVAN, CRAWFORD, GIERKE, and EFFRON concur.
Appendix
The judge’s colloquy with defense counsel before ruling on the motion for production of the defense requested expert witness was as follows:
MJ: Now, let’s — let’s talk about the expert. I’m a little confused as to what you expect this expert to testify on.
DC: The offer of proof on this expert, Professor Denbeaux, Your Honor, is: That he would testify that handwriting analysis, in order to identify the writer of a questioned document as it has been practiced in the United States over the past several decades, is not — should not be accorded the — the halo, if you will, of scientific validity for two main reasons. One being, the studies that have been done as to the reliability of handwriting identification have shown, that it is not reliable. The other reason being that given the way in which handwriting identification experts testify, they testify in such a way that they insulate themselves in effect from cross-examination *6or from effective cross-examination by virtue of the fact that unlike the theory of Newtonian mechanics that could be tested and found wanting, their theories are not subject to that kind of objective verification due to the lack of data as to how vast numbers of people write various letters.
MJ: Has he written on this subject?
DC: He has, Your Honor.
MJ: Have you had access to his writings?
DC: Yes, Your Honor.
MJ: Then why can’t you use his writings to just cross-examine the government expert on those basic points? Why do we have to have him here to accomplish that? It sounds like something very understandable. It’s been the court’s experience, with at least the crime lab in Frankfurt, that they’re not so pontificating [sic] to indicate that they’re [sic] identifications are so iron clad. It sounds like from the basic offer in the defense motion that they’re not going to say anything so strong.
DC: What the handwriting identification expert will say, that’s Mr. Horton, is that there are, quote, strong indications that Specialist Ruth wrote a signature on the documents used to open the bank account in Liechtenstein. And strong indications is right up next to the highest level of — in that certainty — the defense believes Mr. Horton will say that it is virtually certain that Specialist Ruth wrote that signature because that’s how he phrased it when speaking to the CID locally here in Bamberg on, I believe, two occasions.
MJ: Have you talked with him?
DC: Yes, Your Honor, I have.
MJ: And has he come out with that same response to you?
DC: He said “strong indications” as he did in his lab report. He didn’t then use the words “virtually certain,” but he’s used it more than once as indicated in the local CID agent’s activity summary.
MJ: But apparently your witness — this expert — has not had any contact with this particular case?
DC: No, Your Honor.
MJ: It’s just his notoriety for this opinion challenging 30 years of scientific evidence and—
DC: Of purportedly scientific evidence.
MJ: Purportedly scientific evidence.
What’s the Government’s response?
[The Government then argued against production of the witness, and the defense was allowed to respond.]
DC: [In response to the trial counsel, I have] [a] couple of things: The case on which [trial counsel] relies about the defense going before the convening authority and military judge and saying there’s no conceivable government expert that could help me in this matter is a case involving scientific analysis of a urinalysis sample. This is a lot—a very different situation, Your Honor, because handwriting analysis, from the defense’s point of view, is a pseudo-science that’s grown up over several decades and handwriting analysts, frankly, Your Honor, stick together. And it is extremely unlikely there will be found within the CID or even the FBI or the Secret Service, any expert who will testify that all of the studies on reliability of this, when taken together, show that it’s not reliable. Regarding whether the writings of Professor Denbeaux are an adequate substitute for—granted in the military we do have very intelligent discerning court members, however to ex-, pect them to read a law review article of, I believe about 50 pages, and to digest it in the course of a court-martial, is, I think, a bit much to expect of them. It would be far more effective and, for the defense a much fairer trial as to this handwriting issue to have Professor Denbeaux here to explain what -his research has shown. The defense did as—or attempted to get Mr. Horton to talk about the procedures he used running up to his conclusion, but were given answers that weren’t directly responsive to the questions.
*7Lastly, regarding Professor Denbeaux himself, I spoke to him yesterday afternoon, he emphasized that he does have an hourly fee of $250.00 an hour, but he has, in the past, testified for nothing; for instance, when a public defender asked him to do it, and he would do that in this case.
MJ: Well, again, based on the proffer, the court finds that the probative value of the testimony is not such as to warrant the production of this particular witness. The court’s confident at this stage that, through the use of the writings, through the use of cross-examination and other trial tactics, that these issues would be fully focused on by the defense so that the members would be able to understand the issues that the defense is concerned with. And the court’s confident that, as indicated, the military panels are much more sophisticated than civilian panels by and large, by reason of training, education, experience. The court’s confident that they’ll be able to grasp the concepts you’re talking about.
They’re not so complex as to defy common sense and knowledge of human nature and the ways of world [sic], that once it’s brought to the [sic] attention, that they can properly focus then on the testimony of the witness and give it no weight, or some weight, or whatever weight they deem appropriate once you bring these matters to their attention—the subjective nature of the analysis in question. It seems like in the past the agents have freely admitted the subjective aspect of them — of them examination. Now, if during the course of the trial it appears that there remains a material issue on this particular issue, then Captain Kulish, bring it to the court’s attention, and if necessary, I’ll reconsider the ruling at that time. And if the interests of justice requires [sic] the production of the expert, well, then I’ll so order it at that point, and we’ll delay the proceedings until the witness is available.
(Emphasis added.)
. Appellant was tried and convicted at Bamberg, Germany, by a general court-martial consisting of officer and enlisted members. Contrary to his pleas, he was convicted of attempted larceny (37 specifications), conspiracy, using government facilities for other than official purposes, larceny (9 specifications), forgery (32 specifications), and obtaining services under false pretenses, in violation of Articles 80, 81, 92, 121, 123, and 134, Uniform Code of Military Justice, 10 USC §§ 880, 881, 892, 921, 923 and 934, respectively. Appellant was sentenced to a bad-conduct discharge, confinement for 4 years, and reduction to paygrade E-l. The convening authority approved the sentence. The Court of Criminal Appeals affirmed the findings and sentence, although the opinion erroneously states that the sentence included total forfeitures. 42 MJ 730.
. At trial he is erroneously referred to with the middle initial "A”.
. Because the defense did not pursue the issue, we need not decide whether the defense was entitled to a specifically named expert in this case. See United States v. Burnette, 29 MJ 473 (CMA 1990). However, the defense aptly stated in their motion:
Given the vested interest of the CID as the Army's primary law enforcement investigative agency in the scientific claims of handwriting analysis, no expert who questions those claims can be found within the Army. For the same reason, no such expert is likely to be found within any branch of the U.S. armed services. We generally agree with this rationale to support having a non-government expert in this case. To decide otherwise would be to condone the proverbial “fox guarding the hen house.”
. Risinger, Denbeaux, & Saks, Exorcism of Ignorance as a Proxy for Rational Knowledge: The Lessons of Handwriting Identification “Expertise," 137 U.Pa.L.Rev. 731 (1989).
. See Frye v. United States, 293 F. 1013 (D.C.Cir. 1923). This law review article was written before the decision in Daubert v. Meirell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993).
. We do not here opine as to whether the recent decision of Daubert, supra at n. 5, is germane to “handwriting experts.” See Mil.R.Evid. 702, Manual for Courts-Martial, United States (1995 ed.).