[45]
Peter H. Schonemann
On the formal differentiation of traces and determinants
Multivariate Behavioral Research, 1985, 20, 113-139.
Abstract
A compact notation for obtaining and handling matrices of partial derivatives is suggested in an attempt to generalize "symbolic vector differentiation" to matrices of independent variables. The proposed technique differs from methods advocated by Dwyer and McPhail (1948) and Wrobleski (1963) in several respects, notably in a deliberate limitation on the classes of scalar functions concidered: traces and determinants.
To narrow intrest to these two classes of scalar matrix functions allows one to invoke certain algebraic identities which simplifies the problem, because
(a) the treatment of traces of products of matrices can be reduced to that of a few representatives of large equivalence classes of such products, all having the same formal derivatives, and because
(b) the more involved task of differentiating determinants of matrix products can be translated into the more amenable problem of differentiating the traces of such products.
A number of illustrative examples are included in an attempt to show that the above limitation is not as serious as might at first appear, because traces and determinants apply to a wide range of psychometric and statistical problems.
Notes
This paper extends the rudimentary beginnings in the Appendix of my Thesis to include matrix differentiation of determinants and a fairly large number of illustrative examples drawn from the psychometric literature. The extension to determinants widens the scope of the technique considerably, because it now can also be applied to maximum likelihood problems. The strength of my approach was not generality but simplicity. In this respect it differed from other approaches, notably, Dwyer and McPhail, Symbolic matrix derivatives, Annals of Mathematical Statistics, 1948, XIX, 517-534, that were in circulation at that time. Since then, numerous other treatments of matrix differentiation have appeared. See, e.g., Nel (1980) On matrix differentiation in statistics. South African Statistical Journal, 14, 137-193.
Prepublication History
The published version is virtually identical to a Research Memorandum (No. 27) I drew up while holding a postdoctoral fellowship at the Psychometric Laboratory of the University of North Carolina in 1964/1965. I submitted it to Psychometrika on July 29, 1965. Surprisingly, it took 20 years before the paper finally saw the light of day in a (different) peer reviewed journal.
The Psychometrika editor sent me a mildly critical review by a referee whose primary concern seemed to be to defend his own turf and that of one of his students:
"It is also of interest that here is a development which is independent of the results of X and Y. It is also of very special interest ... to show how traces can be used, in connection with certain problems such as those featuring the use of Lagrange multipliers, to obtain an alternative statement of the problem to be optimized.
But it should not be implied ... that the reduction of the problem to traces is necessary or even always to be preferred."
The editor encouraged a revision, which I sent him on January 19, 1966. Then there was no further communication from him for several months until eventually the editorship changed hands. The new editor sat on the manuscript for a few more months. After reminding him that I was still awaiting a decision, he finally, on November 4, 1966, sent an apologetic letter, blaming the delay on the change of editorship. He asked for yet another revision, because a "well-informed reviewer" felt "there is nothing wrong with the paper but ... it is not too important". By this time I had become too digusted to continue the dialogue.
Several years later, the first editor hinted the root cause of the initial delay may have been Rolf Bargmann, a widely respected statistician with expertise in this area. I subsequently learned from him that he had indeed been a reviewer. However,
"As a referee in 1965 I strongly supported publication. In fact, you notice in my collection of formulas in the Chemical Rubber Handbook of Mathematics in 1967, I suggested your paper for further reading, assuming that it would be published."
This serves to illustrate how haphazard and subjective the peer review system can be even under the best of circumstances - up to that time I had never had any problems with either one of the two of the editors. Needless to say, when circumstances are no longer best, the problem worsens. See Schonemann and Steiger, 1978, and Better never than late (unpublished, Publications [86]), for further examples.