Matrices

The Matrix Cookbook
[ http://matrixcookbook.com ]
Kaare Brandt Petersen Michael Syskind Pedersen Version: November 14, 2008
What is this? These pages are a collection of facts (identities, approximations, inequalities, relations, ...) about matrices and matters relating to them. It is collected in this form for the convenience of anyone who wants a quick desktop reference . Disclaimer: Theidentities, approximations and relations presented here were obviously not invented but collected, borrowed and copied from a large amount of sources. These sources include similar but shorter notes found on the internet and appendices in books - see the references for a full list. Errors: Very likely there are errors, typos, and mistakes for which we apologize and would be grateful to receivecorrections at cookbook@2302.dk. Its ongoing: The project of keeping a large repository of relations involving matrices is naturally ongoing and the version will be apparent from the date in the header. Suggestions: Your suggestion for additional content or elaboration of some topics is most welcome at cookbook@2302.dk. Keywords: Matrix algebra, matrix relations, matrix identities, derivative ofdeterminant, derivative of inverse matrix, differentiate a matrix. Acknowledgements: We would like to thank the following for contributions and suggestions: Bill Baxter, Brian Templeton, Christian Rishøj, Christian Schr¨ppel Douglas L. Theobald, Esben Hoegh-Rasmussen, Glynne Casteel, Jan o Larsen, Jun Bin Gao, J¨rgen Struckmeier, Kamil Dedecius, Korbinian Strimu mer, Lars Christiansen, Lars Kai Hansen,Leland Wilkinson, Liguo He, Loic Thibaut, Miguel Bar˜o, Ole Winther, Pavel Sakov, Stephan Hattinger, Vasile a Sima, Vincent Rabaud, Zhaoshui He. We would also like thank The Oticon Foundation for funding our PhD studies. 1

CONTENTS

CONTENTS

Contents
1 Basics 1.1 Trace and Determinants . . . . . . . . . . . . . . . . . . . . . . . 1.2 The Special Case 2x2 . . . . . . . . . . . . . . . . .. . . . . . . . 2 Derivatives 2.1 Derivatives 2.2 Derivatives 2.3 Derivatives 2.4 Derivatives 2.5 Derivatives 2.6 Derivatives 2.7 Derivatives 2.8 Derivatives 5 5 5 7 7 8 9 9 11 13 13 14 16 16 17 19 19 20 20

of of of of of of of of

a Determinant . . . . . . . . . . . . an Inverse . . . . . . . . . . . . . . . Eigenvalues . . . . . . . . . . . . . . Matrices, Vectors and Scalar Forms Traces .. . . . . . . . . . . . . . . . vector norms . . . . . . . . . . . . . matrix norms . . . . . . . . . . . . . Structured Matrices . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

3 Inverses 3.1 Basic . . . . . . . . . . . 3.2 Exact Relations . . . . . 3.3 Implication onInverses . 3.4 Approximations . . . . . 3.5 Generalized Inverse . . . 3.6 Pseudo Inverse . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. .. . . .

. . . . . .

. . . . . .

4 Complex Matrices 23 4.1 Complex Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2 Higher order and non-linear derivatives . . . . . . . . . . . . . . . 26 4.3 Inverse of complex sum . . . . . . . . . . . . . . . . . . . . . . . 26 5 Solutions and Decompositions 5.1 Solutions to linear equations . 5.2 Eigenvalues and Eigenvectors 5.3Singular Value Decomposition 5.4 Triangular Decomposition . . 5.5 LU decomposition . . . . . . 5.6 LDM decomposition . . . . . 5.7 LDL decompositions . . . . . 27 27 29 30 32 32 32 32

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . ....