154 views

Uploaded by alon

Useful formulas dealing with matrix derivations.
Appears in all the theory of image processing

- Ch 4 Determinant
- Cookbook Scipy - Building Array
- Block Diagonal Inversion
- B.a. Part-I(Semester I & II) Subject - Mathematics
- 10172_2_15_2_2013
- Dpp Maths & Computer-new
- chapter_1_2
- euler_en
- Ch10 LU Decomposition 21
- Complex-Numbers-Worksheet.doc
- Florence Bertails- Linear Time Super-Helices
- A Novel Algorithm for Locating Faults on Transposed-Untransposed Transmission Lines Without Utilizing Line Parameters
- Unit-8 Matrices - II.pdf
- Section 6
- MatVP2DEIT_Part3
- Peters en Matrix Cookbook
- LA7-3-09
- Programs of Sir
- wbjee3
- e math

You are on page 1of 4

Matrix Calculus

z

z

z

z

Notation

Derivatives of Linear, Quadratic and Cubic Products

Derivatives of Inverses, Trace and Determinant

Jacobians and Hessian matrices

Notation

z

z

z

z

z

z

z

z

d/dx (y) is a vector whose (i) element is dy/dx(i)

d/dx (yT) is a matrix whose (i,j) element is dy(j)/dx(i)

d/dx (Y) is a matrix whose (i,j) element is dy(i,j)/dx

d/dX (y) is a matrix whose (i,j) element is dy/dx(i,j)

xR and xI are the real and imaginary parts of x

x* is the complex conjugate of x

j is the square root of -1

An expression, y, can only differentiated with respect to a complex x if it satisfies the Cauchy-Riemann

equations: dy/dxR = j dy/dxI . Expressions involving the complex conjugate or Hermitian transpose do not

normally satisfy this requirement, so separate expressions for dy/dxR and dy/dxI are given in these cases.

In the expressions below matrices and vectors A, B, C do not depend on X.

z

z

z

z

{ d/dx (Ay) =A * d/dx (y)

d/dx (xTA) =A

T

{ d/dx (x ) =I

T

T

{ d/dx (x a) = d/dx (a x) = a

d/dX (aTXb) = abT

T

T T

T

{ d/dX (a Xa) = d/dX (a X a) = aa

d/dX (aTXTb) = baT

d/dx (YZ) =Y * d/dx (Z) + d/dx (Y) * Z

dy/dxR (YH) = ( dy/dxR (Y) )H

dy/dxR (xHA) = A

{

z

dy/dxR (xH) = I

{

z

z

z

T

T

{ d/dx (x Cx) = (C+C )x

T

T

[C=C ]: d/dx (x Cx) = 2Cx

T

d/dx (x x) = 2x

T

T

T

{ d/dx (Ax+b) (Dx+e) = A (Dx+e) + D (Ax+b)

T

T

d/dx (Ax+b) (Ax+b) = 2A (Ax+b)

T

T

T

{ [C=C ]: d/dx (Ax+b) C(Ax+b) = 2A C(Ax+b)

d/dX (aTXTXb) = X(abT + baT)

T T

T

{ d/dX (a X Xa) = 2Xaa

d/dX (aTXTCXb) = CTXabT + CXbaT

T T

T

T

{ d/dX (a X CXa) = (C + C )Xaa

T

T T

T

{ [C=C ] d/dX (a X CXa) = 2CXaa

d/dX ((Xa+b)TC(Xa+b)) = (C+CT)(Xa+b)aT

d/dxR (Ax+b)HC(Dx+e) = AHC(Dx+e) + DTCT(Ax+b)*

{

{

z

Derivatives of Inverses

z

z

d/dX (aTX-1b) = -X-TabTX-T [2.6]

Derivative of Trace

Note: matrix dimensions must result in an n*n argument for tr().

z

z

z

z

d/dX (tr(Xk)) =k(Xk-1)T

d/dX (tr(AXk)) = SUMr=0:k-1(XrAXk-r-1)T

d/dX (tr(AX-1B)) = -(X-1BAX-1)T = -(X-TABX-T) [2.5]

-1

-1

-T T -T

{ d/dX (tr(AX )) =d/dX (tr(X A)) = -X A X

d/dX (tr(ATXBT)) = d/dX (tr(BXTA)) = AB [2.4]

T

T

T

T

{ d/dX (tr(XA )) = d/dX (tr(A X)) =d/dX (tr(X A)) = d/dX (tr(AX )) = A

d/dX (tr(AXBXTC)) = ATCTXBT + CAXB

T

T

T

T

{ d/dX (tr(XAX )) = d/dX (tr(AX X)) = d/dX (tr(X XA)) = X(A+A )

T

T

T

T

{ d/dX (tr(X AX)) = d/dX (tr(AXX )) = d/dX (tr(XX A)) = (A+A )X

d/dX (tr(AXBX)) = ATXTBT + BTXTAT

z

z

z

[B,C:symmetric] d/dX (tr((XTCX)-1(XTBX)) = d/dX (tr( (XTBX)(XTCX)-1) = -2(CX(XTCX)-1)

XTBX(XTCX)-1 + 2BX(XTCX)-1

Derivative of Determinant

Note: matrix dimensions must result in an n*n argument for det(). Some of the expressions below involve

inverses: these forms apply only if the quantity being inverted is square and non-singular.

z

T

T

T

-T T

-T

{ d/dX (det(AXB)) = A ADJ(AXB)B = det(AXB)*A (AXB) B = det(AXB)*X

T

-T T

-T

{ d/dX (ln(det(AXB))) = A (AXB) B = X

d/dX (det(Xk)) = k*det(Xk)*X-T

k

-T

{ d/dX (ln(det(X ))) = kX

[Real] d/dX (det(XTCX)) = det(XTCX)*(C+CT)X(XTCX)-1

T

T

T

-1

{ [C: Real,Symmetric] d/dX (det(X CX)) = 2det(X CX)* CX(X CX)

[C: Real,Symmetricc] d/dX (ln(det(XTCX))) = 2CX(XTCX)-1

Jacobian

If y is a function of x, then dyT/dx is the Jacobian matrix of y with respect to x.

Its determinant, |dyT/dx|, is the Jacobian of y with respect to x and represents the ratio of the hypervolumes dy and dx. The Jacobian occurs when changing variables in an integration: Integral(f(y)dy)

=Integral(f(y(x)) |dyT/dx| dx).

Hessian matrix

If f is a function of x then the symmetric matrix d2f/dx2 = d/dxT(df/dx) is the Hessian matrix of f(x). A

value of x for which df/dx = 0 corresponds to a minimum, maximum or saddle point according to whether

the Hessian is positive definite, negative definite or indefinite.

z

z

d2/dx2 (aTx) = 0

d2/dx2 (Ax+b)TC(Dx+e) = ATCD + DTCTA

2

2 T

T

{ d /dx (x Cx) = C+C

d2/dx2 (xTx) = 2I

d2/dx2 (Ax+b)T (Dx+e) = ATD + DTA

2

2

T

T

d /dx (Ax+b) (Ax+b) = 2A A

[C: symmetric]: d2/dx2 (Ax+b)TC(Ax+b) = 2ATCA

The Matrix Reference Manual is written by Mike Brookes, Imperial College, London, UK. Please send any

comments or suggestions to mike.brookes@ic.ac.uk

- Ch 4 DeterminantUploaded byAnshuman Singh
- Cookbook Scipy - Building ArrayUploaded bymohanvel
- Block Diagonal InversionUploaded byKonstantinos Politis
- B.a. Part-I(Semester I & II) Subject - MathematicsUploaded byJasvinder Singh
- 10172_2_15_2_2013Uploaded byAnurag Prabhakar
- Dpp Maths & Computer-newUploaded byMuhammad Atif
- chapter_1_2Uploaded byTajul Aiman Tajul Anuar
- euler_enUploaded byjos3
- Ch10 LU Decomposition 21Uploaded byAmani
- Complex-Numbers-Worksheet.docUploaded byEsai Stephen
- Florence Bertails- Linear Time Super-HelicesUploaded byLokosoo
- A Novel Algorithm for Locating Faults on Transposed-Untransposed Transmission Lines Without Utilizing Line ParametersUploaded bywvargas926
- Unit-8 Matrices - II.pdfUploaded byChandradeep Reddy Teegala
- Section 6Uploaded byLisa Crawford
- MatVP2DEIT_Part3Uploaded byTushar Kanti Bera
- Peters en Matrix CookbookUploaded byHannes Nickisch
- LA7-3-09Uploaded byWilson An
- Programs of SirUploaded bylovablevishal143
- wbjee3Uploaded byDeepanshu
- e mathUploaded byDivan
- Assignment _IPSUploaded byANKUR GHEEWALA
- Network RepresentationUploaded byJay
- CA and PDE_Ex. 1_Amir HalutsUploaded byAmir Haluts
- Homework1 10 SolsUploaded byDave Tello de Flowers
- ex1Uploaded byx420
- MS 12Uploaded bySaurav Patel
- ComplexUploaded byHamid Assadullah
- http___erp.dpsbulandshahr.org_Uploads_doc_revision_paper_Class XI Maths_52556_dpsbsr312 (3).DOCXUploaded bygarg prapti
- Managing investing data with very simple quantitative tools.pdfUploaded byMihaela Theodorescu
- IAS Mains Mathematics 2007 Question PaperUploaded byWilliam Sena

- W14 Functions Exercise 1Uploaded bybidarihassan
- SVD ChapterUploaded byjdmena227
- 07 Basic - Matrices and Arrays Cheat SheetUploaded bywerwerwer
- integrationbyparts.pdfUploaded byJohn Lemar Tirao
- 3-Derivative1.pptUploaded byMuhammad Amirudin
- EC Signals and SystemsUploaded byJeeva Keshav
- Antiderivative of a FunctionUploaded byGie Mamuy
- Determinants and CoFactorsUploaded byCaseyJ.Li
- 9ABS301 Mathematics - IIUploaded bysivabharathamurthy
- Gallian Ch 17Uploaded bywoodbrent7
- MAT2384 ODE Solution 2Uploaded byliveinperson
- SeqUploaded bySidharth Arya
- The Clehsch-Gordan CoefficientsUploaded byCu mi
- GR Assignment 1 SolutionsUploaded byPaul Ryan
- Abstract AlgebraUploaded byBon Siranart
- Digital Signal Processing_ Lecuter-3_ Z TransformUploaded bysubcampus
- Finite Diff.pdfUploaded byElshan Aliyev
- diff'bleUploaded by2008000
- Comparing Linear and Nonlinear Differential Equations of DTM.pdfUploaded byHaziqah Hussin
- A Common Fixed Point Theorem Using in Fuzzy Metric Space Using Implicit RelationUploaded byEditor IJTSRD
- La Places EquationUploaded bymustafa
- Linear Algebra review.pptUploaded byLiliana Garcia
- ET and HomologyUploaded byYau Stein
- II Sem Mathematics All SolutionsUploaded byMrityunjoy Dutta
- Double-Angle, Half-Angle, And Sum-Product IdentitiesUploaded byY D Amon Ganzon
- Midterm Solutions 2013Uploaded byNonitus Octingenti
- Partial DerivativesUploaded byrafqi17
- Cyclic PersuitUploaded byyoursjannat
- factoring polynomialsUploaded byapi-207140275
- 250bookSec3.7Uploaded byodbay