Attribution Non-Commercial (BY-NC)

25 views

Attribution Non-Commercial (BY-NC)

- Ray Theory in Tomoraghy
- HW1
- SUNTIKAN PMRSDANSPM08
- Cholesky_LU_Factorization_1
- Galerkin Discontinuous
- Lec2-CommonAlgo
- Journal
- numeth
- IIT MMAE 501 Set 1 Solution(1)
- 2.2 Inverse of a Matrix Corrected
- Computation Methods Lab#1 Nov:20:12
- 10.1.1.134.203
- Pierre Laurent Wantzel
- sqp+
- std12-maths-em-1
- Applied Linear Algebra Review
- An Improved Artificial Bee Colony Approach to QoS-Aware Service Selection
- A FCC Main Runmefirst
- 150413 - Linear Algebra Syllabus
- f2l

You are on page 1of 22

7. LU factorization

factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm eect of rounding error sparse LU factorization

7-1

Factor-solve approach

to solve Ax = b, rst write A as a product of simple matrices A = A 1 A 2 Ak then solve (A1A2 Ak )x = b by solving k equations A1z1 = b, examples Cholesky factorization (for positive denite A) k = 2, A = LLT A2 z 2 = z 1 , ..., Ak1zk1 = zk2, Ak x = zk1

LU factorization

A = P LLT P

7-2

#ops = f + s f is cost of factoring A as A = A1A2 Ak (factorization step) s is cost of solving the k equations for z1, z2, . . . zk1, x (solve step) usually f s

example: positive denite equations using the Cholesky factorization f = (1/3)n3, s = 2n2

LU factorization

7-3

two equations with the same matrix but dierent right-hand sides Ax = b, factor A once (f ops) solve with right-hand side b (s ops) solve with right-hand side (s ops) b cost: f + 2s instead of 2(f + s) if we solve second equation from scratch A = x b

LU factorization

7-4

LU factorization

LU factorization without pivoting A = LU L unit lower triangular, U upper triangular does not always exist (even if A is nonsingular) LU factorization (with row pivoting) A = P LU P permutation matrix, L unit lower triangular, U upper triangular exists if and only if A is nonsingular (see later) cost: (2/3)n3 if A has order n

LU factorization 7-5

solve Ax = b with A nonsingular of order n factor-solve method using LU factorization 1. factor A as A = P LU ((2/3)n3 ops) 2. solve (P LU )x = b in three steps permutation: z1 = P T b (0 ops) forward substitution: solve Lz2 = z1 (n2 ops) back substitution: solve U x = z2 (n2 ops) total cost: (2/3)n3 + 2n2 ops, or roughly (2/3)n3 this is the standard method for solving Ax = b

LU factorization

7-6

two equations with the same matrix A (nonsingular and n n): Ax = b, factor A once forward/back substitution to get x forward/back substitution to get x cost: (2/3)n3 + 4n2 or roughly (2/3)n3 exercise: propose an ecient method for solving Ax = b, AT x = b A = x b

LU factorization

7-7

suppose A is nonsingular of order n, with LU factorization A = P LU

LU factorization

z2 = L1z1,

x = U 1z2

7-8

solve AX = I by solving n equations AX1 = e1, AX2 = e2, ..., AXn = en

Xi is the ith column of X; ei is the ith unit vector of size n one LU factorization of A: 2n3/3 ops n solve steps: 2n3 ops total: (8/3)n3 ops

LU factorization

7-9

partition A, L, U as block matrices: A= a11 A12 A21 A22 , L= 1 0 L21 L22 , U= u11 U12 0 U22

a11 and u11 are scalars L22 unit lower-triangular, U22 upper triangular of order n 1 determine L and U from A = LU , i.e., a11 A12 A21 A22 = = 1 0 L21 L22 u11 U12 0 U22

LU factorization

7-10

recursive algorithm: determine rst row of U and rst column of L u11 = a11, U12 = A12, L21 = (1/a11)A21

factor the (n 1) (n 1)-matrix A22 L21U12 as A22 L21U12 = L22U22 this is an LU factorization (without pivoting) of order n 1

LU factorization

7-11

Example

LU factorization (without pivoting) of 8 2 9 A= 4 9 4 6 7 9 write as A = LU with L unit lower triangular, U upper triangular u11 u12 u13 1 0 0 8 2 9 A = 4 9 4 = l21 1 0 0 u22 u23 0 0 u33 l31 l32 1 6 7 9

LU factorization

7-12

rst row of U , 8 4 6 9 4 7 9

rst column of L: 8 2 9 1 0 0 2 9 9 4 = 1/2 1 0 0 u22 u23 0 0 u33 3/4 l32 1 7 9 1/2 3/4 1 l32 0 1 u22 u23 0 u33 8 1/2 0 u33

1 0 11/16 1

third row of U : u33 = 9/4 + 11/32 = 83/32 conclusion: 8 2 9 1 0 0 8 2 9 1 0 0 8 1/2 A = 4 9 4 = 1/2 0 0 83/32 3/4 11/16 1 6 7 9

LU factorization 7-13

1 0 0 u11 u12 u13 1 0 0 2 = l21 1 0 0 u22 u23 A= 0 0 0 0 u33 l31 l32 1 0 1 1 rst row of U , 1 0 0 rst column of L: 1 0 0 1 0 0 0 0 0 2 = 0 1 0 0 u22 u23 0 0 u33 0 l32 1 1 1

LU factorization 7-14

if A is n n and nonsingular, then it can be factored as A = P LU P is a permutation matrix, L is unit lower triangular, U is upper triangular not unique; there may be several possible choices for P , L, U interpretation: permute the rows of A and factor P T A as P T A = LU also known as Gaussian elimination with partial pivoting (GEPP) cost: (2/3)n3 ops

LU factorization

7-15

Example

the factorization is not unique; the same matrix can be factored as 2 9 0 1 0 0 0 1 0 0 5 5 2 9 0 = 1 0 0 0 1 0 0 5 5 0 0 27 3 19/5 1 0 0 1 6 8 8

LU factorization

7-16

x1 x2

1 0

1 , 5 1 10

x2 =

1 1 105

let us solve the equations using LU factorization, rounding intermediate results to 4 signicant decimal digits we will do this for the two possible permutation matrices: P = 1 0 0 1 or P = 0 1 1 0

LU factorization

7-17

L, U rounded to 4 decimal signicant digits L= forward substitution 1 0 105 1 back substitution 105 1 0 105 error in x1 is 100%

LU factorization 7-18

1 0 105 1

U=

105 1 0 105

z1 z2

1 0

z1 = 1,

z2 = 105

x1 x2

1 105

x1 = 0,

x2 = 1

L, U rounded to 4 decimal signicant digits L= forward substitution 1 0 105 1 backward substitution 1 1 0 1 x1 x2 = 0 1 = x1 = 1, x2 = 1 z1 z2 = 0 1 = z1 = 0, z2 = 1 1 0 105 1 , U= 1 1 0 1

LU factorization 7-19

conclusion: for some choices of P , small rounding errors in the algorithm cause very large errors in the solution this is called numerical instability: for the rst choice of P , the algorithm is unstable; for the second choice of P , it is stable from numerical analysis: there is a simple rule for selecting a good (stable) permutation (well skip the details, since we skipped the details of the factorization algorithm) in the example, the second permutation is better because it permutes the largest element (in absolute value) of the rst column of A to the 1,1-position

LU factorization

7-20

if A is sparse, it is usually factored as A = P1LU P2 P1 and P2 are permutation matrices

T T interpretation: permute rows and columns of A and factor A = P1 AP2

A = LU choice of P1 and P2 greatly aects the sparsity of L and U : many heuristic methods exist for selecting good permutations in practice: #ops (2/3)n3; exact value is a complicated function of n, number of nonzero elements, sparsity pattern

LU factorization 7-21

Conclusion

dierent levels of understanding how linear equation solvers work: highest level: x = A\b costs (2/3)n3; more ecient than x = inv(A)*b intermediate level: factorization step A = P LU followed by solve step lowest level: details of factorization A = P LU

for most applications, level 1 is sucient in some situations (e.g., multiple right-hand sides) level 2 is useful level 3 is important only for experts who write numerical libraries

LU factorization

7-22

- Ray Theory in TomoraghyUploaded byatrij6124
- HW1Uploaded byMygod
- SUNTIKAN PMRSDANSPM08Uploaded bysahmozac
- Cholesky_LU_Factorization_1Uploaded byJigar Sheth
- Lec2-CommonAlgoUploaded byAl Muzammil Muhamad
- Galerkin DiscontinuousUploaded byxhanatos
- JournalUploaded byjoaov_36
- numethUploaded byNrapen Sipani
- IIT MMAE 501 Set 1 Solution(1)Uploaded byMatt Bishop
- 2.2 Inverse of a Matrix CorrectedUploaded byBosche Nero
- Computation Methods Lab#1 Nov:20:12Uploaded byข้าวเม่า ทอด
- 10.1.1.134.203Uploaded byDiponegoro Muhammad Khan
- Pierre Laurent WantzelUploaded byEddy R. Vélez
- sqp+Uploaded byAlneo Lesag
- std12-maths-em-1Uploaded byapi-320105488
- Applied Linear Algebra ReviewUploaded byEllie
- An Improved Artificial Bee Colony Approach to QoS-Aware Service SelectionUploaded bybebo_ha2008
- A FCC Main RunmefirstUploaded byArman Dabiri
- 150413 - Linear Algebra SyllabusUploaded byavmyemail
- f2lUploaded byVivian Villagran
- Best F2L AlgorithmsUploaded byGleiciano Santos
- final proj 2 paper esci 1Uploaded byapi-403978095
- 14-Solving Fem EquationsUploaded byOkoni Demas
- guidednotesUploaded byapi-431410685
- cljpracticlefileadaUploaded byaman
- 191 in Verses MatricesUploaded byvahid
- 009 Informed Search 1_ Hill Climbing.pptxUploaded byRashdeep Singh
- Tute_2_19 (MAN 001)Uploaded byUmesh Kumar
- Greedy AlgorithmUploaded bysadia abbas
- docx-1.docUploaded byMarcela Dobre

- Carl RogersUploaded byChristianieAnn
- OrganisationDevelopment NotesUploaded byS- Ajmeri
- Fundamentals of Resistance TrainingUploaded bySean Webster
- Infosys Hw4Uploaded byAdam Ong
- Internal Environment AnalysisUploaded byArifHossain
- SOA-12c-Installation-Guide.pdfUploaded byNTGDN
- Animal Clinical ChemistryUploaded byAngel Garcia
- 10g NF-II D39100Uploaded byAnonymous EZO2aZ
- Applying Lean Six SigmaUploaded byjames007spy
- Avoiding Pressure Surge Damage in PipelineUploaded byFerlie Indrapati
- Babaylan reaction paperUploaded byPatrisha Ivana Cordova
- Newsletter 2016 oUploaded byusmle
- bare_jrnlUploaded byMuhammad Rahmandani
- CoD Black Ops CheatsUploaded byAndré Hatushikano
- part b expectationsUploaded byapi-283978472
- Biological Effects of Electromagnetic InformationUploaded byGuido AB
- MuL 135 Music of the 20th Century 2018.DocUploaded byMichelle Mariposa
- Guia Adminis Traci on Res Cur SosUploaded byYolanda González González
- emnlp12Uploaded byMausam
- Principal Components RegressionUploaded byManjinder Singh
- 19 SlideUploaded byHoàng Vũ
- Fortidb User Guide 430Uploaded byrobertlatino16
- div-class-title-introduction-constructing-balkan-identity-in-recent-media-discourses-div.pdfUploaded byMarija Tepavac
- Shear Strength in the New Eurocode 2. a Step ForwardUploaded byChineseAssassin
- Optimizer.pdfUploaded byjhbr23
- Dell Dimension 4600 Owners ManualUploaded byRobert Goodall
- VESFG001Uploaded byromeo112976
- Activity Book for 3-4 years Children.pdfUploaded byMarietta Mosolygó
- Dickerman-Stock Illustrations of Animal Intelligence (TAPhA 42 [1911])Uploaded byBudbudabuddha
- installation_manual_ads_design_kit_v2-1.pdfUploaded byTeddy112

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.