jj

© All Rights Reserved

26 views

jj

© All Rights Reserved

- 1
- Nonlinear Regression EBS127
- SPE-129126-MS
- Breking Glass Cieling
- Neural Networks - Slides - CMU - Aarti Singh & Barnabas Poczos
- Linear Regression Models
- Teaching plan Beka2453
- An Update and Extension to SEM Guidelines for Administrative.
- Introduction Unit 2
- Due-Diligence-to-Assess-and-Validate-the-Students-Entrepreneurs-and-their-Business-Plans---An-Investors--Contemporary-Approach_2014_Procedia-Economics-and-Finance.pdf
- AgainstAllOdds_StudentGuide_Unit30
- The Relationship Between Absenteeism
- black box
- Practical Data Analysis With JMP
- Comm 215 Quiz 6
- RELATIONSHIP BETWEEN CADMIUM ACCUMULATION AND SOIL EC (A CASE STUDY OF FIVE AUTOMOBILE WORKSHOPS).
- 1_Optimal loading for maximizing power during sled- resisted sprinting.pdf
- A Theoretical Overview of Prospective Applications of Business Analytics in Electronic Commerce
- icoest_Kopsidas
- Linear Regression

You are on page 1of 36

Ingmar Schuster

Patrick Jhnichen

using slides by Andrew Ng

Institut fr Informatik

Linear Regression

Hypothesis formulation,

hypthesis space

Descent

Using multiple input features

with Linear Regression

Feature Scaling

Nonlinear Regression

derivatives

Linear Regression

Institut fr Informatik

Regression Problem

Linear regression w. gradient descent

x is input (predictor) variable

features in ML-speek

y is output (response) variable

Notation

Square meters

Price in 1000

73

174

146

367

38

69

124

257

...

...

Learning procedure

Hypothesis parameters

Training data

linear regression,

one input variable (univariate)

Learning Algorithm

Size

of flat

Estimated

price

hypothesis

(mapping between

input and output)

Linear regression w. gradient descent

Optimization objective

optimization objective and cost function (often called J)

...

(h(x) close to y for (x,y) in training data)

Cost function often named J

Number

Number of

of

training

training instances

instances

Squaring

Linear regression w. gradient descent

Optimizing Cost

with Gradient Descent

Want to minimize

Keep changing

to reduce

until we end up at minimum

10

Stepwise

Stepwise

descent

descent

towards

towards

minimum

minimum

Derivatives

Derivatives

work

work only

only for

for

few

few parameters

parameters

Gradient descent

partial

partial

derivative

derivative

beware: incremental

update incorrect!

steps

steps become

become smaller

smaller

without

without changing

changing

learning

learning rate

rate

12

convergence

not lead to convergence or to

divergence

Often

13

Checking convergence

correctly if

decreases

with every step

Possible convergence

criterion: converged if

decreases by less than

constant

14

Local Minima

(e.g. J not squared error for regression with only one variable)

Random restart

with different

parameter(s)

15

16

Multiple features

Square Bedrooms Floors Age of building

meters

(years)

x1

x2

x3

x4

Price in

1000

y

200

45

460

131

40

232

142

30

315

756

36

178

Notation

17

Hypothesis representation

More compact

with definition

18

19

20

21

(scales can vary strongly)

Square meters 30 - 400

Bedrooms

1 - 10

Price

80 000

2 000 000

22

Feature Scaling

23

Feature scaling

scale

Z-score conversion

24

Z-Score conversion

Center data on 0

mean

mean // empirical

empirical

expected

expected value

value

(mu)

(mu)

empirical

empirical standard

standard

deviation

deviation (sigma)

(sigma)

25

26

Nonlinear Regression

(by cheap trickery)

27

28

29

30

31

Optimizing cost

using derivatives

32

solve

for all i

33

Gradient Descent

Need to choose

Needs many iterations,

random restarts etc.

Works well for many features

Derivation

No need to choose

No iterations

34

Linear Regression

Hypothesis formulation,

hypthesis space

Descent

Using multiple input features

with Linear Regression

Feature Scaling

Nonlinear Regression

derivatives

35

Pictures

en.wikipedia.org and

de.wikipedia.org

36

- 1Uploaded byRajesh Vernekar
- Nonlinear Regression EBS127Uploaded byHamza Shaikh
- SPE-129126-MSUploaded bysanty222
- Breking Glass CielingUploaded byAastha Bhatia
- Neural Networks - Slides - CMU - Aarti Singh & Barnabas PoczosUploaded byTeppi Nguyen
- Linear Regression ModelsUploaded byMagdalena Simic
- Teaching plan Beka2453Uploaded byKoh Guan Keong
- An Update and Extension to SEM Guidelines for Administrative.Uploaded byPayal Anand
- Introduction Unit 2Uploaded bydarebusi1
- Due-Diligence-to-Assess-and-Validate-the-Students-Entrepreneurs-and-their-Business-Plans---An-Investors--Contemporary-Approach_2014_Procedia-Economics-and-Finance.pdfUploaded byRupali
- AgainstAllOdds_StudentGuide_Unit30Uploaded byHacı Osman
- The Relationship Between AbsenteeismUploaded byAnupa De Silva
- black boxUploaded byBocah Ilang
- Practical Data Analysis With JMPUploaded byrajv88
- Comm 215 Quiz 6Uploaded byzigzaro
- RELATIONSHIP BETWEEN CADMIUM ACCUMULATION AND SOIL EC (A CASE STUDY OF FIVE AUTOMOBILE WORKSHOPS).Uploaded byIJAR Journal
- 1_Optimal loading for maximizing power during sled- resisted sprinting.pdfUploaded byMateusz
- A Theoretical Overview of Prospective Applications of Business Analytics in Electronic CommerceUploaded byarcherselevators
- icoest_KopsidasUploaded byAnonymous zyIfM5S
- Linear RegressionUploaded byRishi
- 9390_4301Operationalisation II -Data AnalysisUploaded byAmiya Arnav
- final paperUploaded byapi-374258253
- Breathing ExcerciseUploaded byponnuswamyv
- research methodology.docxUploaded byK Raghavendra Bhat
- Sample Linear Regression DataUploaded byDejan Marić
- 17cbUploaded byShubhamAgarwal
- Contribution of Infrastructure to Economic Growth in AfricaUploaded byInternational Journal of Business Marketing and Management
- Utilizing Statistical Techniques in Estimating UncollectedUploaded byVISITHOD CIVIL
- Stick Slides tablesUploaded byJose David Lopez-Rivas
- In Statistical ModelingUploaded byBelen Loreto Gallano

- sostools.pdfUploaded bypalasscribd
- asmeUploaded byAnshu Kshetri
- Survey of Metaheuristics Solution MethodUploaded byJuan Sebastián Poveda Gulfo
- A Review of Genetic Algorithm Optimization: Operations and Applications to Water Pipeline SystemsUploaded byRobert Aguedo
- Heat ExchangerUploaded byFrancisca Hanna
- Reference Handbook on Power, Control and Communication Systems: Recent HeadwaysUploaded byElixir Publications
- Airfoil Shape OptimizationUploaded byCris Roppa
- MIE376 Lec 1a IntroUploaded byk2050896
- Ad Interim Report Group 11Uploaded byrussell_mahmood
- A L GUploaded byAtomuus
- Logic Synthesis 1 PavanKVUploaded bydk13071987
- LPP.CLASS SCHEDULING.pdfUploaded bySrijith M Menon
- Lecture 2 (Introduction to Lpp)Uploaded byShipra Narauney
- An Oil Pipeline Design ProblemUploaded byThe Ganteng
- Structural_Engg.pdfUploaded byMohammed Abid
- Methods of System Identification.pdfUploaded byAmmar Al-Kindy
- LambdaUploaded byRenato Goitia
- ArtificialRecharge OfGround WaterUploaded byDanang
- The Training Programming During the Competition Microcycle in HandballUploaded byAleksandar Mitevski
- Graphical LP SolutionUploaded byFanuel Njobvu
- Automation & Control - Theory and PracticeUploaded byoprw
- SDVRP GOlden Algunos Algoritmos y AproximacionesUploaded bygambito2000
- Lingo 11 Users ManualUploaded byAbdelatif Hr
- Fluent Workshop Lecture NotesUploaded byzubbbu
- Comparison of ILP Formulations for the RWA ProblemUploaded byKhông Có Tên
- me4315 engineriingUploaded byaldoartho
- Power Factor Iee PaperUploaded byPrakash Rao
- Modelling of Balçova District Heating SystemUploaded byİbrahim Akgün
- Optimal Tuning of PI Coefficients by Using Fuzzy-genetic ForUploaded bynoorania
- A Comprehensive Cost-Volume-Profit AnalysisUploaded byHardian_Kastho_9855