0 Up votes0 Down votes

9 views4 pageskjhkjhjk

Jun 05, 2013

© Attribution Non-Commercial (BY-NC)

PDF, TXT or read online from Scribd

kjhkjhjk

Attribution Non-Commercial (BY-NC)

9 views

kjhkjhjk

Attribution Non-Commercial (BY-NC)

- Steve Jobs
- Wheel of Time
- NIV, Holy Bible, eBook
- NIV, Holy Bible, eBook, Red Letter Edition
- Cryptonomicon
- The Woman Who Smashed Codes: A True Story of Love, Spies, and the Unlikely Heroine who Outwitted America's Enemies
- Contagious: Why Things Catch On
- Crossing the Chasm: Marketing and Selling Technology Project
- Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are
- Zero to One: Notes on Start-ups, or How to Build the Future
- Console Wars: Sega, Nintendo, and the Battle that Defined a Generation
- Dust: Scarpetta (Book 21)
- Hit Refresh: The Quest to Rediscover Microsoft's Soul and Imagine a Better Future for Everyone
- The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution
- Crushing It!: How Great Entrepreneurs Build Their Business and Influence—and How You Can, Too
- Make Time: How to Focus on What Matters Every Day
- Algorithms to Live By: The Computer Science of Human Decisions
- Wild Cards

You are on page 1of 4

May 31, 2013 1. Suppose you are interviewing job applicants for a position. There are N applicants to interview lined up outside your o ce and you interview them one by one. Assume that each candidate gives a random reward R to your company that is distributed as an exponential random variable Pr[R < r] = 1; r 0 . r 1 e ; r>0

After each interview, you discover the realization of R for the candidate and may oer them the job, or tell them that you will not hire them. If you do the former then you stop and get R. If you do the latter then the candidate is lost. Let Vn be the optimal expected reward after interviewing (and rejecting) n 1 candidates. R1 (a) What is VN ? VN = E [R] = 0 re r dr = 1: (b) What is the optimal decision to make after interviewing and rejecting N dates? Accept the (N 1)st candidate if their value of R exceeds 1. (c) Show that VN

1

2 candi-

=e

0 1

=[ e + [ e r (r + 1)]1 1 1 = 1 e + 2e =e 1+1

r 1 ]0 1

1 Vn+1

Vn =

0 Vn+1 r V [ Vn+1 e r ]0 n+1 + [ Vn+1 Vn+1 e Vn+1 Vn+1

= Vn =

(r + 1)]1 Vn+1

Vn+1

+ Vn+1 e

+e

Vn+1

= Vn+1 + e

2. Consider computing innite horizon shortest paths in the following directed graph.

1

2 3 4

Let the costs of the arcs be given by cT = c12 c14 c23 c24 c31 c41 = 5 4 3 2 1 3 :

and suppose the discount discount factor is . Consider the problem of computing a never-ending path in which a move must be made from node to node at the start of every year. Write down a linear program in six variables that can be used to determine a policy that has minimum discounted cost. Let c(j; y ) = cjy as above, n =4: Pn Pn c(j; y )x(j; y ) D: minimize j =1 y =1 Pn Pn s.t. y =1 x(y; j ) = 1; j = 1; 2; : : : ; n: y =1 x(j; y ) x(j; y ) 0:

3. Consider an innite-horizon Markov decision process with states x =1, 2, and 3, and discount factor 0.9. In each state there are two actions (y = 1 and y = 2). The transition matrices are 2 1 1 3 2 1 1 1 3 0 2 2 3 3 3 P1 = 4 1 0 0 5; P2 = 4 1 0 0 5: 1 0 1 0 0 1 2 2 In each period there is a reward R(i) for being in state x, where R(1) = 0, R(2) = 3, R(3) = 2. (a) Write down a dynamic programming recursion that the optimal value V (x) = > V1 V2 V3 must satisfy. Then

1 > 2 V1 = maxfR(1) + re> 1 P V; R(1) + re1 P V g 1 > 2 V3 = maxfR(3) + re> 3 P V; R(3) + re3 P V g 2 1 1 32 3 2 1 3 1 0 V V + V 1 1 2 2 2 2 2 4 1 0 0 5 4 V2 5 = 4 5 V1 V3 V3 0 0 1 3 2 1 3 2 1 1 1 32 V1 V +1 V +1 V 3 3 3 3 1 3 2 3 3 5 4 1 0 0 5 4 V2 5 = 4 V1 1 1 1 1 V3 V + 2 V3 0 2 2 2 2 1 > 2 V2 = maxfR(2) + re> 2 P V; R(2) + re2 P V g

1 1 1 1 1 V1 = maxfr( V1 + V2 ); r( V1 + V2 + V3 )g 2 2 3 3 3 V2 = maxf3 + rV1 ; 3 + rV1 g 1 1 V3 = maxf2 + rV3 ; 2 + r( V2 + V3 )g 2 2 (b) Write down a formulation for solving the problem using linear programming. min V1 + V2 + V3 1 r( 2 V1 + 1 V) 2 2 1 1 V) r( 3 V1 + 3 V2 + 1 3 3 3 + rV1 3 + rV1 2 + rV3 1 2 + r( 1 V +2 V3 ) 2 2 V1 V1 V2 V2 V3 V4

(c) Write down an algorithm for solving the problem using policy iteration. (This is from notes. We illustrate this with one step). Consider the policy dened by actions y (1), y (2), y (3) in each state. We now compute V (x) corresponding to this policy. We can do this by choosing equations from the above set of inequaklities to dene V . Thus if y (1) = 2, y (2) = 1, y (3) = 1, we get 1 1 1 V1 = r( V1 + V2 + V3 ) 3 3 3 V2 = 3 + rV1 V3 = 2 + rV3

which has solution V1 = 16: 047, V2 = 17: 442; V3 = 20:0; We update the policy by putting these values into the recursion and testing y . 1 1 1 1 1 V1 = maxfr( V1 + V2 ); r( V1 + V2 + V3 )g = maxf15:070 05; 16:046 7g 2 2 3 3 3 V2 = maxf3 + rV1 ; 3 + rV1 g = maxf17: 442; 17: 442g 1 1 V3 = maxf2 + rV3 ; 2 + r( V2 + V3 )g = maxf20; 18:849g 2 2 This gives the optimal policy y (1) = 2, y (2) = 1, y (3) = 1, which has converged.

- Quantitative Techniques - TheoryUploaded byJoju Johny
- Deep Reinforcement Learning Hands-OnUploaded byCuong Ngoc Nguyen
- Churchland - A Critique of Pure VisionUploaded byEckart Zimmermann
- Tutorial Urbano 7 HydraUploaded byStroe George
- Leviticus Chapter 23Uploaded byWilliam Duodu
- Minimal Sense of Self, Temporality and the BrainUploaded byszshariq
- PsihologieUploaded bybp1977
- ApendiceUploaded byjtnylson
- b 035108011Uploaded byInternational Journal of computational Engineering research (IJCER)
- pavement Management System text bookUploaded byGoutham Athem
- Dissucion Traffic LightUploaded bySyed Zulfaizzuan Aljufri
- Spontaneous Pattern Formation in Primary Visual Cortex - Bressloff CowanUploaded byshangrilol
- Reseach ProjectUploaded byUsman Raza
- PustakaUploaded bypuspa
- Cluster AnalysisUploaded byNakul Garg
- Backus-2005-jov-5-11-10Uploaded bykaeru
- Assessor selection process for multysensory applicationsUploaded byHisbeth Mendez
- AoA important questionUploaded byShubham Kumar
- Dynamic Programming KnapsackUploaded byamitanna01
- Kober_IJRR_2013 (1).pdfUploaded byLaxmikanth Cherukupalli
- Nordtest Method Nt Fire 049Uploaded bymail2021
- DAAQBI-VUploaded bydgk84_id
- Graph ColouringUploaded byNivedita Mone
- Code Generation 1Uploaded byMaria Ghazalia Cameroon
- parallel DP 2Uploaded byapi-3756602
- Epidemiology of J Wave and Early Repolarization SyndromeUploaded bypuchio
- Module a Power Point > ModA2_ch04_engUploaded byapi-3812894
- MITRES_6_007S11_hw19Uploaded byxabihdez05
- process scheduling algorithmsUploaded byKartik Verma
- 171473Uploaded byPradeep Joshi

- Go-Jek in IndonesiaUploaded byJonathan Febrian
- Energy Dissipation in Hydraulic (RESUMO)Uploaded byDaniel Tavares
- Esoteric Astrology ArticleUploaded byValentin Badea
- T.C. Melewar, Elif Karaosmanoglu (2006) Seven Dimensions of Corporate Identity a Categorisation From the Practitioners' Perspectives, European Journal of Marketing, Vol. 40 Iss 78, Pp.846 - 869Uploaded bySiti Sarah Zolkifli
- gps101_meetgeodesyUploaded byosqureshi
- CV.docUploaded byshadik
- PMP_ITTOUploaded byabirami.narayanan857
- Firmware-ipc r6 Std 5.3.6Uploaded bygarbage10
- From BPEL Processes to YAWL WorkﬂowsUploaded byRazvan Popescu
- Business MeetingUploaded byDileepa Jayasiri
- Dynamics of Rigid BodiesUploaded byFernando Alcala Dela Cruz
- Deepak.docxUploaded byharikrishnan
- ER_1110-3-1301Uploaded byAhmad Ramin Abasy
- Thesis)Uploaded bylinkxyousuf
- Vir Sapiens Dominabitur Astris - Helena Avelar de CarvalhoUploaded byVeljko Gavric
- Trainz Simulator Mac - ManualUploaded byMiguel Ángel López Ortiz
- [9789004395114 - Reading Proclus and the <i>Book of Causes<_i> Volume 1] Reading Proclus and the <i>Book of Causes<_i> Volume 1.pdfUploaded byMiguel Leão
- IEM TALKUploaded bynitiran
- Group CommunicationUploaded bysurya
- Published Curry Leaves Research ArticleUploaded byDharti Kapadiya
- Project DescriptionUploaded byMehdi Zeinali
- SamplingDistributionExamples SolutionUploaded byMichael Brian Torres
- Secured Image Transmission using Mosaic ImagesUploaded byIJSTE
- Listong Pamilya FrameworkUploaded byEfren Jr Megio
- Density Ref5Uploaded byTechnos_Inc
- The-ITIL-Practitioner-Certificate-Syllabus.pdfUploaded bysupertester777
- Challenges and Perceptions Towards Use of Social Media in Higher Education in Zimbabwe (1)Uploaded byAhmad Sina Sabawoon
- The Design of Safe Chemical ProcessesUploaded bymarcalpi
- active demo - optics 2005 1Uploaded bydavparan
- Work Measurement ReportUploaded byAnamika Nigam

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.