0 Up votes0 Down votes

0 views2 pagesSalient points in Continuous time markov chains

Jan 26, 2017

CTMC_SALIENTPOINTS_lec0322

© © All Rights Reserved

PDF, TXT or read online from Scribd

Salient points in Continuous time markov chains

© All Rights Reserved

0 views

CTMC_SALIENTPOINTS_lec0322

Salient points in Continuous time markov chains

© All Rights Reserved

- Neuromancer
- The E-Myth Revisited: Why Most Small Businesses Don't Work and
- How Not to Be Wrong: The Power of Mathematical Thinking
- Drive: The Surprising Truth About What Motivates Us
- Chaos: Making a New Science
- The Joy of x: A Guided Tour of Math, from One to Infinity
- How to Read a Person Like a Book
- Moonwalking with Einstein: The Art and Science of Remembering Everything
- The Wright Brothers
- The Other Einstein: A Novel
- The 6th Extinction
- The Housekeeper and the Professor: A Novel
- The Power of Discipline: 7 Ways it Can Change Your Life
- The 10X Rule: The Only Difference Between Success and Failure
- A Short History of Nearly Everything
- The Kiss Quotient: A Novel
- The End of Average: How We Succeed in a World That Values Sameness
- Made to Stick: Why Some Ideas Survive and Others Die
- Algorithms to Live By: The Computer Science of Human Decisions
- The Universe in a Nutshell

You are on page 1of 2

These questions provide a guide to your reading on the basics of continuous-time Markov

chains, covering material in Sections 1-4 in the long CTMC notes.

Here are SOME QUESTIONS:

1. What is a stochastic process?

2. What are the finite-dimensional distributions (f.d.d.s) of a stochastic process?

3. What is the probability law of a stochastic process?

4. When is a continuous-time stochastic process a Markov process? (See (2.1).)

5. What are the transition probabilities of a CTMC? (See (2.2)

6. How do you get from the transition probabilities of a CTMC to the finite-dimensional

distributions of a CTMC? (See (2.8.)

7. What are the Chapman-Kolmogorov equations of a CTMC? (See Lemma 2.1.)

8. What are the Kolmogorov ODEs of a CTMC, i.e., the Kolmogorov forward and

backward ordinary differential equations (ODEs) for a CTMC? (See Theorem 3.1)

9. What are the three main ways to construct a CTMC model? (See Sections 3.1, 3.2 and

3.3.)

10. How is the rate matrix Q defined? (See (3.1)-(3.3).)

11. Is the rate matrix Q a nonnegative matrix? Why or why not? (See (3.3)-(3.5).)

12. What are the Kolmogorov forward and backward ordinary differential equations (ODEs)

for a CTMC? (See Theorem 3.1.)

13. What is the solution of a usual one-dimensional ODE? (See the first display on p. 9.)

14. For an irreducible CTMC, is it possible to have Pi,j (t) = 0 for some i, j and t > 0?

(See Lemma 4.1.)

15. For a continuous-time stochastic process {X(t) : t 0}, what is a limiting probability vector? (See Section 4.)

16. For an irreducible finite-state CTMC, what is the limiting probability vector? (See

(4.1). We denote it by ; the component j is the limiting probability of being in state j.)

17. For a continuous-time stochastic process {X(t) : t 0}, what is a stationary probability vector? (See Section 4.)

18. For a continuous-time stochastic process {X(t) : t 0}, if the process has a unique

stationary vector, then does it necessarily also have a limiting a stationary probability vector?

(See Example 4.1.)

19. For a continuous-time stochastic process {X(t) : t 0}, if the process has a unique

limiting vector, then does it necessarily also have a stationary probability vector? (See Example

4.1.)

20. Does a finite-state CTMC necessarily have a unique stationary probability vector?

21. Does an irreducible finite-state CTMC necessarily have a unique stationary probability vector? (See Theorem 4.1.)

22. Given the rate matrix Q {Qi,j } of a finite-state irreducible CTMC, how can we

compute the limiting probability vector ? (See Theorem 4.2 (c).)

22. Given the matrix of transition probabilities P (t) {Pi,j (t)} of a finite-state irreducible

CTMC, for any fixed t > 0, how can the limiting probability vector be computed? (See

Theorem 4.2 (d).)

22. Given a finite-state irreducible CTMC characterized by an embedded DTMC with

one-step transition probabilities given by the matrix P {Pi,j } and exponential transition

times with means 1/i in state i, how can the limiting probability vector be computed? (See

Theorem 4.2 (b).) (See Theorem 4.2 (d).)

23. When is a CTMC a birth-and-death process? (See Section 5.)

24. How does the answer to question 21 simplify when the CTMC is a birth-and-death

process? (See Theorem 5.2.)

- Stochastic Calculus Final Exam With SolutionsUploaded byTrbvm
- Shin 2016Uploaded byhendra lam
- DTMCUploaded byVadthyavath Ramu
- Probabilités ULB302-9bisUploaded byAlcalinaPhysics
- Hamiltoniano VCUploaded byErick Santillan
- CubeSat Design Bleeping Interplanetary Radiations Determination Yo-yo (BIRDY) - AOCS Subsystem-reportUploaded byHaoChihLIN
- Slide 11Uploaded byMann Ot
- Modified Modified Modified Page Page Rank Rank Model Model in Investigation InvestigationInvestigationInvestigation Investigation of Criminal Criminal Criminal GangUploaded byATS
- paper1-integration.docUploaded byParidah Yaacob
- Building Models for Credit SpreadsUploaded byxiaojunm
- Hull White Tech NoteUploaded byAlexis Maenhout
- Modelling and estimation of social interaction effects in new product diffusionUploaded byapi-3851548
- Lec12_HMMUploaded byAnshy Singh
- Introduction to IntegrationUploaded byAbdul_Rahman_virk
- Discount factorUploaded byDevin Maa
- Stochastic modelling Notes DiscreteUploaded byBrofessor Paul Nguyen
- Pal GraveUploaded bycold_acid25
- Questions on Markov ChainUploaded byChandrashekhar Choudhari
- Lab Tti Rezolvari Intrebari EnglezaUploaded byElena Damoc
- CHAP02Uploaded byaavizu
- CLASS XII MCQ TEST (Ch 2-8) 2019-20-1.docxUploaded byJIVANSHU SHARMA
- Equation 2Uploaded byylfadei
- SimonUploaded byAdil Reza
- 3100_boc_memo_syllabus_2013-01-15Uploaded byafol_10190
- Economics-CBCS-PU-2018 (2).pdfUploaded byAhan
- Operations Management Module 3Uploaded byDiana Bluesea
- inphoUploaded byNikhil Raut
- Tute_1.pdfUploaded byपिकांशु कुमार
- mt-m-hs-set1Uploaded byjyotirmoy mondal
- Introduction to Random Vibration and Stochastic AnalysisUploaded byPongpasin Kamolchavarat

- 5c7e48cb9c233Uploaded byAaron Wilson
- input/output organisaion of computer architectureUploaded bynikhilmalagi
- kimiUploaded byHakimy Imy
- Informatica PowerCenter Scenario-IIUploaded bynelrick1984
- Introduction to Zoo GeographyUploaded byNununk Anaz
- Group and SubqueriesUploaded bykiran
- Maintenance Organization in a Fertilizer ComplexUploaded byKhurramRehmaan3167
- 120zxrgUploaded bygeekmania1
- 1405+UF1+(part+1+as+presented)Uploaded byChi Cụp
- BACKTRACKINGUploaded bylaklira
- Energy & Climate Change.pptUploaded byashwini_kumar1984
- Using Clues to Identify Elements LabUploaded byspecops47
- Automation PPTUploaded byMecstech Automation Jodhpur
- 60591428 New Tribological WaysUploaded byoperationmanager
- Outlining Critical Psychology of WorkUploaded byHernan Camilo
- PPC Syllabusq.docxUploaded byKannan Muthusamy
- Syllabus Regional Planning and PolicyUploaded byNelfania
- Appendix a MatlabTutorialUploaded byNick Johnson
- Decision and GamesUploaded byAmer Rahmah
- 2nd Grading Science 10Uploaded byRussel Ocho
- Fundamental Structural Analysis (2)Uploaded byChiều Tàn
- Cpd Sdu GuidelinesUploaded byJohn Kurong
- W701 W701ds DatasheetUploaded bysalahuddintipu
- Beyond Artifacts 2011Uploaded byJohn Alexander Gallin
- README Avid Editor v6.5.2 v10.5.2Uploaded byscribd!!
- Use of Spreadsheets in Solving Heat Conduction Problems in FinsUploaded byvinod patil
- Gyratory CrusherUploaded byadhiloveachan
- BestPractice02-ReduceMemoryUploaded byGerard Sorto
- PS.1.24Uploaded bybeikelboom
- Jason ReadUploaded bypenguin555

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.