0 Up votes0 Down votes

12 views11 pagesJan 29, 2013

© Attribution Non-Commercial (BY-NC)

PDF, TXT or read online from Scribd

Attribution Non-Commercial (BY-NC)

12 views

Attribution Non-Commercial (BY-NC)

- Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race
- Hidden Figures Young Readers' Edition
- The Law of Explosive Growth: Lesson 20 from The 21 Irrefutable Laws of Leadership
- The Art of Thinking Clearly
- The E-Myth Revisited: Why Most Small Businesses Don't Work and
- The Wright Brothers
- The Other Einstein: A Novel
- State of Fear
- State of Fear
- The Power of Discipline: 7 Ways it Can Change Your Life
- The Kiss Quotient: A Novel
- The 10X Rule: The Only Difference Between Success and Failure
- Being Wrong: Adventures in the Margin of Error
- Algorithms to Live By: The Computer Science of Human Decisions

You are on page 1of 11

Emmanuel Anguiano-Hernndez a

LabTL // CCC // INAOE

Emmanuel Anguiano-Hernndez a

Content

1 2 3 4 5

Boltzmann Distribution The Boltzmann Machine Restricted Boltzmann Machine Constrastive Divergenge RBM with temporal Restrictions

Emmanuel Anguiano-Hernndez a

Boltzmann Distribution

Probability for the distribution of the states of a system BD for the fractional number of particles states i possessing energy Ei is: Ni gi e (kB T ) = N Z (T )

kB is the Boltzmann Constant T is temperature gi is degeneracy, number of levels with energy Ei N particles number and Z (T ) is the partition function

Ei Ei

Ni N

occupying a set of

Z (T ) =

i

gi e (kB T )

Emmanuel Anguiano-Hernndez a

Boltzmann Machines

Stochastic Recurrent Neural Network Binary units Energy value dened for the network Can learn internal representations and solve dicult combinatoric problems (given enought time) Not practical if connectivity is unrestricted

Emmanuel Anguiano-Hernndez a

Boltzmann Machines

Global energy for a BM is: E =

i<j

wij si sj

i

i si

wi j is the strenght of the connection between unit i and unit j si correspond to the state of unit i, s {0, 1} is the threshold of unit i

j

E T )

At thermal equilibrium, the probability of any global state s when the network is free-running is given by the Boltzmann Distribution

Emmanuel Anguiano-Hernndez a Introduction to Restricted Boltzmann Machines

Biparthite graph, no intralayer connections Visible binary units act like inputs, hidden units can capture latent factors Can be analyzed with factor analysis

Emmanuel Anguiano-Hernndez a

Global energy for a given conguration (v, h) is: E (v , h) =

i

ai vi

j

bj hj

i j

hj wij vi

E (v , h) = aT v bT h hT Wv

W consist of a matriz of weights W = (wij ) h,v are hidden and visible unit states a,b are bias weights for visible and hidden units

Probabilty over hidden and visible vectors are dened in function of energy: 1 P(vh ) = e E (v ,h) Z And the marginal probability for a given visible conguration is a function of all hidden congurations: P(v ) =

Emmanuel Anguiano-Hernndez a

1 Z

e E (v ,h)

h

Introduction to Restricted Boltzmann Machines

Due to partition, activation of any layer is independent given the other layer.

m

P(v |h) =

i=1 n

P(vi |h)

P(h|v ) =

j=1

P(hj |v )

n

i=1 m

wij vi )

j=1

wij hj )

Emmanuel Anguiano-Hernndez a Introduction to Restricted Boltzmann Machines

Contrastive Divergence

State Activation:

To update the state of some units given others, lets assume we know the weights Compute activation energy of unit i ai =

j

wij xj

1 1+e x

The turn on unit i with probability pi = (ai ) and (ai ) = And turn o with probability 1 pi

Positive connections implies units to be in same states Negative connections implies dierent states

Emmanuel Anguiano-Hernndez a

Contrastive Divergence

Learning weights:

for each example: Put example in visible units Update states of hidden units using logistic activation, then for each edge eij : Pos(eij ) = xi \ xj Reconstruct visible units, for each compute ai and update states, then update hidden units again and compute: Neg (eij ) = xi \ xj Update the weight of each edge eij : wij = wij + L \ (Pos(eij ) Neg (eij )) Repeat over all examples

Emmanuel Anguiano-Hernndez a

Some units could be temporal Weight for this temporal units can contain a decay function over time or number of examples Experiments about how to integrate temporal decaying weights must be done Contrastive divergence should be extended in order to accept temporal units

Emmanuel Anguiano-Hernndez a Introduction to Restricted Boltzmann Machines

- Obstacle Avoidance - Duino-RoboticsUploaded byEmmanuel Mecanosaurio
- #Install Vlc on Rpi3.ShUploaded byEmmanuel Mecanosaurio
- README.txtUploaded byEmmanuel Mecanosaurio
- Tips Extensions OcUploaded byEmmanuel Mecanosaurio
- Circiutos Electronicos MusicalesUploaded byEmmanuel Mecanosaurio
- 3 Sound Synthesis ProcessingUploaded byEmmanuel Mecanosaurio
- CenturyUploaded byEmmanuel Mecanosaurio
- OfApp - vSynth.07Uploaded byEmmanuel Mecanosaurio
- uno o dos problemasUploaded byEmmanuel Mecanosaurio
- introduction to latent thingsUploaded byEmmanuel Mecanosaurio
- 450968421DDS_Tutorial_rev12-2-99Uploaded byAdriana Valo
- MIT6_096IAP11_lec01Uploaded byeluttner
- Scrapping the webUploaded byEmmanuel Mecanosaurio
- Introduction to RBMUploaded byEmmanuel Mecanosaurio
- method and notesUploaded byEmmanuel Mecanosaurio
- Revolutionary Self Theory Spectacular TimesUploaded byKelly Wood
- plsa introUploaded byEmmanuel Mecanosaurio
- LSA IntroductionUploaded bytristanperry

- A Graph Model TutorialUploaded byFarooq Sultan
- Jacobi Elliptic Functions - Wikipedia, The Free EncyclopediaUploaded bybmxengineering
- Linear Int (2)Uploaded byParasPratapSingh
- Essays on Number Theory 2Uploaded byagonza70
- Unit 2 CST Review EquationsUploaded byMisterLemus
- Pertemuan 6Uploaded byMuhammad Taufiq
- Vector SpaceUploaded byRahul Saha
- Indefinite IntegrationUploaded byAnmol Dev
- 01.Book About FuzzyUploaded byNita Ferdiana
- STQP2034 Tutorial 3Uploaded bySabariah Othman
- 18it03 ForneyUploaded bygonzoif
- ecl4Uploaded byD.n.Prasad
- test10.pdfUploaded byCarlo
- test review - 4 4 - 4-6Uploaded byapi-262532023
- TR645.pdfUploaded byShivaJps
- Notes Chapter 6 7Uploaded byawa_caem
- Solution of Linear and Nonlinear Partial Differential Equations Using Mixture of Elzaki Transform and the Projected Differential Transform MethodUploaded byAlexander Decker
- Part I - Basics FlexibilityUploaded byDawit Gebrehana
- Unit 0 - Review of Essential Mathematical ConceptsUploaded byMike Dalzim
- MATH HW 03052015Uploaded byQuinnNgo
- NaUploaded byarulmurugu
- Proof That e is IrrationalUploaded byRafih Yahya
- Class XII - Part 1.pdfUploaded bySathish Nirmal
- short-term-preparation-fsc-1-ptb-salman-sherazi.pdfUploaded byMuhammad Ali
- ZMLIB_ Fortran Multiple Precision LibraryUploaded byAnonymous 1rLNlqU
- 2ESO UNIT 2 Powers and RootsUploaded byArancha Blanco Garrido
- kmean-clustering.docxUploaded byminichel
- Tuple.pdfUploaded byDavid
- Hello WorldUploaded byMatteo Esposito
- OR Assignment.docxUploaded byshh

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.