Ierarchical Emporal Emory: HTM Cortical Learning Algorithms

HIERARCHICALTEMPORALMEMORY
including
HTMCorticalLearningAlgorithms
VERSION0.1.1,NOVEMBER23,2010
Numenta,Inc.2010
UseofNumentassoftwareandintellectualproperty,includingtheideascontainedinthis
document,arefreefornoncommercialresearchpurposes.Fordetails,see
http://www.numenta.com/softwareoverview/licensing.php.
Numenta2010 Page2
ReadThisFirst!
Thisisanearlydraftofthisdocument.Thereareseveralthingsmissingthatyou
shouldbeawareof.
WhatISinthisdocument:
Thisdocumentdescribesindetailnewalgorithmsforlearningandprediction
developedbyNumentain2010.Thenewalgorithmsaredescribedinsufficient
detailthataprogrammercanunderstandandimplementthemifdesired.Itstarts
withanintroductorychapter.IfyouhavebeenfollowingNumentaandhaveread
someofourpastwhitepapers,thematerialintheintroductorychapterwillbe
familiar.Theothermaterialisnew.
WhatisNOTinthisdocument:
Thereareseveraltopicsrelatedtotheimplementationofthesenewalgorithmsthat
didnotmakeitintothisearlydraft.
Althoughmostaspectsofthealgorithmshavebeenimplementedandtestedin
software,noneofthetestresultsarecurrentlyincluded.
Thereisnodescriptionofhowthealgorithmscanbeappliedtopracticalproblems.
Missingisadescriptionofhowyouwouldconvertdatafromasensorordatabase
intoadistributedrepresentationsuitableforthealgorithms.
Thealgorithmsarecapableofonlinelearning.Afewdetailsneededtofully
implementonlinelearninginsomerarercasesarenotdescribed.
Thereareseveralplannedchaptersthatarenotyetincluded.Theyinclude
chaptersonthebiologicalbasisofthealgorithms,adiscussionofthepropertiesof
sparsedistributedrepresentations,andchaptersonapplicationsandexamples.
Wearemakingthisdocumentavailableinitscurrentformbecausewethinkthe
algorithmswillbeofinteresttoothers.Themissingcomponentsofthedocument
shouldnotimpedeunderstandingandexperimentingwiththealgorithmsby
motivatedresearchers.Wewillrevisethisdocumentregularlytoreflectour
progress.

Numenta2010 Page3
TableofContents
Preface 4
Chapter1:HTMOverview 7
Chapter2:HTMCorticalLearningAlgorithms 19
Chapter3:SpatialPoolingImplementationandPseudocode 34
Chapter4:TemporalPoolingImplementationandPseudocode 39
Glossary 47
AppendixA:AComparisonbetweenBiologicalNeurons 51
andHTMCells
Futurechapterswillincludeinformationonapplications,examples,comparisonto
othermodelsofmachinelearning,abibliography,achapteronthebiologicalbasis
ofHTM,andsparsedistributedrepresentations.
Numenta2010 Page4
Preface
Therearemanythingshumansfindeasytodothatcomputersarecurrentlyunable
todo.Taskssuchasvisualpatternrecognition,understandingspokenlanguage,
recognizingandmanipulatingobjectsbytouch,andnavigatinginacomplexworld
areeasyforhumans.Yetdespitedecadesofresearch,wehavefewviable
algorithmsforachievinghumanlikeperformanceonacomputer.
Inhumans,thesecapabilitiesarelargelyperformedbytheneocortex.Hierarchical
TemporalMemory(HTM)isatechnologymodeledonhowtheneocortexperforms
thesefunctions.HTMoffersthepromiseofbuildingmachinesthatapproachor
exceedhumanlevelperformanceformanycognitivetasks.
ThisdocumentdescribesHTMtechnology.Chapter1providesabroadoverviewof
HTM,outliningtheimportanceofhierarchicalorganization,sparsedistributed
representations,andlearningtimebasedtransitions.Chapter2describestheHTM
corticallearningalgorithmsindetail.Chapters3and4providepseudocodeforthe
HTMlearningalgorithmsdividedintwopartscalledthespatialpoolerandtemporal
pooler.Afterreadingchapters2through4,experiencedsoftwareengineersshould
beabletoreproduceandexperimentwiththealgorithms.Hopefully,somereaders
willgofurtherandextendourwork.
Intendedaudience
Thisdocumentisintendedforatechnicallyeducatedaudience.Whilewedont
assumepriorknowledgeofneuroscience,wedoassumeyoucanunderstand
mathematicalandcomputerscienceconcepts.Wevewrittenthisdocumentsuch
thatitcouldbeusedasassignedreadinginaclass.Ourprimaryimaginedreaderis
astudentincomputerscienceorcognitivescience,orasoftwaredeveloperwhois
interestedinbuildingartificialcognitivesystemsthatworkonthesameprinciples
asthehumanbrain.
Nontechnicalreaderscanstillbenefitfromcertainpartsofthedocument,
particularlyChapter1:HTMOverview.
Softwarerelease

Itisourintentiontoreleasesoftwarebasedonthealgorithmsdescribedinthis
documentinmid2011.
Numenta2010 Page5
Relationtopreviousdocuments

PartsofHTMtheoryaredescribedinthe2004bookOnIntelligence,inwhitepapers
publishedbyNumenta,andinpeerreviewedpaperswrittenbyNumenta
employees.Wedontassumeyouvereadanyofthispriormaterial,muchofwhich
hasbeenincorporatedandupdatedinthisvolume.NotethattheHTMlearning
algorithmsdescribedinChapters24havenotbeenpreviouslypublished.Thenew
algorithmsreplaceourfirstgenerationalgorithms,calledZeta1.Forashorttime,
wecalledthenewalgorithmsFixeddensityDistributedRepresentations,orFDR,
butwearenolongerusingthisterminology.WecallthenewalgorithmstheHTM
CorticalLearningAlgorithms,orsometimesjusttheHTMLearningAlgorithms.
WeencourageyoutoreadOnIntelligence,writtenbyNumentacofounderJeff
HawkinswithSandraBlakeslee.AlthoughthebookdoesnotmentionHTMbyname,
itprovidesaneasytoread,nontechnicalexplanationofHTMtheoryandthe
neurosciencebehindit.AtthetimeOnIntelligencewaswritten,weunderstoodthe
basicprinciplesunderlyingHTMbutwedidntknowhowtoimplementthose
principlesalgorithmically.Youcanthinkofthisdocumentascontinuingthework
startedinOnIntelligence.

AboutNumenta

Numenta,Inc.(www.numenta.com)wasformedin2005todevelopHTMtechnology
forbothcommercialandscientificuse.Toachievethisgoalwearefully
documentingourprogressanddiscoveries.Wealsopublishoursoftwareinaform
thatotherpeoplecanuseforbothresearchandcommercialdevelopment.Wehave
structuredoursoftwaretoencouragetheemergenceofanindependent,application
developercommunity.UseofNumentassoftwareandintellectualpropertyisfree
forresearchpurposes.Wewillgeneraterevenuebysellingsupport,licensing
software,andlicensingintellectualpropertyforcommercialdeployments.We
alwayswillseektomakeourdeveloperpartnerssuccessful,aswellasbesuccessful
ourselves.
NumentaisbasedinRedwoodCity,California.Itisprivatelyfunded.
Abouttheauthors
ThisdocumentisacollaborativeeffortbytheemployeesofNumenta.Thenamesof
theprincipalauthorsforeachsectionarelistedintherevisionhistory.
Numenta2010 Page6
Revisionhistory
Wenoteinthetablebelowmajorchangesbetweenversions.Minorchangessuchassmall
clarificationsorformattingchangesarenotnoted.
Version Date Changes PrincipalAuthors

0.1 Nov9,2010 1.FirstreleaseofPreface,
Chapters1,2,3,4,andGlossary.
JeffHawkins,
SubutaiAhmad,
DonnaDubinsky
0.1.1 Nov23,2010 1.InChapter1,theRegions
sectionwaseditedtoclarify
terminology,suchaslevels,
columnsandlayers.
2.FirstreleaseofAppendixA.
JeffHawkins
Numenta2010 Page7
Chapter1:HTMOverview

HierarchicalTemporalMemory(HTM)isamachinelearningtechnologythataimsto
capturethestructuralandalgorithmicpropertiesoftheneocortex.
Theneocortexistheseatofintelligentthoughtinthemammalianbrain.Highlevel
vision,hearing,touch,movement,language,andplanningareallperformedbythe
neocortex.Givensuchadiversesuiteofcognitivefunctions,youmightexpectthe
neocortextoimplementanequallydiversesuiteofspecializedneuralalgorithms.
Thisisnotthecase.Theneocortexdisplaysaremarkablyuniformpatternofneural
circuitry.Thebiologicalevidencesuggeststhattheneocorteximplementsa
commonsetofalgorithmstoperformmanydifferentintelligencefunctions.
HTMprovidesatheoreticalframeworkforunderstandingtheneocortexandits
manycapabilities.Todatewehaveimplementedasmallsubsetofthistheoretical
framework.Overtime,moreandmoreofthetheorywillbeimplemented.Today
webelievewehaveimplementedasufficientsubsetofwhattheneocortexdoesto
beofcommercialandscientificvalue.
ProgrammingHTMsisunlikeprogrammingtraditionalcomputers.Withtodays
computers,programmerscreatespecificprogramstosolvespecificproblems.By
contrast,HTMsaretrainedthroughexposuretoastreamofsensorydata.The
HTMscapabilitiesaredeterminedlargelybywhatithasbeenexposedto.
HTMscanbeviewedasatypeofneuralnetwork.Bydefinition,anysystemthat
triestomodelthearchitecturaldetailsoftheneocortexisaneuralnetwork.
However,onitsown,thetermneuralnetworkisnotveryusefulbecauseithas
beenappliedtoalargevarietyofsystems.HTMsmodelneurons(calledcellswhen
referringtoHTM),whicharearrangedincolumns,inlayers,inregions,andina
hierarchy.Thedetailsmatter,andinthisregardHTMsareanewformofneural
network.
Asthenameimplies,HTMisfundamentallyamemorybasedsystem.HTMnetworks
aretrainedonlotsoftimevaryingdata,andrelyonstoringalargesetofpatterns
andsequences.Thewaydataisstoredandaccessedislogicallydifferentfromthe
standardmodelusedbyprogrammerstoday.Classiccomputermemoryhasaflat
organizationanddoesnothaveaninherentnotionoftime.Aprogrammercan
implementanykindofdataorganizationandstructureontopoftheflatcomputer
memory.Theyhavecontroloverhowandwhereinformationisstored.Bycontrast,
HTMmemoryismorerestrictive.HTMmemoryhasahierarchicalorganizationand
isinherentlytimebased.Informationisalwaysstoredinadistributedfashion.A
userofanHTMspecifiesthesizeofthehierarchyandwhattotrainthesystemon,
buttheHTMcontrolswhereandhowinformationisstored.
Numenta2010 Page8
AlthoughHTMnetworksaresubstantiallydifferentthanclassiccomputing,wecan
usegeneralpurposecomputerstomodelthemaslongasweincorporatethekey
functionsofhierarchy,timeandsparsedistributedrepresentations(describedin
detaillater).Webelievethatovertime,specializedhardwarewillbecreatedto
generatepurposebuiltHTMnetworks.
Inthisdocument,weoftenillustrateHTMpropertiesandprinciplesusingexamples
drawnfromhumanvision,touch,hearing,language,andbehavior.Suchexamples
areusefulbecausetheyareintuitiveandeasilygrasped.However,itisimportantto
keepinmindthatHTMcapabilitiesaregeneral.Theycanjustaseasilybeexposed
tononhumansensoryinputstreams,suchasradarandinfrared,ortopurely
informationalinputstreamssuchasfinancialmarketdata,weatherdata,Webtraffic
patterns,ortext.HTMsarelearningandpredictionmachinesthatcanbeappliedto
manytypesofproblems.
HTMprinciples
Inthissection,wecoversomeofthecoreprinciplesofHTM:whyhierarchical
organizationisimportant,howHTMregionsarestructured,whydataisstoredas
sparsedistributedrepresentations,andwhytimebasedinformationiscritical.
Hierarchy

AnHTMnetworkconsistsofregionsarrangedinahierarchy.Theregionisthemain
unitofmemoryandpredictioninanHTM,andwillbediscussedindetailinthenext
section.Typically,eachHTMregionrepresentsonelevelinthehierarchy.Asyou
ascendthehierarchythereisalwaysconvergence,multipleelementsinachild
regionconvergeontoanelementinaparentregion.However,duetofeedback
connections,informationalsodivergesasyoudescendthehierarchy.(Aregion
andalevelarealmostsynonymous.Weusethewordregionwhendescribing
theinternalfunctionofaregion,whereasweusethewordlevelwhenreferring
specificallytotheroleoftheregionwithinthehierarchy.)

Numenta2010 Page9
Figure 1.1: Simplified diagram of four HTM regions arranged in a four-level hierarchy,
communicating information within levels, between levels, and to/from outside the hierarchy
ItispossibletocombinemultipleHTMnetworks.Thiskindofstructuremakes
senseifyouhavedatafrommorethanonesourceorsensor.Forexample,one
networkmightbeprocessingauditoryinformationandanothernetworkmightbe
processingvisualinformation.Thereisconvergencewithineachseparatenetwork,
withtheseparatebranchesconvergingonlytowardsthetop.
Figure 1.2: Converging networks from different sensors
Thebenefitofhierarchicalorganizationisefficiency.Itsignificantlyreduces
trainingtimeandmemoryusagebecausepatternslearnedateachlevelofthe
hierarchyarereusedwhencombinedinnovelwaysathigherlevels.Foran
illustration,letsconsidervision.Atthelowestlevelofthehierarchy,yourbrain
storesinformationabouttinysectionsofthevisualfieldsuchasedgesandcorners.
Anedgeisafundamentalcomponentofmanyobjectsintheworld.Theselowlevel
patternsarerecombinedatmidlevelsintomorecomplexcomponentssuchas
curvesandtextures.Anarccanbetheedgeofanear,thetopofasteeringwheelor
therimofacoffeecup.Thesemidlevelpatternsarefurthercombinedtorepresent
highlevelobjectfeatures,suchasheads,carsorhouses.Tolearnanewhighlevel
objectyoudonthavetorelearnitscomponents.
Numenta2010 Page10
Asanotherexample,considerthatwhenyoulearnanewword,youdontneedto
relearnletters,syllables,orphonemes.
Sharingrepresentationsinahierarchyalsoleadstogeneralizationofexpected
behavior.Whenyouseeanewanimal,ifyouseeamouthandteethyouwillpredict
thattheanimaleatswithhismouthandthatitmightbiteyou.Thehierarchy
enablesanewobjectintheworldtoinherittheknownpropertiesofitssub
components.
HowmuchcanasinglelevelinanHTMhierarchylearn?Orputanotherway,how
manylevelsinthehierarchyarenecessary?Thereisatradeoffbetweenhowmuch
memoryisallocatedtoeachlevelandhowmanylevelsareneeded.Fortunately,
HTMsautomaticallylearnthebestpossiblerepresentationsateachlevelgiventhe
statisticsoftheinputandtheamountofresourcesallocated.Ifyouallocatemore
memorytoalevel,thatlevelwillformrepresentationsthatarelargerandmore
complex,whichinturnmeansfewerhierarchicallevelsmaybenecessary.Ifyou
allocatelessmemory,alevelwillformrepresentationsthataresmallerandsimpler,
whichinturnmeansmorehierarchicallevelsmaybeneeded.
Uptothispointwehavebeendescribingdifficultproblems,suchasvisioninference
(inferenceissimilartopatternrecognition).Butmanyvaluableproblemsare
simplerthanvision,andasingleHTMregionmightprovesufficient.Forexample,
weappliedanHTMtopredictingwhereapersonbrowsingawebsiteislikelyto
clicknext.ThisprobleminvolvedfeedingtheHTMnetworkstreamsofwebclick
data.Inthisproblemtherewaslittleornospatialhierarchy,thesolutionmostly
requireddiscoveringthetemporalstatistics,i.e.predictingwheretheuserwould
clicknextbyrecognizingtypicaluserpatterns.Thetemporallearningalgorithmsin
HTMsareidealforsuchproblems.
Insummary,hierarchiesreducetrainingtime,reducememoryusage,andintroduce
aformofgeneralization.However,manysimplerpredictionproblemscanbesolved
withasingleHTMregion.
Regions
Thenotionofregionswiredinahierarchycomesfrombiology.Theneocortexisa
largesheetofneuraltissueabout2mmthick.Biologistsdividetheneocortexinto
differentareasorregionsprimarilybasedonhowtheregionsconnecttoeach
other.Someregionsreceiveinputdirectlyfromthesensesandotherregions
receiveinputonlyafterithaspassedthroughseveralotherregions.Itistheregion
toregionconnectivitythatdefinesthehierarchy.
Allneocorticalregionslooksimilarintheirdetails.Theyvaryinsizeandwherethey
areinthehierarchy,butotherwisetheyaresimilar.Ifyoutakeasliceacrossthe
2mmthicknessofaneocorticalregion,youwillseesixlayers,fivelayersofcellsand
Numenta2010 Page11
onenoncellularlayer(thereareafewexceptionsbutthisisthegeneralrule).Each
layerinaneocorticalregionhasmanyinterconnectedcellsarrangedincolumns.
HTMregionsalsoarecomprisedofasheetofhighlyinterconnectedcellsarrangedin
columns.Layer3inneocortexisoneoftheprimaryfeedforwardlayersof
neurons.ThecellsinanHTMregionareroughlyequivalenttotheneuronsinlayer
3inaregionoftheneocortex.
Figure 1.3: A section of an HTM region. HTM regions are comprised of many cells. The cells
are organized in a two dimensional array of columns. This figure shows a small section of an
HTM region with four cells per column. Each column connects to a subset of the input and each
cell connects to other cells in the region (connections not shown). Note that this HTM region,
including its columnar structure, is equivalent to one layer of neurons in a neocortical region.
AlthoughanHTMregionisequivalenttoonlyaportionofaneocorticalregion,itcan
doinferenceandpredictiononcomplexdatastreamsandthereforecanbeusefulin
manyproblems.
SparseDistributedRepresentations
Althoughneuronsintheneocortexarehighlyinterconnected,inhibitoryneurons
guaranteethatonlyasmallpercentageoftheneuronsareactiveatonetime.Thus,
informationinthebrainisalwaysrepresentedbyasmallpercentageofactive
neuronswithinalargepopulationofneurons.Thiskindofencodingiscalleda
sparsedistributedrepresentation.Sparsemeansthatonlyasmallpercentageof
neuronsareactiveatonetime.Distributedmeansthattheactivationsofmany
neuronsarerequiredinordertorepresentsomething.Asingleactiveneuron
conveyssomemeaningbutitmustbeinterpretedwithinthecontextofapopulation
ofneuronstoconveythefullmeaning.
HTMregionsalsousesparsedistributedrepresentations.Infact,thememory
mechanismswithinanHTMregionaredependentonusingsparsedistributed
representations,andwouldntworkotherwise.TheinputtoanHTMregionis
alwaysadistributedrepresentation,butitmaynotbesparse,sothefirstthingan
HTMregiondoesistoconvertitsinputintoasparsedistributedrepresentation.
Numenta2010 Page12
Forexample,aregionmightreceive20,000inputbits.Thepercentageofinputbits
thatare1and0mightvarysignificantlyovertime.Onetimetheremightbe
5,0001bitsandanothertimetheremightbe9,0001bits.TheHTMregioncould
convertthisinputintoaninternalrepresentationof10,000bitsofwhich2%,or
200,areactiveatonce,regardlessofhowmanyoftheinputbitsare1.Asthe
inputtotheHTMregionvariesovertime,theinternalrepresentationalsowill
change,buttherealwayswillbeabout200bitsoutof10,000active.
Itmayseemthatthisprocessgeneratesalargelossofinformationasthenumberof
possibleinputpatternsismuchgreaterthanthenumberofpossiblerepresentations
intheregion.However,bothnumbersareincrediblybig.Theactualinputsseenby
aregionwillbeaminisculefractionofallpossibleinputs.Laterwewilldescribe
howaregioncreatesasparserepresentationfromitsinput.Thetheoreticallossof
informationwillnothaveapracticaleffect.
Figure 1.4: An HTM region showing sparse distributed cell activation

Sparsedistributedrepresentationshaveseveraldesirablepropertiesandare
integraltotheoperationofHTMs.Theywillbetouchedonagainlater.
Theroleoftime
Timeplaysacrucialroleinlearning,inference,andprediction.
Letsstartwithinference.Withoutusingtime,wecaninferalmostnothingfromour
tactileandauditorysenses.Forexampleifyouareblindfoldedandsomeoneplaces
anappleinyourhand,youcanidentifywhatitisaftermanipulatingitforjusta
secondorso.Asyoumoveyourfingersovertheapple,althoughthetactile
informationisconstantlychanging,theobjectitselftheapple,aswellasyourhigh
levelperceptforapplestaysconstant.However,ifanapplewasplacedonyour
outstretchedpalm,andyouwerentallowedtomoveyourhandorfingers,you
wouldhavegreatdifficultyidentifyingitasanappleratherthanalemon.
Numenta2010 Page13
Thesameistrueforhearing.Astaticsoundconveyslittlemeaning.Awordlike
apple,orthecrunchingsoundsofsomeonebitingintoanapple,canonlybe
recognizedfromthedozensorhundredsofrapid,sequentialchangesovertimeof
thesoundspectrum.
Vision,incontrast,isamixedcase.Unlikewithtouchandhearing,humansareable
torecognizeimageswhentheyareflashedinfrontofthemtoofasttogivetheeyesa
chancetomove.Thus,visualinferencedoesnotalwaysrequiretimechanging
inputs.However,duringnormalvisionweconstantlymoveoureyes,headsand
bodies,andobjectsintheworldmovearoundustoo.Ourabilitytoinferbasedon
quickvisualexposureisaspecialcasemadepossiblebythestatisticalpropertiesof
visionandyearsoftraining.Thegeneralcaseforvision,hearing,andtouchisthat
inferencerequirestimechanginginputs.
Havingcoveredthegeneralcaseofinference,andthespecialcaseofvisioninference
ofstaticimages,letslookatlearning.Inordertolearn,allHTMsystemsmustbe
exposedtotimechanginginputsduringtraining.Eveninvision,wherestatic
inferenceissometimespossible,wemustseechangingimagesofobjectstolearn
whatanobjectlookslike.Forexample,imagineadogisrunningtowardyou.At
eachinstanceintimethedogcausesapatternofactivityontheretinainyoureye.
Youperceivethesepatternsasdifferentviewsofthesamedog,butmathematically
thepatternsareentirelydissimilar.Thebrainlearnsthatthesedifferentpatterns
meanthesamethingbyobservingtheminsequence.Timeisthesupervisor,
teachingyouwhichspatialpatternsgotogether.
Notethatitisntsufficientforsensoryinputmerelytochange.Asuccessionof
unrelatedsensorypatternswouldonlyleadtoconfusion.Thetimechanginginputs
mustcomefromacommonsourceintheworld.Notealsothatalthoughweuse
humansensesasexamples,thegeneralcaseappliestononhumansensesaswell.If
wewanttotrainanHTMtorecognizepatternsfromapowerplantstemperature,
vibrationandnoisesensors,theHTMwillneedtobetrainedondatafromthose
sensorschangingthroughtime.
Typically,anHTMnetworkneedstobetrainedwithlotsofdata.Youlearnedto
identifydogsbyseeingmanyinstancesofmanybreedsofdogs,notjustonesingle
viewofonesingledog.ThejoboftheHTMalgorithmsistolearnthetemporal
sequencesfromastreamofinputdata,i.e.tobuildamodelofwhichpatternsfollow
whichotherpatterns.Thisjobisdifficultbecauseitmaynotknowwhensequences
startandend,theremaybeoverlappingsequencesoccurringatthesametime,
learninghastooccurcontinuously,andlearninghastooccurinthepresenceof
noise.
Learningandrecognizingsequencesisthebasisofformingpredictions.Oncean
HTMlearnswhatpatternsarelikelytofollowotherpatterns,itcanpredictthelikely
Numenta2010 Page14
nextpattern(s)giventhecurrentinputandimmediatelypastinputs.Predictionis
coveredinmoredetaillater.
WenowwillturntothefourbasicfunctionsofHTM:learning,inference,prediction,
andbehavior.EveryHTMregionperformsthefirstthreefunctions:learning,
inference,andprediction.Behavior,however,isdifferent.Weknowfrombiology
thatmostneocorticalregionshavearoleincreatingbehaviorbutwedonotbelieve
itisessentialformanyinterestingapplications.Thereforewehavenotincluded
behaviorinourcurrentimplementationofHTM.Wementionitherefor
completeness.
Learning

AnHTMregionlearnsaboutitsworldbyfindingpatternsandthensequencesof
patternsinsensorydata.Theregiondoesnotknowwhatitsinputsrepresent;it
worksinapurelystatisticalrealm.Itlooksforcombinationsofinputbitsthatoccur
togetheroften,whichwecallspatialpatterns.Itthenlooksforhowthesespatial
patternsappearinsequenceovertime,whichwecalltemporalpatternsor
sequences.
Iftheinputtotheregionrepresentsenvironmentalsensorsonabuilding,theregion
mightdiscoverthatcertaincombinationsoftemperatureandhumidityonthenorth
sideofthebuildingoccuroftenandthatdifferentcombinationsoccuronthesouth
sideofthebuilding.Thenitmightlearnthatsequencesofthesecombinationsoccur
aseachdaypasses.
Iftheinputtoaregionrepresentedinformationrelatedtopurchaseswithinastore,
theHTMregionmightdiscoverthatcertaintypesofarticlesarepurchasedon
weekends,orthatwhentheweatheriscoldcertainpricerangesarefavoredinthe
evening.Thenitmightlearnthatdifferentindividualsfollowsimilarsequential
patternsintheirpurchases.
AsingleHTMregionhaslimitedlearningcapability.Aregionautomaticallyadjusts
whatitlearnsbasedonhowmuchmemoryithasandthecomplexityoftheinputit
receives.Thespatialpatternslearnedbyaregionwillnecessarilybecomesimplerif
thememoryallocatedtoaregionisreduced.Orthespatialpatternslearnedcan
becomemorecomplexiftheallocatedmemoryisincreased.Ifthelearnedspatial
patternsinaregionaresimple,thenahierarchyofregionsmaybeneededto
understandcompleximages.Weseethischaracteristicinthehumanvisionsystem
wheretheneocorticalregionreceivinginputfromtheretinalearnsspatialpatterns
forsmallpartsofthevisualspace.Onlyafterseverallevelsofhierarchydospatial
patternscombineandrepresentmostorallofthevisualspace.
Numenta2010 Page15
Likeabiologicalsystem,thelearningalgorithmsinanHTMregionarecapableof
onlinelearning,i.e.theycontinuallylearnfromeachnewinput.Thereisntaneed
foralearningphaseseparatefromaninferencephase,thoughinferenceimproves
afteradditionallearning.Asthepatternsintheinputchange,theHTMregionwill
graduallychange,too.
Afterinitialtraining,anHTMcancontinuetolearnor,alternatively,learningcanbe
disabledafterthetrainingphase.Anotheroptionistoturnofflearningonlyatthe
lowestlevelsofthehierarchybutcontinuetolearnatthehigherlevels.Oncean
HTMhaslearnedthebasicstatisticalstructureofitsworld,mostnewlearning
occursintheupperlevelsofthehierarchy.IfanHTMisexposedtonewpatterns
thathavepreviouslyunseenlowlevelstructure,itwilltakelongerfortheHTMto
learnthesenewpatterns.Weseethistraitinhumans.Learningnewwordsina
languageyoualreadyknowisrelativelyeasy.However,ifyoutrytolearnnew
wordsfromaforeignlanguagewithunfamiliarsounds,youllfinditmuchharder
becauseyoudontalreadyknowthelowlevelsounds.
Simplydiscoveringpatternsisapotentiallyvaluablecapability.Understandingthe
highlevelpatternsinmarketfluctuations,disease,weather,manufacturingyield,or
failuresofcomplexsystems,suchaspowergrids,isvaluableinitself.Evenso,
learningspatialandtemporalpatternsismostlyaprecursortoinferenceand
prediction.

Inference

AfteranHTMhaslearnedthepatternsinitsworld,itcanperforminferenceon
novelinputs.WhenanHTMreceivesinput,itwillmatchittopreviouslylearned
spatialandtemporalpatterns.Successfullymatchingnewinputstopreviously
storedsequencesistheessenceofinferenceandpatternmatching.
Thinkabouthowyourecognizeamelody.Hearingthefirstnoteinamelodytells
youlittle.Thesecondnotenarrowsdownthepossibilitiessignificantlybutitmay
stillnotbeenough.Usuallyittakesthree,four,ormorenotesbeforeyourecognize
themelody.InferenceinanHTMregionissimilar.Itisconstantlylookingata
streamofinputsandmatchingthemtopreviouslylearnedsequences.AnHTM
regioncanfindmatchesfromthebeginningofsequencesbutusuallyitismorefluid,
analogoustohowyoucanrecognizeamelodystartingfromanywhere.Because
HTMregionsusedistributedrepresentations,theregionsuseofsequencememory
andinferencearemorecomplicatedthanthemelodyexampleimplies,butthe
examplegivesaflavorforhowitworks.
Itmaynotbeimmediatelyobvious,buteverysensoryexperienceyouhaveeverhad
hasbeennovel,yetyoueasilyfindfamiliarpatternsinthisnovelinput.For
example,youcanunderstandthewordbreakfastspokenbyalmostanyone,no
Numenta2010 Page16
matterwhethertheyareoldoryoung,maleorfemale,arespeakingquicklyor
slowly,orhaveastrongaccent.Evenifyouhadthesamepersonsaythesameword
breakfastahundredtimes,thesoundwouldneverstimulateyourcochleae
(auditoryreceptors)inexactlythesamewaytwice.
AnHTMregionfacesthesameproblemyourbraindoes:inputsmayneverrepeat
exactly.Consequently,justlikeyourbrain,anHTMregionmusthandlenovelinput
duringinferenceandtraining.OnewayanHTMregioncopeswithnovelinputis
throughtheuseofsparsedistributedrepresentations.Akeypropertyofsparse
distributedrepresentationsisthatyouonlyneedtomatchaportionofthepattern
tobeconfidentthatthematchissignificant.

Prediction

EveryregionofanHTMstoressequencesofpatterns.Bymatchingstored
sequenceswithcurrentinput,aregionformsapredictionaboutwhatinputswill
likelyarrivenext.HTMregionsactuallystoretransitionsbetweensparse
distributedrepresentations.Insomeinstancesthetransitionscanlooklikealinear
sequence,suchasthenotesinamelody,butinthegeneralcasemanypossible
futureinputsmaybepredictedatthesametime.AnHTMregionwillmakedifferent
predictionsbasedoncontextthatmightstretchbackfarintime.Themajorityof
memoryinanHTMisdedicatedtosequencememory,orstoringtransitions
betweenspatialpatterns.
FollowingaresomekeypropertiesofHTMprediction.
1)Predictioniscontinuous.
Withoutbeingconsciousofit,youareconstantlypredicting.HTMsdothesame.
Whenlisteningtoasong,youarepredictingthenextnote.Whenwalkingdownthe
stairs,youarepredictingwhenyourfootwilltouchthenextstep.Whenwatchinga
baseballpitcherthrow,youarepredictingthattheballwillcomenearthebatter.In
anHTMregion,predictionandinferencearealmostthesamething.Predictionis
notaseparatestepbutintegraltothewayanHTMregionworks.
2)Predictionoccursineveryregionateverylevelofthehierarchy.
IfyouhaveahierarchyofHTMregions,predictionwilloccurateachlevel.Regions
willmakepredictionsaboutthepatternstheyhavelearned.Inalanguageexample,
lowerlevelregionsmightpredictpossiblenextphonemes,andhigherlevelregions
mightpredictwordsorphrases.
3)Predictionsarecontextsensitive.
Predictionsarebasedonwhathasoccurredinthepast,aswellaswhatisoccurring
now.Thusaninputwillproducedifferentpredictionsbasedonpreviouscontext.
AnHTMregionlearnstouseasmuchpriorcontextasneeded,andcankeepthe
Numenta2010 Page17
contextoverbothshortandlongstretchesoftime.Thisabilityisknownasvariable
ordermemory.Forexample,thinkaboutamemorizedspeechsuchasthe
GettysburgAddress.Topredictthenextword,knowingjustthecurrentwordis
rarelysufficient;thewordandisfollowedbysevenandlaterbydedicatedjust
inthefirstsentence.Sometimes,justalittlebitofcontextwillhelpprediction;
knowingfourscoreandwouldhelppredictseven.Othertimes,thereare
repetitivephrases,andonewouldneedtousethecontextofafarlongertimeframe
toknowwhereyouareinthespeech,andthereforewhatcomesnext.
4)Predictionleadstostability.
Theoutputofaregionisitsprediction.OneofthepropertiesofHTMsisthatthe
outputsofregionsbecomemorestablethatisslowerchanging,longerlasting
thehighertheyareinthehierarchy.Thispropertyresultsfromhowaregion
predicts.Aregiondoesntjustpredictwhatwillhappenimmediatelynext.Ifitcan,
itwillpredictmultiplestepsaheadintime.Letssayaregioncanpredictfivesteps
ahead.Whenanewinputarrives,thenewlypredictedstepchangesbutthefourof
thepreviouslypredictedstepsmightnot.Consequently,eventhougheachnew
inputiscompletelydifferent,onlyapartoftheoutputischanging,makingoutputs
morestablethaninputs.Thischaracteristicmirrorsourexperienceofthereal
world,wherehighlevelconceptssuchasthenameofasongchangemoreslowly
thanlowlevelconceptstheactualnotesofthesong.
5)Apredictiontellsusifanewinputisexpectedorunexpected.
EachHTMregionisanoveltydetector.Becauseeachregionpredictswhatwill
occurnext,itknowswhensomethingunexpectedhappens.HTMscanpredict
manypossiblenextinputssimultaneously,notjustone.Soitmaynotbeableto
predictexactlywhatwillhappennext,butifthenextinputdoesntmatchanyofthe
predictionstheHTMregionwillknowthatananomalyhasoccurred.
6)Predictionhelpsmakethesystemmorerobusttonoise.
WhenanHTMpredictswhatislikelytohappennext,thepredictioncanbiasthe
systemtowardinferringwhatitpredicted.Forexample,ifanHTMwereprocessing
spokenlanguage,itwouldpredictwhatsounds,words,andideasarelikelytobe
utterednext.Thispredictionhelpsthesystemfillinmissingdata.Ifanambiguous
soundarrives,theHTMwillinterpretthesoundbasedonwhatitisexpecting,thus
helpinginferenceeveninthepresenceofnoise.
InanHTMregion,sequencememory,inference,andpredictionareintimately
integrated.Theyarethecorefunctionsofaregion.
Numenta2010 Page18
Behavior

Ourbehaviorinfluenceswhatweperceive.Aswemoveoureyes,ourretina
receiveschangingsensoryinput.Movingourlimbsandfingerscausesvaryingtouch
sensationtoreachthebrain.Almostallouractionschangewhatwesense.Sensory
inputandmotorbehaviorareintimatelyentwined.
Fordecadestheprevailingviewwasthatasingleregionintheneocortex,the
primarymotorregion,waswheremotorcommandsoriginatedintheneocortex.
Overtimeitwasdiscoveredthatmostorallregionsintheneocortexhaveamotor
output,evenlowlevelsensoryregions.Itappearsthatallcorticalregionsintegrate
sensoryandmotorfunctions.
WeexpectthatamotoroutputcouldbeaddedtoeachHTMregionwithinthe
currentlyexistingframeworksincegeneratingmotorcommandsissimilarto
makingpredictions.However,alltheimplementationsofHTMstodatehavebeen
purelysensory,withoutamotorcomponent.
ProgresstowardtheimplementationofHTM
WehavemadesubstantialprogressturningtheHTMtheoreticalframeworkintoa
practicaltechnology.WehaveimplementedandtestedseveralversionsoftheHTM
corticallearningalgorithmsandhavefoundthebasicarchitecturetobesound.As
wetestthealgorithmsonnewdatasets,wewillrefinethealgorithmsandadd
missingpieces.Wewillupdatethisdocumentaswedo.Thenextthreechapters
describethecurrentstateofthealgorithms.
Therearemanycomponentsofthetheorythatarenotyetimplemented,including
attention,feedbackbetweenregions,specifictiming,andbehavior/sensorymotor
integration.Thesemissingcomponentsshouldfitintotheframeworkalready
created.

Numenta2010 Page19
Chapter2:HTMCorticalLearningAlgorithms
ThischapterdescribesthelearningalgorithmsatworkinsideanHTMregion.
Chapters3and4describetheimplementationofthelearningalgorithmsusing
pseudocode,whereasthischapterismoreconceptual.
Terminology
Beforewegetstarted,anoteaboutterminologymightbehelpful.Weusethe
languageofneuroscienceindescribingtheHTMlearningalgorithms.Termssuchas
cells,synapses,potentialsynapses,dendritesegments,andcolumnsareused
throughout.Thisterminologyislogicalsincethelearningalgorithmswerelargely
derivedbymatchingneurosciencedetailswiththeoreticalneeds.However,inthe
processofimplementingthealgorithmswewereconfrontedwithperformance
issuesandthereforeoncewefeltweunderstoodhowsomethingworkedwewould
lookforwaystospeedprocessing.Thisofteninvolveddeviatingfromastrict
adherencetobiologicaldetailsaslongaswecouldgetthesameresults.Ifyouare
newtoneurosciencethiswontbeaproblem.However,ifyouarefamiliarwith
neuroscienceterms,youmightfindyourselfconfusedasouruseoftermsvaries
fromyourexpectation.Weintendtowriteachapteronthebiologicalmappingof
theHTMlearningalgorithms,butrightnowitwillbehelpfultomentionafewofthe
deviationsthatarelikelytocausethemostconfusion.
Cellstates
HTMcellshavethreeoutputstates,activefromfeedforwardinput,activefrom
lateralinput,andinactive.Thefirstoutputstatecorrespondstoashortburstof
actionpotentials.Thesecondoutputstatecorrespondstoaslower,steadyrateof
actionpotentials.Wehavenotfoundaneedformodelingindividualaction
potentialsorevenscalarratesofactivitybeyondthetwoactivestates.Theuseof
distributedrepresentationsseemstoovercometheneedtomodelscalaractivity
ratesincells.
Dendritesegments
HTMcellshavearelativelyrealistic(andthereforecomplex)dendritemodel.In
theoryeachHTMcellhasoneproximaldendritesegmentandadozenortwodistal
dendritesegments.Theproximaldendritesegmentreceivesfeedforwardinputand
thedistaldendritesegmentsreceivelateralinputfromnearbycells.Aclassof
inhibitorycellsforcesallthecellsinacolumntorespondtosimilarfeedforward
input.Tosimplify,weremovedtheproximaldendritesegmentfromeachcelland
replaceditwithasingleshareddendritesegmentpercolumnofcells.Thespatial
poolerfunction(describedbelow)operatesontheshareddendritesegment,atthe
levelofcolumns.Thetemporalpoolerfunctionoperatesondistaldendrite
segments,atthelevelofindividualcellswithincolumns.Thissimplification
Numenta2010 Page20
achievesthesamefunctionality,thoughinbiologythereisnoequivalenttoa
dendritesegmentattachedtoacolumn.
Synapses
HTMsynapseshavebinaryweights.Biologicalsynapseshavevaryingweightsbut
theyarealsopartiallystochastic,suggestingabiologicalneuroncannotrelyon
precisesynapticweights.TheuseofdistributedrepresentationsinHTMsplusour
modelofdendriteoperationallowsustoassignbinaryweightstoHTMsynapses
withnoilleffect.Tomodeltheformingandunformingofsynapsesweusetwo
additionalconceptsfromneurosciencethatyoumaynotbefamiliarwith.Oneisthe
conceptofpotentialsynapses.Thisrepresentsalltheaxonsthatpassclose
enoughtoadendritesegmentthattheycouldpotentiallyformasynapse.The
secondiscalledpermanence.Thisisascalarvalueassignedtoeachpotential
synapse.Thepermanenceofasynapserepresentsarangeofconnectedness
betweenanaxonandadendrite.Biologically,therangewouldgofromcompletely
unconnected,tostartingtoformasynapsebutnotconnectedyet,toaminimally
connectedsynapse,toalargefullyconnectedsynapse.Thepermanenceofa
synapseisascalarvaluerangingfrom0to1.Learninginvolvesincrementingand
decrementingasynapsespermanence.Whenasynapsespermanenceisabovea
threshold,itisconnectedwithaweightof1.Whenitisbelowthethreshold,itis
unconnectedwithaweightof0.
Overview
ImaginethatyouarearegionofanHTM.Yourinputconsistsofthousandsortensof
thousandsofbits.Theseinputbitsmayrepresentsensorydataortheymaycome
fromanotherregionlowerinthehierarchy.Theyareturningonandoffincomplex
ways.Whatareyousupposedtodowiththisinput?
Wealreadyhavediscussedtheanswerinitssimplestform.EachHTMregionlooks
forcommonpatternsinitsinputandthenlearnssequencesofthosepatterns.From
itsmemoryofsequences,eachregionmakespredictions.Thathighlevel
descriptionmakesitsoundeasy,butinrealitythereisalotgoingon.Letsbreakit
downalittlefurtherintothefollowingthreesteps:
1)Formasparsedistributedrepresentationoftheinput
2)Formarepresentationoftheinputinthecontextofpreviousinputs
3)Formapredictionbasedonthecurrentinputinthecontextofpreviousinputs
Wewilldiscusseachofthesestepsinmoredetail.
Numenta2010 Page21
1)Formasparsedistributedrepresentationoftheinput
Whenyouimagineaninputtoaregion,thinkofitasalargenumberofbits.Ina
brainthesewouldbeaxonsfromneurons.Atanypointintimesomeoftheseinput
bitswillbeactive(value1)andotherswillbeinactive(value0).Thepercentageof
inputbitsthatareactivevary,sayfrom0%to60%.ThefirstthinganHTMregion
doesistoconvertthisinputintoanewrepresentationthatissparse.Forexample,
theinputmighthave40%ofitsbitsonbutthenewrepresentationhasjust2%of
itsbitson.
AnHTMregionislogicallycomprisedofasetofcolumns.Eachcolumniscomprised
ofoneormorecells.Columnsmaybelogicallyarrangedina2Darraybutthisisnot
arequirement.Eachcolumninaregionisconnectedtoauniquesubsetoftheinput
bits(usuallyoverlappingwithothercolumnsbutneverexactlythesamesubsetof
inputbits).Asaresult,differentinputpatternsresultindifferentlevelsofactivation
ofthecolumns.Thecolumnswiththestrongestactivationinhibit,ordeactivate,the
columnswithweakeractivation.(Theinhibitionoccurswithinaradiusthatcan
spanfromverylocaltotheentireregion.)Thesparserepresentationoftheinputis
encodedbywhichcolumnsareactiveandwhichareinactiveafterinhibition.The
inhibitionfunctionisdefinedtoachievearelativelyconstantpercentageofcolumns
tobeactive,evenwhenthenumberofinputbitsthatareactivevariessignificantly.

Figure 2.1: An HTM region consists of columns of cells. Only a small portion of a region is shown.
Each column of cells receives activation from a unique subset of the input. Columns with the
strongest activation inhibit columns with weaker activation. The result is a sparse distributed
representation of the input (active columns are shown in light grey).
Imaginenowthattheinputpatternchanges.Ifonlyafewinputbitschange,some
columnswillreceiveafewmoreorafewlessinputsintheonstate,butthesetof
activecolumnswillnotlikelychangemuch.Thussimilarinputpatterns(onesthat
haveasignificantnumberofactivebitsincommon)willmaptoarelativelystable
setofactivecolumns.Howstabletheencodingisdependsgreatlyonwhatinputs
eachcolumnisconnectedto.Theseconnectionsarelearnedviaamethoddescribed
later.
Numenta2010 Page22
Allthesesteps(learningtheconnectionstoeachcolumnfromasubsetoftheinputs,
determiningthelevelofinputtoeachcolumn,andusinginhibitiontoselectasparse
setofactivecolumns)isreferredtoastheSpatialPooler.Thetermmeans
patternsthatarespatiallysimilar(meaningtheysharealargenumberofactive
bits)arepooled(meaningtheyaregroupedtogetherinacommon
representation).
2)Formarepresentationoftheinputinthecontextofpreviousinputs
Thenextfunctionperformedbyaregionistoconvertthecolumnarrepresentation
oftheinputintoanewrepresentationthatincludesstate,orcontext,fromthepast.
Thenewrepresentationisformedbyactivatingasubsetofthecellswithineach
column,typicallyonlyonecellpercolumn.
Considerhearingtwospokensentences,IateapearandIhaveeightpears.The
wordsateandeightarehomonyms;theysoundidentical.Wecanbecertain
thatatsomepointinthebrainthereareneuronsthatrespondidenticallytothe
spokenwordsateandeight.Afterall,identicalsoundsareenteringtheear.
However,wealsocanbecertainthatatanotherpointinthebraintheneuronsthat
respondtothisinputaredifferent,indifferentcontexts.Therepresentationsforthe
soundatewillbedifferentwhenyouhearIatevs.Ihaveeight.Imaginethat
youhavememorizedthetwosentencesIateapearandIhaveeightpears.
HearingIateleadstoadifferentpredictionthanIhaveeight.Theremustbe
differentinternalrepresentationsafterhearingIateandIhaveeight.
Thisprincipleofencodinganinputdifferentlyindifferentcontextsisauniversal
featureofperceptionandactionandisoneofthemostimportantfunctionsofan
HTMregion.Itishardtooveremphasizetheimportanceofthiscapability.
EachcolumninanHTMregionconsistsofmultiplecells.Eachcellinacolumncan
beactiveornotactive.Byselectingdifferentactivecellsineachactivecolumn,we
canrepresenttheexactsameinputdifferentlyindifferentcontexts.Aspecific
examplemighthelp.Sayeverycolumnhas4cellsandtherepresentationofevery
inputconsistsof100activecolumns.Ifonlyonecellpercolumnisactiveatatime,
wehave4^100waysofrepresentingtheexactsameinput.Thesameinputwill
alwaysresultinthesame100columnsbeingactive,butindifferentcontexts
differentcellsinthosecolumnswillbeactive.Nowwecanrepresentthesameinput
inaverylargenumberofcontexts,buthowuniquewillthosedifferent
representationsbe?Nearlyallrandomlychosenpairsofthe4^100possible
patternswilloverlapbyabout25cells.Thustworepresentationsofaparticular
inputindifferentcontextswillhaveabout25cellsincommonand75cellsthatare
different,makingthemeasilydistinguishable.
ThegeneralruleusedbyanHTMregionisthefollowing.Whenacolumnbecomes
active,itwillactivateallthecellsinthecolumniftheinputwasunexpected.Ifone
Numenta2010 Page23
ormorecellsinthecolumnareinthepredictivestate,onlythosecellswillbecome
active,therestofthecellsinthecolumnremaininactive.
Ifthereisnopriorstate,andthereforenocontextandprediction,allthecellsina
columnwillbecomeactivewhenthecolumnbecomesactive.Thisscenariois
similartohearingthefirstnoteinasong.Withoutcontextyouusuallycantpredict
whatwillhappennext;alloptionsareavailable.Ifthereispriorstatebuttheinput
doesnotmatchwhatisexpected,allthecellsintheactivecolumnwillbecome
active.Thisdeterminationisdoneonacolumnbycolumnbasissoapredictive
matchormismatchisneveranallornothingevent.

Figure 2.2: By activating a subset of cells in each column, an HTM region can represent the same
input in many different contexts. Columns only activate predicted cells. Columns with no
predicted cells activate all the cells in the column. The figure shows some columns with one cell
active and some columns with all cells active.
.

Asmentionedintheterminologysectionabove,HTMcellscanbeinoneofthree
states.Ifacellisactiveduetofeedforwardinputwejustusethetermactive.If
thecellisactiveduetolateralconnectionstoothernearbycellswesayitisinthe
predictivestate.
3)Formapredictionbasedontheinputinthecontextofpreviousinputs
Thefinalstepforourregionistomakeapredictionofwhatislikelytohappennext.
Thepredictionisbasedontherepresentationformedinstep2),whichincludes
contextfromallpreviousinputs.
Whenaregionmakesapredictionitactivates(intothepredictivestate)allthecells
thatwilllikelybecomeactiveduetofuturefeedforwardinput.Because
representationsinaregionaresparse,multiplepredictionscanbemadeatthesame
time.Forexampleif2%ofthecolumnsareactiveduetoaninput,youcouldexpect
thattendifferentpredictionscouldbemaderesultingin20%ofthecolumnshaving
apredictedcell.Or,twentydifferentpredictionscouldbemaderesultingin40%of
Numenta2010 Page24
thecolumnshavingapredictedcell.Ifeachcolumnhadfourcells,withoneactiveat
atime,then10%ofthecellswouldbeinthepredictivestate.
Afuturechapteronsparsedistributedrepresentationswillshowthateventhough
differentpredictionsaremergedtogether,aregioncanknowwithhighcertainty
whetheraparticularinputwaspredictedornot.
Howdoesaregionmakeaprediction?Wheninputpatternschangeovertime,
differentsetsofcolumnsandcellsbecomeactiveinsequence.Whenacellbecomes
active,itformsconnectionstoasubsetofthecellsnearbythatwereactive
immediatelyprior.Theseconnectionscanbeformedquicklyorslowlydepending
onthelearningraterequiredbytheapplication.Later,allacellneedstodoisto
lookattheseconnectionsforcoincidentactivity.Iftheconnectionsbecomeactive,
thecellcanexpectthatitmightbecomeactiveshortlyandentersapredictivestate.
Thusthefeedforwardactivationofasetofcellswillleadtothepredictive
activationofothersetsofcellsthattypicallyfollow.Thinkofthisasthemoment
whenyourecognizeasongandstartpredictingthenextnotes.

Figure 2.3: At any point in time, some cells in an HTM region will be active due to feed-forward
input (shown in light gray). Other cells that receive lateral input from active cells will be in a
predictive state (shown in dark gray).

Insummary,whenanewinputarrives,itleadstoasparsesetofactivecolumns.
Oneormoreofthecellsineachcolumnbecomeactive,theseinturncauseother
cellstoenterapredictivestatethroughlearnedconnectionsbetweencellsinthe
region.Thecellsactivatedbyconnectionswithintheregionconstituteaprediction
ofwhatislikelytohappennext.Whenthenextfeedforwardinputarrives,itselects
anothersparsesetofactivecolumns.Ifanewlyactivecolumnisunexpected,
meaningitwasnotpredictedbyanycells,itwillactivateallthecellsinthecolumns.
Ifanewlyactivecolumnhasoneormorepredictedcells,onlythosecellswill
becomeactive.Theoutputofaregionistheactivityofallcellsintheregion,
includingthecellsactivebecauseoffeedforwardinputandthecellsactiveinthe
predictivestate.
Numenta2010 Page25
Asmentionedearlier,predictionsarenotjustforthenexttimestep.Predictionsin
anHTMregioncanbeforseveraltimestepsintothefuture.Usingmelodiesas
example,anHTMregionwouldnotjustpredictthenextnoteinamelody,butmight
predictthenextfournotes.Thisleadstoadesirableproperty.Theoutputofa
region(theunionofalltheactiveandpredictedcellsinaregion)changesmore
slowlythantheinput.Imaginetheregionispredictingthenextfournotesina
melody.WewillrepresentthemelodybythelettersequenceA,B,C,D,E,F,G.After
hearingthefirsttwonotes,theregionrecognizesthesequenceandstartspredicting.
ItpredictsC,D,E,F.TheBcellsarealreadyactivesocellsforB,C,D,E,Fareallinone
ofthetwoactivestates.NowtheregionhearsthenextnoteC.Thesetofactive
andpredictivecellsnowrepresentsC,D,E,F,G.Notethattheinputpatternchanged
completelygoingfromBtoC,butonly20%ofthecellschanged.
BecausetheoutputofanHTMregionisavectorrepresentingtheactivityofallthe
regionscells,theoutputinthisexampleisfivetimesmorestablethantheinput.In
ahierarchicalarrangementofregions,wewillseeanincreaseintemporalstability
asyouascendthehierarchy.
Weusethetermtemporalpoolertodescribethetwostepsofaddingcontextto
therepresentationandpredicting.Bycreatingslowlychangingoutputsfor
sequencesofpatterns,weareinessencepoolingtogetherdifferentpatternsthat
followeachotherintime.
Nowwewillgointoanotherlevelofdetail.Westartwithconceptsthatareshared
bythespatialpoolerandtemporalpooler.Thenwediscussconceptsanddetails
uniquetothespatialpoolerfollowedbyconceptsanddetailsuniquetothetemporal
pooler.
Sharedconcepts
Learninginthespatialpoolerandtemporalpooleraresimilar.Learninginboth
casesinvolvesestablishingconnections,orsynapses,betweencells.Thetemporal
poolerlearnsconnectionsbetweencellsinthesameregion.Thespatialpooler
learnsfeedforwardconnectionsbetweeninputbitsandcolumns.
Binaryweights
HTMsynapseshaveonlya0or1effect;theirweightisbinary,apropertyunlike
manyneuralnetworkmodelswhichusescalarvariablevaluesintherangeof0to1.
Permanence
Synapsesareformingandunformingconstantlyduringlearning.Weassignascalar
valuetoeachsynapse(0.0to1.0)toindicatehowpermanenttheconnectionis.
Whenaconnectionisreinforced,itspermanenceisincreased.Underother
Numenta2010 Page26
conditions,thepermanenceisdecreased.Whenthepermanenceisabovea
threshold(e.g.0.2),thesynapseisconsideredtobeestablished.Ifthepermanence
isbelowthethreshold,thesynapsewillhavenoeffect.
Dendritesegments
Synapsesconnecttodendritesegments.Therearetwotypesofdendritesegments.
Onetypeofdendritesegmentformssynapseswithfeedforwardinputs.Theactive
synapsesonthistypeofsegmentarelinearlysummedtodeterminethefeed
forwardactivationofacolumn.
Theothertypeofdendritesegmentformssynapseswithcellswithintheregion.
Everycellhasseveraldendritesegmentsofthistype.Ifthesumoftheactive
synapsesonthistypeofsegmentexceedsathreshold,thentheassociatedcell
becomesactiveinapredictedstate.Sincetherearemultipledendritesegmentsper
cell,acellspredictivestateisthelogicalORoperationofseveralconstituent
thresholddetectors.
PotentialSynapses
Theconceptofpotentialsynapsesisalsoborrowedfromneuroscience.Itrefersto
thefactthatnotallconnectionsarephysicallypossible.Thereforeeachdendrite
segmenthasalistofpotentialsynapses.Allthepotentialsynapsesaregivena
permanencevalueandmaybecomefunctionalsynapsesiftheirpermanencevalues
exceedathreshold.
Learning
Learninginvolvesincrementingordecrementingthepermanencevaluesof
potentialsynapsesonadendritesegment.Therulesusedformakingsynapses
moreorlesspermanentaresimilartoHebbianlearningrules.Forexample,ifa
postsynapticcellisactiveduetoadendritesegmentreceivinginputaboveits
threshold,thenthepermanencevaluesofthesynapsesonthatsegmentare
modified.Synapsesthatareactive,andthereforecontributedtothecellbeing
active,havetheirpermanenceincreased.Synapsesthatareinactive,andtherefore
didnotcontribute,havetheirpermanencedecreased.Theexactconditionsunder
whichsynapsepermanencevaluesareupdateddifferinthespatialandtemporal
pooler.Thedetailsaredescribedbelow.
Nowwewilldiscussconceptsspecifictothespatialandtemporalpoolerfunctions.
Spatialpoolerconcepts
Themostfundamentalfunctionofthespatialpooleristoconvertaregionsinput
intoasparsepattern.Thisfunctionisimportantbecausethemechanismusedto
learnsequencesandmakepredictionsrequiresstartingwithsparsedistributed
patterns.
Numenta2010 Page27
Thereareseveraloverlappinggoalsforthespatialpooler,whichdeterminehowthe
spatialpooleroperatesandlearns.
1)Useallcolumns
AnHTMregionhasafixednumberofcolumnsthatlearntorepresentcommon
patternsintheinput.Oneobjectiveistomakesureallthecolumnslearnto
representsomethingusefulregardlessofhowmanycolumnsyouhave.Wedont
wantcolumnsthatareneveractive.Topreventthisfromhappening,wekeeptrack
ofhowoftenacolumnisactiverelativetoitsneighbors.Iftherelativeactivityofa
columnistoolow,itboostsitsinputactivityleveluntilitstartstobepartofthe
winningsetofcolumns.Inessence,allcolumnsarecompetingwiththeirneighbors
tobeaparticipantinrepresentinginputpatterns.Ifacolumnisnotveryactive,it
willbecomemoreaggressive.Whenitdoes,othercolumnswillbeforcedtomodify
theirinputandstartrepresentingslightlydifferentinputpatterns.
2)Maintaindesireddensity
Aregionneedstoformasparserepresentationofitsinputs.Columnswiththemost
inputinhibittheirneighbors.Thereisaradiusofinhibitionwhichisproportionalto
thesizeofthereceptivefieldsofthecolumns(andthereforecanrangefromsmallto
thesizeoftheentireregion).Withintheradiusofinhibition,weallowonlya
percentageofthecolumnswiththemostactiveinputtobewinners.The
remaindersofthecolumnsaredisabled.(Aradiusofinhibitionimpliesa2D
arrangementofcolumns,buttheconceptcanbeadaptedtoothertopologies.)
3)Avoidtrivialpatterns
Wewantallourcolumnstorepresentnontrivialpatternsintheinput.Thisgoal
canbeachievedbysettingaminimumthresholdofinputforthecolumntobeactive.
Forexample,ifwesetthethresholdto50,itmeansthatacolumnmusthavealeast
50activesynapsesonitsdendritesegmenttobeactive,guaranteeingacertainlevel
ofcomplexitytothepatternitrepresents.
4)Avoidextraconnections
Ifwearentcareful,acolumncouldformalargenumberofvalidsynapses.Itwould
thenrespondstronglytomanydifferentunrelatedinputpatterns.Differentsubsets
ofthesynapseswouldrespondtodifferentpatterns.Toavoidthisproblem,we
decrementthepermanencevalueofanysynapsethatisntcurrentlycontributingto
awinningcolumn.Bymakingsurenoncontributingsynapsesaresufficiently
penalized,weguaranteeacolumnrepresentsalimitednumberinputpatterns,
sometimesonlyone.
5)Selfadjustingreceptivefields
Realbrainsarehighlyplastic;regionsoftheneocortexcanlearntorepresent
entirelydifferentthingsinreactiontovariouschanges.Ifpartoftheneocortexis
damaged,otherpartswilladjusttorepresentwhatthedamagedpartusedto
represent.Ifasensoryorganisdamagedorchanged,theassociatedpartofthe
neocortexwilladjusttorepresentsomethingelse.Thesystemisselfadjusting.
Numenta2010 Page28
WewantourHTMregionstoexhibitthesameflexibility.Ifweallocate10,000
columnstoaregion,itshouldlearnhowtobestrepresenttheinputwith10,000
columns.Ifweallocate20,000columns,itshouldlearnhowbesttousethat
number.Iftheinputstatisticschange,thecolumnsshouldchangetobestrepresent
thenewreality.Inshort,thedesignerofanHTMshouldbeabletoallocateany
resourcestoaregionandtheregionwilldothebestjobitcanofrepresentingthe
inputbasedontheavailablecolumnsandinputstatistics.Thegeneralruleisthat
withmorecolumnsinaregion,eachcolumnwillrepresentlargerandmoredetailed
patternsintheinput.Typicallythecolumnsalsowillbeactivelessoften,yetwewill
maintainarelativeconstantsparsitylevel.
Nonewlearningrulesarerequiredtoachievethishighlydesirablegoal.By
boostinginactivecolumns,inhibitingneighboringcolumnstomaintainconstant
sparsity,establishingminimalthresholdsforinput,maintainingalargepoolof
potentialsynapses,andaddingandforgettingsynapsesbasedontheircontribution,
theensembleofcolumnswilldynamicallyconfiguretoachievethedesiredeffect.
Spatialpoolerdetails
Wecannowgothrougheverythingthespatialpoolingfunctiondoes.
1)Startwithaninputconsistingofafixednumberofinputbits.Thesebitsmight
representsensorydataortheymightcomefromanotherregionlowerinthe
hierarchy.
2)Assignafixednumberofcolumnstotheregionreceivingthisinput.Eachcolumn
hasanassociateddendritesegment.Eachdendritesegmenthasasetofpotential
synapsesrepresentingasubsetoftheinputbits.Eachpotentialsynapsehasa
permanencevalue.Basedontheirpermanencevalues,someofthepotential
synapseswillbevalid.
3)Foranygiveninput,determinehowmanyvalidsynapsesoneachcolumnare
connectedtoactiveinputbits.
4)Thenumberofactivesynapsesismultipliedbyaboostingfactorwhichis
dynamicallydeterminedbyhowoftenacolumnisactiverelativetoitsneighbors.
5)Thecolumnswiththehighestboostedinputdisableallbutafixedpercentageof
thecolumnswithinaninhibitionradius.Theinhibitionradiusisitselfdynamically
determinedbythespread(orfanout)ofinputbits.Thereisnowasparsesetof
activecolumns.
Numenta2010 Page29
6)Foreachoftheactivecolumns,weadjustthepermanencevaluesofallthe
potentialsynapses.Thepermanencevaluesofsynapsesalignedwithactiveinput
bitsareincreased.Thepermanencevaluesofsynapsesalignedwithinactiveinput
bitsaredecreased.Thechangesmadetopermanencevaluesmaychangesome
synapsesfrombeingvalidtonotvalid,andviceversa.
Temporalpoolerconcepts
Recallthatthetemporalpoolerlearnssequencesandmakespredictions.Thebasic
methodisthatwhenacellbecomesactive,itformsconnectionstoothercellsthat
wereactivejustprior.Cellscanthenpredictwhentheywillbecomeactiveby
lookingattheirconnections.Ifallthecellsdothis,collectivelytheycanstoreand
recallsequences,andtheycanpredictwhatislikelytohappennext.Thereisno
centralstorageforasequenceofpatterns;instead,memoryisdistributedamongthe
individualcells.Becausethememoryisdistributed,thesystemisrobusttonoise
anderror.Individualcellscanfail,usuallywithlittleornodiscernibleeffect.
Itisworthnotingafewimportantpropertiesofsparsedistributedrepresentations
thatthetemporalpoolerexploits.
Assumewehaveahypotheticalregionthatalwaysformsrepresentationsbyusing
200activecellsoutofatotalof10,000cells(2%ofthecellsareactiveatanytime).
Howcanwerememberandrecognizeaparticularpatternof200activecells?A
simplewaytodothisistomakealistofthe200activecellswecareabout.Ifwesee
thesame200cellsactiveagainwerecognizethepattern.However,whatifwemade
alistofonly20ofthe200activecellsandignoredtheother180?Whatwould
happen?Youmightthinkthatrememberingonly20cellswouldcauselotsoferrors,
thatthose20cellswouldbeactiveinmanydifferentpatternsof200.Butthisisnt
thecase.Becausethepatternsarelargeandsparse(inthisexample200activecells
outof10,000),remembering20activecellsisalmostasgoodasrememberingall
200.Thechanceforerrorinapracticalsystemisexceedinglysmall.
ThecellsinanHTMregiontakeadvantageofthisproperty.Eachofacellsdendrite
segmentshasasetofconnectionstoothercellsintheregion.Adendritesegment
formstheseconnectionsasameansofrecognizingthestateofthenetworkatsome
pointintime.Theremaybehundredsorthousandsofactivecellsnearbybutthe
dendritesegmentonlyhastoconnectto15or20ofthem.Whenthedendrite
segmentsees15ofthoseactivecells,itcanbefairlycertainthelargerpatternis
occurring.ThistechniqueiscalledsubsamplingandisusedthroughouttheHTM
algorithms.
Everycellparticipatesinmanydifferentdistributedpatternsandinmanydifferent
sequences.Aparticularcellmightbepartofdozensorhundredsoftemporal
transitions.Thereforeeverycellhasseveraldendritesegments,notjustone.
Numenta2010 Page30
Ideallyacellwouldhaveonedendritesegmentforeachpatternofactivityitwants
torecognize.Practicallythough,adendritesegmentcanlearnconnectionsfor
severalcompletelydifferentpatternsandstillworkwell.Forexample,onesegment
mightlearn20connectionsforeachof4differentpatterns,foratotalof80
connections.Wesetathresholdsothedendritesegmentbecomesactivewhenany
15ofitsconnectionsareactive.Thisintroducesthepossibilityforerroras
connectionsfromdifferentpatternsmightincorrectlycombinetoexceedthe
threshold.However,thiskindoferrorisveryunlikely,againduetothesparseness
oftherepresentations.
Nowwecanseehowacellwithoneortwodozendendritesegmentsandafew
thousandsynapsescanrecognizehundredsofseparatestatesofcellactivity.
Temporalpoolerdetails
Hereweenumeratethestepsperformedbythetemporalpooler.Westartwhere
thespatialpoolerleftoff,withasetofactivecolumnsrepresentingthefeedforward
input.
1)Foreachactivecolumn,checkforcellsinthecolumnthatareinapredictivestate,
andactivatethem.Ifnocellsareinapredictivestate,activateallthecellsinthe
column.Theresultingsetofactivecellsistherepresentationoftheinputinthe
contextofpriorinput.
2)Foreverydendritesegmentoneverycellintheregion,counthowmany
establishedsynapsesareconnectedtoactivecells.Ifthenumberexceedsa
threshold,thatdendritesegmentismarkedasactive.Cellswithactivedendrite
segmentsareputinthepredictivestateunlesstheyarealreadyactiveduetofeed
forwardinput.Cellswithnoactivedendritesandnotactiveduetobottomupinput
becomeorremaininactive.Thecollectionofcellsnowinthepredictivestateisthe
predictionoftheregion.
3)Whenadendritesegmentbecomesactive,modifythepermanencevaluesofall
thesynapsesassociatedwiththesegment.Foreverypotentialsynapseontheactive
dendritesegment,increasethepermanenceofthosesynapsesthatareconnectedto
activecellsanddecrementthepermanenceofthosesynapsesconnectedtoinactive
cells.Thesechangestosynapsepermanencearemarkedastemporary.
Thismodifiesthesynapsesonsegmentsthatarealreadytrainedsufficientlytomake
thesegmentactive,andthusleadtoaprediction.However,wealwayswantto
extendpredictionsfurtherbackintimeifpossible.Thus,wepickaseconddendrite
segmentonthesamecelltotrain.Forthesecondsegmentwechoosetheonethat
bestmatchesthestateofthesystemintheprevioustimestep.Forthissegment,
usingthestateofthesystemintheprevioustimestep,increasethepermanenceof
Numenta2010 Page31
thosesynapsesthatareconnectedtoactivecellsanddecrementthepermanenceof
thosesynapsesconnectedtoinactivecells.Thesechangestosynapsepermanence
aremarkedastemporary.
4)Wheneveracellswitchesfrombeinginactivetoactiveduetofeedforwardinput,
wetraverseeachpotentialsynapseassociatedwiththecellandremoveany
temporarymarks.Thusweupdatethepermanenceofsynapsesonlyifthey
correctlypredictedthefeedforwardactivationofthecell.
5)Whenacellswitchesfromeitheractivestatetoinactive,undoanypermanence
changesmarkedastemporaryforeachpotentialsynapseonthiscell.Wedontwant
tostrengthenthepermanenceofsynapsesthatincorrectlypredictedthefeed
forwardactivationofacell.
Notethatonlycellsthatareactiveduetofeedforwardinputpropagateactivity
withintheregion,otherwisepredictionswouldleadtofurtherpredictions.Butall
theactivecells(feedforwardandpredictive)formtheoutputofaregionand
propagatetothenextregioninthehierarchy.
Firstorderversusvariableordersequencesandprediction

Thereisonemoremajortopictodiscussbeforeweendourdiscussiononthespatial
andtemporalpoolers.Itmaynotbeofinteresttoallreadersanditisnotneededto
understandChapters3and4.
Whatistheeffectofhavingmoreorfewercellspercolumn?Specifically,what
happensifwehaveonlyonecellpercolumn?Intheexampleusedearlier,we
showedthatarepresentationcomprisedof100activecolumnswith4cellsper
columncanbeencodedin4^100differentways,creatingaverybignumber.
Thereforethesameinputcanbeusedinagreatmanydifferentcontexts.Ifeach
inputpatternrepresentedaword,thenaregioncouldremembermanysentences
thatusethesamewordsoverandoveragainandnotgetconfused.Alternatelywe
couldusethesamenumberofwordstolearnfewerbutlongersentences.Thiskind
ofmemoryiscalledvariableorder,meaningthattheamountofpriorcontext
neededtopredictthenextinputvaries.AnHTMregionisavariableordermemory.
Ifweincreasetofivecellspercolumn,theavailablenumberofencodingsofany
particularinputinourexamplewouldincreaseto5^100,ahugeincreaseover
4^100.Butboththesenumbersaresolargethatformanypracticalproblemsthe
increaseincapacitymightnotbeuseful.
However,makingthenumberofcellspercolumnmuchsmallerdoesmakeabig
difference.
Numenta2010 Page32
Ifwegoallthewaytoonecellpercolumn,welosetheabilitytoincludecontextin
ourrepresentations.Aninputtoaregionalwaysresultsinthesameprediction,
regardlessofpreviousactivity.Withonecellpercolumn,thememoryofanHTM
regionisafirstordermemory;predictionsarebasedonlyonthecurrentinput.
Firstorderpredictionisideallysuitedforonetypeofproblemthatbrainssolve:
staticspatialinference.Asstatedearlier,ahumanexposedtoabriefvisualimage
canrecognizewhattheobjectiseveniftheexposureistooshortfortheeyesto
move.Withhearing,youalwaysneedtohearasequenceofpatternstorecognize
whatitis.Visionisusuallylikethat,youusuallyprocessastreamofvisualimages.
Butundercertainconditionsyoucanrecognizeanimagewithasingleexposure.
Temporalandstaticrecognitionmightappeartorequiredifferentinference
mechanisms.Onerequiresrecognizingsequencesofpatternsandmaking
predictionsbasedonvariablelengthcontext.Theotherrequiresrecognizinga
staticspatialpatternwithoutusingtemporalcontext.AnHTMregionwithmultiple
cellspercolumnisideallysuitedforrecognizingtimebasedsequences,andanHTM
regionwithonecellpercolumnisideallysuitedtorecognizingspatialpatterns.At
Numenta,wehaveperformedmanyexperimentsusingonecellpercolumnregions
appliedtovisionproblems.Thedetailsoftheseexperimentsarebeyondthescope
ofthischapter;howeverwewillcovertheimportantconcepts.
IfweexposeanHTMregiontoimages,thecolumnsintheregionlearntorepresent
commonspatialarrangementsofpixels.Thekindofpatternslearnedaresimilarto
whatisobservedinregionV1inneocortex(aneocorticalregionextensivelystudied
inbiology),typicallylinesandcornersatdifferentorientations.Bytrainingon
movingimages,theHTMregionlearnstransitionsofthesebasicshapes.For
example,averticallineatonepositionisoftenfollowedbyaverticallineshiftedto
theleftorright.Allthecommonlyobservedtransitionsofpatternsareremembered
bytheHTMregion.
Nowwhathappensifweexposearegiontoanimageofaverticallinemovingtothe
right?Ifourregionhasonlyonecellpercolumn,itwillpredictthelinemightnext
appeartotheleftortotheright.Itcantusethecontextofknowingwheretheline
wasinthepastandthereforeknowifitismovingleftorright.Whatyoufindisthat
theseonecellpercolumncellsbehavelikesimplecellsintheneocortex.The
predictiveoutputofsuchacellwillbeactiveforavisiblelineindifferentpositions,
regardlessofwhetherthelineismovingleftorrightornotatall.Wehavefurther
observedthataregionlikethisexhibitsstabilitytotranslation,changesinscale,etc.
whilemaintainingtheabilitytodistinguishbetweendifferentimages.Thisbehavior
iswhatisneededforspatialinvariance(recognizingthesamepatternindifferent
locationsofanimage).
IfwenowdothesameexperimentonanHTMregionwithmultiplecellspercolumn,
wefindthatthecellsbehavelikecomplexcellsintheneocortex.Thepredictive
Numenta2010 Page33
outputofacellwillbeactiveforalinemovingtotheleftoralinemovingtothe
right,butnotboth.
Puttingthisalltogether,wemakethefollowinghypothesis.Theneocortexhastodo
bothfirstorderandvariableorderinferenceandprediction.Therearefourorfive
layersofcellsineachregionoftheneocortex.Thelayersdifferinseveralwaysbut
theyallhavesharedcolumnarresponsepropertiesandlargehorizontalconnectivity
withinthelayer.Wespeculatethateachlayerofcellsinneocortexisperforminga
variationoftheHTMinferenceandlearningrulesdescribedinthischapter.The
differentlayersofcellsplaydifferentroles.Forexampleitisknownfrom
anatomicalstudiesthatlayer6createsfeedbackinthehierarchyandlayer5is
involvedinmotorbehavior.Thetwoprimaryfeedforwardlayersofcellsarelayers
4and3.Wespeculatethatoneofthedifferencesbetweenlayers4and3isthatthe
cellsinlayer4areactingindependently,i.e.onecellpercolumn,whereasthecellsin
layer3areactingasmultiplecellspercolumn.Thusregionsintheneocortexnear
sensoryinputhavebothfirstorderandvariableordermemory.Thefirstorder
sequencememory(roughlycorrespondingtolayer4neurons)isusefulinforming
representationsthatareinvarianttospatialchanges.Thevariableordersequence
memory(roughlycorrespondingtolayer3neurons)isusefulforinferenceand
predictionofmovingimages.
Insummary,wehypothesizethatthealgorithmssimilartothosedescribedinthis
chapterareatworkinalllayersofneuronsintheneocortex.Thelayersinthe
neocortexvaryinsignificantdetailswhichmakethemplaydifferentrolesrelatedto
feedforwardvs.feedback,attention,andmotorbehavior.Inregionscloseto
sensoryinput,itisusefultohavealayerofneuronsperformingfirstordermemory
asthisleadstospatialinvariance.
AtNumenta,wehaveexperimentedwithfirstorder(singlecellpercolumn)HTM
regionsforimagerecognitionproblems.Wealsohaveexperimentedwithvariable
order(multiplecellspercolumn)HTMregionsforrecognizingandpredicting
variableordersequences.Inthefuture,itwouldbelogicaltotrytocombinethese
inasingleregionandtoextendthealgorithmstootherpurposes.However,we
believemanyinterestingproblemscanbeaddressedwiththeequivalentofsingle
layer,multiplecellpercolumnregions,eitheraloneorinahierarchy.
Numenta2010 Page34
Chapter3:SpatialPoolingImplementationandPseudocode

Thischaptercontainsthedetailedpseudocodeforafirstimplementationofthe
spatialpoolerfunction.Theinputtothiscodeisanarrayofbottomupbinary
inputsfromsensorydataorthepreviouslevel.Thecodecomputes
activeColumns(t)thelistofcolumnsthatwinduetothebottomupinputattimet.
Thislististhensentasinputtothetemporalpoolerroutinedescribedinthenext
chapter,i.e.activeColumns(t)istheoutputofthespatialpoolingroutine.
Thepseudocodeissplitintothreedistinctphasesthatoccurinsequence:
Phase1:computetheoverlapwiththecurrentinputforeachcolumn
Phase2:computethewinningcolumnsafterinhibition
Phase3:updatesynapsepermanenceandinternalvariables
Althoughspatialpoolerlearningisinherentlyonline,youcanturnofflearningby
simplyskippingPhase3.
Therestofthechaptercontainsthepseudocodeforeachofthethreesteps.The
variousdatastructuresandsupportingroutinesusedinthecodearedefinedatthe
end.

Initialization
Priortoreceivinganyinputs,theregionisinitializedbycomputingalistofinitial
potentialsynapsesforeachcolumn.Thisconsistsofarandomsetofinputsselected
fromtheinputspace.Eachinputisrepresentedbyasynapseandassigneda
randompermanencevalue.Therandompermanencevaluesarechosenwithtwo
criteria.First,thevaluesarechosentobeinasmallrangearoundconnectedPerm
(theminimumpermanencevalueatwhichasynapseisconsidered"connected").
Thisenablespotentialsynapsestobecomeconnected(ordisconnected)aftera
smallnumberoftrainingiterations.Second,eachcolumnhasanaturalcenterover
theinputregion,andthepermanencevalueshaveabiastowardsthiscenter(they
havehighervaluesnearthecenter).

Numenta2010 Page35
Phase1:Overlap
Givenaninputvector,thefirstphasecalculatestheoverlapofeachcolumnwiththat
vector.Theoverlapforeachcolumnissimplythenumberofconnectedsynapses
withactiveinputs,multipliedbyitsboost.IfthisvalueisbelowminOverlap,weset
theoverlapscoretozero.
1. for c in columns
2.
3. overlap(c) = 0
4. for s in connectedSynapses(c)
5. overlap(c) = overlap(c) + input(t, s.sourceInput)
6.
7. if overlap(c) < minOverlap then
8. overlap(c) = 0
9. else
10. overlap(c) = overlap(c) * boost(c)

Phase2:Inhibition
Thesecondphasecalculateswhichcolumnsremainaswinnersaftertheinhibition
step.desiredLocalActivityisaparameterthatcontrolsthenumberofcolumnsthat
endupwinning.Forexample,ifdesiredLocalActivityis10,acolumnwillbea
winnerifitsoverlapscoreisgreaterthanthescoreofthe10'thhighestcolumn
withinitsinhibitionradius.
11. for c in columns

12.
13. minLocalActivity = kthScore(neighbors(c), desiredLocalActivity)
14.
15. if overlap(c) > 0 and overlap(c) < minLocalActivity then
16. activeColumns(t).append(c)
17.

Numenta2010 Page36
Phase3:Learning
Thethirdphaseperformslearning;itupdatesthepermanencevaluesofallsynapses
asnecessary,aswellastheboostandinhibitionradius.
Themainlearningruleisimplementedinlines2026.Forwinningcolumns,ifa
synapseisactive,itspermanencevalueisincremented,otherwiseitisdecremented.
Permanencevaluesareconstrainedtobebetween0and1.
Lines2836implementboosting.Therearetwoseparateboostingmechanismsin
placetohelpacolumnlearnconnections.Ifacolumndoesnotwinoftenenough(as
measuredbyactiveDutyCycle),itsoverallboostvalueisincreased(line3032).
Alternatively,ifacolumn'sconnectedsynapsesdonotoverlapwellwithanyinputs
oftenenough(asmeasuredbyoverlapDutyCycle),itspermanencevaluesare
boosted(line3436).Note:oncelearningisturnedoff,boost(c)isfrozen.
Finally,attheendofPhase3theinhibitionradiusisrecomputed(line38).
18. for c in activeColumns(t)

19.
20. for s in potentialSynapses(c)
21. if active(s) then
22. s.permanence += permanenceInc
23. s.permanence = min(1.0, s.permanence)
24. else
25. s.permanence -= permanenceDec
26. s.permanence = max(0.0, s.permanence)
27.
28. for c in columns:
29.
30. minDutyCycle(c) = 0.01 * maxDutyCycle(neighbors(c))
31. activeDutyCycle(c) = updateActiveDutyCycle(c)
32. boost(c) = boostFunction(activeDutyCycle(c), minDutyCycle(c))
33.
34. overlapDutyCycle(c) = updateOverlapDutyCycle(c)
35. if overlapDutyCycle(c) < minDutyCycle(c) then
36. increasePermanences(c, 0.1*connectedPerm)
37.
38. inhibitionRadius = averageReceptiveFieldSize()
39.

Numenta2010 Page37
Supportingdatastructuresandroutines
Thefollowingvariablesanddatastructuresareusedinthepseudocode:

columns Listofallcolumns.
input(t,j) Theinputtothislevelattimet.input(t,j)is1ifthej'th
inputison.
overlap(c) Thespatialpooleroverlapofcolumncwithaparticular
inputpattern.
activeColumns(t) Listofcolumnindicesthatarewinnersduetobottomup
input.
desiredLocalActivity Aparametercontrollingthenumberofcolumnsthatwillbe
winnersaftertheinhibitionstep.
inhibitionRadius Averageconnectedreceptivefieldsizeofthecolumns.
neighbors(c) AlistofallthecolumnsthatarewithininhibitionRadiusof
columnc.
minOverlap Aminimumnumberofinputsthatmustbeactivefora
columntobeconsideredduringtheinhibitionstep.
boost(c) Theboostvalueforcolumncascomputedduringlearning
usedtoincreasetheoverlapvalueforinactivecolumns.
synapse Adatastructurerepresentingasynapsecontainsa
permanencevalueandthesourceinputindex.
connectedPerm Ifthepermanencevalueforasynapseisgreaterthanthis
value,itissaidtobeconnected.
potentialSynapses(c) Thelistofpotentialsynapsesandtheirpermanencevalues.
connectedSynapses(c) AsubsetofpotentialSynapses(c)wherethepermanence
valueisgreaterthanconnectedPerm.Thesearethe
bottomupinputsthatarecurrentlyconnectedtocolumnc.
permanenceInc Amountpermanencevaluesofsynapsesareincremented
duringlearning.
permanenceDec Amountpermanencevaluesofsynapsesaredecremented
duringlearning.
activeDutyCycle(c) Aslidingaveragerepresentinghowoftencolumnchas
beenactiveafterinhibition(e.g.overthelast1000
iterations).
Numenta2010 Page38
overlapDutyCycle(c) Aslidingaveragerepresentinghowoftencolumnchashad
significantoverlap(i.e.greaterthanminOverlap)withits
inputs(e.g.overthelast1000iterations).
minDutyCycle(c) Avariablerepresentingtheminimumdesiredfiringratefor
acell.Ifacell'sfiringratefallsbelowthisvalue,itwillbe
boosted.Thisvalueiscalculatedas1%ofthemaximum
firingrateofitsneighbors.

Thefollowingsupportingroutinesareusedintheabovecode.

kthScore(cols, k)
Giventhelistofcolumns,returnthek'thhighestoverlapvalue.

updateActiveDutyCycle(c)
Computesamovingaverageofhowoftencolumnchasbeenactiveafter
inhibition.

updateOverlapDutyCycle(c)
Computesamovingaverageofhowoftencolumnchasoverlapgreater
thanminOverlap.

averageReceptiveFieldSize()
Theradiusoftheaverageconnectedreceptivefieldsizeofallthecolumns.
Theconnectedreceptivefieldsizeofacolumnincludesonlytheconnected
synapses(thosewithpermanencevalues>=connectedPerm).Thisisused
todeterminetheextentoflateralinhibitionbetweencolumns.

maxDutyCycle(cols)
Returnsthemaximumactivedutycycleofthecolumnsinthegivenlistof
columns.

increasePermanences(c, s)
Increasethepermanencevalueofeverysynapseincolumncbyascale
factors.

boostFunction(c)
Returnstheboostvalueofacolumn.Theboostvalueisascalar>=1.If
activeDutyCyle(c)isaboveminDutyCycle(c),theboostvalueis1.The
boostincreaseslinearlyoncethecolumn'sactiveDutyCylestartsfalling
belowitsminDutyCycle.
Numenta2010 Page39
Chapter4:TemporalPoolingImplementationandPseudocode

Thischaptercontainsthedetailedpseudocodeforafirstimplementationofthe
temporalpoolerfunction.TheinputtothiscodeisactiveColumns(t),ascomputed
bythespatialpooler.Thecodecomputestheactiveandpredictivestateforeach
cellatthecurrenttimestep,t.ThebooleanORoftheactiveandpredictivestatesfor
eachcellformstheoutputofthetemporalpoolerforthenextlevel.
Thepseudocodeissplitintothreedistinctphasesthatoccurinsequence:
Phase1:computetheactivestate,activeState(t),foreachcell
Phase2:computethepredictedstate,predictiveState(t),foreachcell
Phase3:updatesynapses
Phase3isonlyrequiredforlearning.However,unlikespatialpooling,Phases1and
2containsomelearningspecificoperationswhenlearningisturnedon.Since
temporalpoolingissignificantlymorecomplicatedthanspatialpooling,wefirstlist
theinferenceonlyversionofthetemporalpooler,followedbyaversionthat
combinesinferenceandlearning.Adescriptionofsomeoftheimplementation
details,terminology,andsupportingroutinesareattheendofthechapter,afterthe
pseudocode.

Numenta2010 Page40
Temporalpoolerpseudocode:inferencealone

Phase1

Thefirstphasecalculatestheactivestateforeachcell.Foreachwinningcolumnwe
determinewhichcellsshouldbecomeactive.Ifthebottomupinputwaspredicted
byanycell(i.e.itspredictiveStatewas1duetoasequencesegmentintheprevious
timestep),thenthosecellsbecomeactive(lines49).Ifthebottomupinputwas
unexpected(i.e.nocellshadpredictiveStateoutputon),theneachcellinthecolumn
becomesactive(lines1113).

2.
3. buPredicted = false
4. for i = 0 to cellsPerColumn - 1
5. if predictiveState(c, i, t-1) == true then
6. s = getActiveSegment(c, i, t-1, activeState)
7. if s.sequenceSegment == true then
8. buPredicted = true
9. activeState(c, i, t) = 1
10.
11. if buPredicted == false then
Phase2
Thesecondphasecalculatesthepredictivestateforeachcell.Acellwillturnonits
predictiveStateifanyoneofitssegmentsbecomesactive,i.e.ifenoughofits
horizontalconnectionsarecurrentlyfiringduetofeedforwardinput.
14. for c, i in cells

15. for s in segments(c, i)
16. if segmentActive(c, i, s, t) then
17. predictiveState(c, i, t) = 1

Numenta2010 Page41
Temporalpoolerpseudocode:combinedinferenceandlearning

Phase1
ThefirstphasecalculatestheactiveStateforeachcellthatisinawinningcolumn.
Forthosecolumns,thecodefurtherselectsonecellpercolumnasthelearningcell
(learnState).Thelogicisasfollows:ifthebottomupinputwaspredictedbyanycell
(i.e.itspredictiveStateoutputwas1duetoasequencesegment),thenthosecells
becomeactive(lines510).Ifthatsegmentbecameactivefromcellschosenwith
learnStateon,thiscellisselectedasthelearningcell(lines1113).Ifthebottomup
inputwasnotpredicted,thenallcellsinthebecomeactive(lines1517).In
addition,thebestmatchingcellischosenasthelearningcell(lines1924)andanew
segmentisaddedtothatcell.

19.
20. buPredicted = false
21. lcChosen = false
23. if predictiveState(c, i, t-1) == true then
24. s = getActiveSegment(c, i, t-1, activeState)
25. if s.sequenceSegment == true then
26. buPredicted = true
28. if segmentActive(s, t-1, learnState) then
29. lcChosen = true
30. learnState(c, i, t) = 1
31.
32. if buPredicted == false then
35.
36. if lcChosen == false then
37. i = getBestMatchingCell(c, t-1)
38. learnState(c, i, t) = 1
39. sUpdate = getSegmentActiveSynapses (c, i, -1, t-1, true)
40. sUpdate.sequenceSegment = true
41. segmentUpdateList.add(sUpdate)

Numenta2010 Page42
Phase2
Thesecondphasecalculatesthepredictivestateforeachcell.Acellwillturnonits
predictivestateoutputifoneofitssegmentsbecomesactive,i.e.ifenoughofits
lateralinputsarecurrentlyactiveduetofeedforwardinput.Inthiscase,thecell
queuesupthefollowingchanges:a)reinforcementofthecurrentlyactivesegment
(lines3031),andb)reinforcementofasegmentthatcouldhavepredictedthis
activation,i.e.asegmentthathasa(potentiallyweak)matchtoactivityduringthe
previoustimestep(lines3336).

43. for s in segments(c, i)
44. if segmentActive(s, t, activeState) then
45. predictiveState(c, i, t) = 1
46.
47. activeUpdate = getSegmentActiveSynapses (c, i, s, t, false)
48. segmentUpdateList.add(activeUpdate)
49.
50. predSegment = getBestMatchingSegment(c, i, t-1)
51. predUpdate = getSegmentActiveSynapses(
52. c, i, predSegment, t-1, true)
53. segmentUpdateList.add(predUpdate)
Phase3
Thethirdandlastphaseactuallycarriesoutlearning.Inthisphasesegment
updatesthathavebeenqueuedupareactuallyimplementedoncewegetfeed
forwardinputandthecellischosenasalearningcell(lines3840).Otherwise,ifthe
celleverstopspredictingforanyreason,wenegativelyreinforcethesegments
(lines4143).

55. if learnState(s, i, t) == 1 then
56. adaptSegments (segmentUpdateList(c, i), true)
57. segmentUpdateList(c, i).delete()
58. else if predictiveState(c, i, t) == 0 and predictiveState(c, i, t-1)==1 then
59. adaptSegments (segmentUpdateList(c,i), false)
60. segmentUpdateList(c, i).delete()
61.

Numenta2010 Page43
Implementationdetailsandterminology
Inthissectionwedescribesomeofthedetailsofourtemporalpooler
implementationandterminology.Eachcellisindexedusingtwonumbers:a
columnindex,c,andacellindex,i.Cellsmaintainalistofdendritesegments,where
eachsegmentcontainsalistofsynapsesplusapermanencevalueforeachsynapse.
Changestoacell'ssynapsesaremarkedastemporaryuntilthecellbecomesactive
fromfeedforwardinput.Thesetemporarychangesaremaintainedin
segmentUpdateList.Eachsegmentalsomaintainsabooleanflag,sequenceSegment,
indicatingwhetherthesegmentpredictsfeedforwardinputonthenexttimestep.
Theimplementationofpotentialsynapsesisdifferentfromtheimplementationin
thespatialpooler.Inthespatialpooler,thecompletelistofpotentialsynapsesis
representedasanexplicitlist.Inthetemporalpooler,eachsegmentcanhaveits
own(possiblylarge)listofpotentialsynapses.Inpracticemaintainingalonglistfor
eachsegmentiscomputationallyexpensiveandmemoryintensive.Thereforeinthe
temporalpooler,werandomlyaddactivesynapsestoeachsegmentduringlearning
(controlledbytheparameternewSynapseCount).Thisoptimizationhasasimilar
effecttomaintainingthefulllistofpotentialsynapses,butthelistpersegmentisfar
smallerwhilestillmaintainingthepossibilityoflearningnewtemporalpatterns.
Thepseudocodealsousesasmallstatemachinetokeeptrackofthecellstatesat
differenttimesteps.Wemaintainthreedifferentstatesforeachcell.Thearrays
activeStateandpredictiveStatekeeptrackoftheactiveandpredictivestatesofeach
cellateachtimestep.ThearraylearnStatedetermineswhichcelloutputsareused
duringlearning.Whenaninputisunexpected,allthecellsinaparticularcolumn
becomeactiveinthesametimestep.Onlyoneofthesecells(thecellthatbest
matchestheinput)hasitslearnStateturnedon.Weonlyaddsynapsesfromcells
thathavelearnStatesettoone(thisavoidsoverrepresentingafullyactivecolumnin
dendriticsegments).
Numenta2010 Page44
Thefollowingdatastructuresareusedinthetemporalpoolerpseudocode:

cell(c,i) Alistofallcells,indexedbyiandc.
cellsPerColumn Numberofcellsineachcolumn.
activeColumns(t) Listofcolumnindicesthatarewinnersduetobottomup
input(thisistheoutputofthespatialpooler).
activeState(c, i, t) Abooleanvectorwithonenumberpercell.Itrepresentsthe
activestateofthecolumnccelliattimetgiventhecurrent
feedforwardinputandthepasttemporalcontext.
activeState(c,i,t)isthecontributionfromcolumnccelliat
timet.If1,thecellhascurrentfeedforwardinputaswellas
anappropriatetemporalcontext.
predictiveState(c, i, t) Abooleanvectorwithonenumberpercell.Itrepresentsthe
predictionofthecolumnccelliattimet,giventhebottomup
activityofothercolumnsandthepasttemporalcontext.
predictiveState(c,i,t)isthecontributionofcolumnccelliat
timet.If1,thecellispredictingfeedforwardinputinthe
currenttemporalcontext.
learnState(c, i, t) Abooleanindicatingwhethercelliincolumncischosenas
thecelltolearnon.
activationThreshold Activationthresholdforasegment.Ifthenumberofactive
connectedsynapsesinasegmentisgreaterthan
activationThreshold,thesegmentissaidtobeactive.
learningRadius Theareaaroundatemporalpoolercellfromwhichitcanget
lateralconnections.
initialPerm Initialpermanencevalueforasynapse.
connectedPerm Ifthepermanencevalueforasynapseisgreaterthanthis
value,itissaidtobeconnected.
minThreshold Minimumsegmentactivityforlearning.
newSynapseCount Themaximumnumberofsynapsesaddedtoasegmentduring
learning.
permanenceInc Amountpermanencevaluesofsynapsesareincremented
whenactivitybasedlearningoccurs.
permanenceDec Amountpermanencevaluesofsynapsesaredecremented
whenactivitybasedlearningoccurs.
Numenta2010 Page45
segmentUpdate Datastructureholdingthreepiecesofinformationrequiredto
updateagivensegment:a)segmentindex(1ifit'sanew
segment),b)alistofexistingactivesynapses,andc)aflag
indicatingwhetherthissegmentshouldbemarkedasa
sequencesegment(defaultstofalse).
segmentUpdateList AlistofsegmentUpdatestructures.segmentUpdateList(c,i)is
thelistofchangesforcelliincolumnc.

Thefollowingsupportingroutinesareusedintheabovecode:

segmentActive(s, t, state)
Thisroutinereturnstrueifthenumberofconnectedsynapsesonsegment
sthatareactiveduetothegivenstateattimetisgreaterthan
activationThreshold.TheparameterstatecanbeactiveState,or
learnState.

getActiveSegment(c, i, t, state)
Forthegivencolumnccelli,returnasegmentindexsuchthat
segmentActive(s,t,state)istrue.Ifmultiplesegmentsareactive,sequence
segmentsaregivenpreference.Otherwise,segmentswithmostactivity
aregivenpreference.

getBestMatchingSegment(c, i, t)
Forthegivencolumnccelliattimet,findthesegmentwiththelargest
numberofactivesynapses.Thisroutineisaggressiveinfindingthebest
match.Thepermanencevalueofsynapsesisallowedtobebelow
connectedPerm.Thenumberofactivesynapsesisallowedtobebelow
activationThreshold,butmustbeaboveminThreshold.Theroutine
returnsthesegmentindex.Ifnosegmentsarefound,thenanindexof1is
returned.

getBestMatchingCell(c)
Forthegivencolumn,returnthecellwiththebestmatchingsegment(as
definedabove).Ifnocellhasamatchingsegment,thenreturnthecellwith
thefewestnumberofsegments.

Numenta2010 Page46
getSegmentActiveSynapses(c, i, t, s, newSynapses= false)

ReturnasegmentUpdatedatastructurecontainingalistofproposed
changestosegments.LetactiveSynapsesbethelistofactivesynapses
wheretheoriginatingcellshavetheiractiveStateoutput=1attimestept.
(Thislistisemptyifs=1sincethesegmentdoesn'texist.)newSynapses
isanoptionalargumentthatdefaultstofalse.IfnewSynapsesistrue,then
newSynapseCountcount(activeSynapses)synapsesareaddedto
activeSynapses.Thesesynapsesarerandomlychosenfromthesetofcells
thathavelearnStateoutput=1attimestept.

adaptSegments(segmentList, positiveReinforcement)
ThisfunctioniteratesthroughalistofsegmentUpdate'sandreinforces
eachsegment.ForeachsegmentUpdateelement,thefollowingchangesare
performed.IfpositiveReinforcementistruethensynapsesontheactive
listgettheirpermanencecountsincrementedbypermanenceInc.Allother
synapsesgettheirpermanencecountsdecrementedbypermanenceDec.If
positiveReinforcementisfalse,thensynapsesontheactivelistgettheir
permanencecountsdecrementedbypermanenceDec.Afterthisstep,any
synapsesinsegmentUpdatethatdoyetexistgetaddedwithapermanence
countofinitialPerm.

Numenta2010 Page47
Glossary
Notes:Definitionsherecapturehowtermsareusedinthis
document,andmayhaveothermeaningsingeneraluse.
Capitalizedtermsrefertootherdefinedtermsinthis
glossary.
ActiveState astateinwhichCellsareactiveduetoFeedForward
input
BottomUp synonymtoFeedForward
Cells HTMequivalentofaNeuron
CellsareorganizedintocolumnsinHTMregions.
CoincidentActivity twoormoreCellsareactiveatthesametime
Column agroupofoneormoreCellsthatfunctionasaunit
inanHTMRegion
Cellswithinacolumnrepresentthesamefeedforward
input,butindifferentcontexts.
DendriteSegment aunitofintegrationofSynapsesassociatedwithCellsand
Columns
HTMshavetwodifferenttypesofdendritesegments.Oneis
associatedwithlateralconnectionstoacell.Whenthe
numberofactivesynapsesonthedendritesegmentexceeds
athreshold,theassociatedcellentersthepredictivestate.
Theotherisassociatedwithfeedforwardconnectionstoa
column.Thenumberofactivesynapsesissummedto
generatethefeedforwardactivationofacolumn.
DesiredDensity desiredpercentageofColumnsactiveduetoFeed
ForwardinputtoaRegion
Thepercentageonlyapplieswithinaradiusthatvaries
basedonthefanoutoffeedforwardinputs.Itisdesired
becausethepercentagevariessomebasedonthe
particularinput.

Numenta2010 Page48
FeedForward movinginadirectionawayfromaninput,orfroma
lowerLeveltoahigherLevelinaHierarchy(sometimes
calledBottomUp)
Feedback movinginadirectiontowardsaninput,orfromahigher
LeveltoalowerlevelinaHierarchy(sometimescalled
TopDown)
FirstOrderPrediction apredictionbasedonlyonthecurrentinputandnoton
thepriorinputscomparetoVariableOrderPrediction
HierarchicalTemporal
Memory(HTM)
atechnologythatreplicatessomeofthestructuraland
algorithmicfunctionsoftheneocortex
Hierarchy anetworkofconnectedelementswheretheconnections
betweentheelementsareuniquelyidentifiedasFeed
ForwardorFeedback
HTMCorticalLearning
Algorithms
thesuiteoffunctionsforSpatialPooling,Temporal
Pooling,andlearningandforgettingthatcomprisean
HTMRegion,alsoreferredtoasHTMLearning
Algorithms
HTMNetwork aHierarchyofHTMRegions
HTMRegion themainunitofmemoryandPredictioninanHTM
AnHTMregioniscomprisedofalayerofhighly
interconnectedcellsarrangedincolumns.AnHTMregion
todayhasasinglelayerofcells,whereasintheneocortex
(andultimatelyinHTM),aregionwillhavemultiplelayers
ofcells.Whenreferredtointhecontextofitspositionina
hierarchy,aregionmaybereferredtoasalevel.
Inference recognizingaspatialandtemporalinputpatternas
similartopreviouslylearnedpatterns
InhibitionRadius
definestheareaaroundaColumnthatitactivelyinhibits
LateralConnections connectionsbetweenCellswithinthesameRegion
Level anHTMRegioninthecontextoftheHierarchy

Numenta2010 Page49
Neuron aninformationprocessingCellinthebrain
Inthisdocument,weusethewordneuronspecificallywhen
referringtobiologicalcells,andcellwhenreferringtothe
HTMunitofcomputation.
Permanence ascalarvaluewhichindicatestheconnectionstateofa
PotentialSynapse
Apermanencevaluebelowathresholdindicatesthe
synapseisnotformed.Apermanencevalueabovethe
thresholdindicatesthesynapseisvalid.Learninginan
HTMregionisaccomplishedbymodifyingpermanence
valuesofpotentialsynapses.
PotentialSynapse thesubsetofallCellsthatcouldpotentiallyform
SynapseswithaparticularDendriteSegment
Onlyasubsetofpotentialsynapseswillbevalidsynapsesat
anytimebasedontheirpermanencevalue.
Prediction activatingCells(intoapredictivestate)thatwilllikely
becomeactiveinthenearfutureduetoFeedForward
input
AnHTMregionoftenpredictsmanypossiblefutureinputs
atthesametime.
ReceptiveField thesetofinputstowhichaColumnorCellisconnected
IftheinputtoanHTMregionisorganizedasa2Darrayof
bits,thenthereceptivefieldcanbeexpressedasaradius
withintheinputspace.
Sensor asourceofinputsforanHTMNetwork
SparseDistributed
Representation
representationcomprisedofmanybitsinwhichasmall
percentageareactiveandwherenosinglebitissufficient
toconveymeaning

Numenta2010 Page50
SpatialPooling theprocessofformingasparsedistributed
representationofaninput
Oneofthepropertiesofspatialpoolingisthatoverlapping
inputpatternsmaptothesamesparsedistributed
representation.
SubSampling recognizingalargedistributedpatternbymatchingonly
asmallsubsetoftheactivebitsinthelargepattern
Synapse connectionbetweenCellsformedwhilelearning
TemporalPooling theprocessofformingarepresentationofasequenceof
inputpatternswheretheresultingrepresentationis
morestablethantheinput
TopDown synonymforFeedback
VariableOrderPrediction apredictionbasedonvaryingamountsofpriorcontext
comparetoFirstOrderPrediction
Itiscalledvariablebecausethememorytomaintain
priorcontextisallocatedasneeded.Thusamemory
systemthatimplementsvariableorderpredictioncanuse
contextgoingwaybackintimewithoutrequiring
exponentialamountsofmemory.

Numenta2010 Page51
AppendixA:AComparisonbetweenBiologicalNeuronsand
HTMCells

Theimageaboveshowsapictureofabiologicalneuronontheleft,asimpleartificial
neuroninthemiddle,andanHTMneuronorcellontheright.Thepurposeofthis
appendixistoprovideabetterunderstandingofHTMcellsandhowtheyworkby
comparingthemtorealneuronsandsimplerartificialneurons.
Realneuronsaretremendouslycomplicatedandvaried.Wewillfocusonthemost
generalprinciplesandonlythosethatapplytoourmodel.Althoughweignore
manydetailsofrealneurons,thecellsusedintheHTMcorticallearningalgorithms
arefarmorerealisticthantheartificialneuronsusedinmostneuralnetworks.All
theelementsincludedinHTMcellsarenecessaryfortheoperationofanHTM
region.
Biologicalneurons

Neuronsaretheinformationcarryingcellsinthebrain.Theimageontheleftabove
isofatypicalexcitatoryneuron.Thevisualappearanceofaneuronisdominatedby
thebranchingdendrites.Alltheexcitatoryinputstoaneuronareviasynapses
alignedalongthedendrites.Inrecentyearsourknowledgeofneuronshas
advancedconsiderably.Thebiggestchangehasbeeninrealizingthatthedendrites
ofaneuronarenotjustconduitstobringinputstothecellbody.Wenowknowthe
dendritesarecomplexnonlinearprocessingelementsinthemselves.TheHTM
corticallearningalgorithmstakeadvantageofthesenonlinearproperties.
Numenta2010 Page52
Neuronshaveseveralparts.
Cellbody
Thecellbodyisthesmallvolumeinthecenteroftheneuron.Theoutputofthecell,
theaxon,originatesatthecellbody.Theinputstothecellarethesynapsesaligned
alongthedendriteswhichfeedtothecellbody.
ProximalDendrites
Thedendritebranchesclosesttothecellbodyarecalledproximaldendrites.Inthe
diagramsomeoftheproximaldendritesaremarkedwithgreenlines.
Multipleactivesynapsesonproximaldendriteshavearoughlylinearadditiveeffect
atthecellbody.Fiveactivesynapseswillleadtoroughlyfivetimesthe
depolarizationatthecellbodycomparedtooneactivesynapse.Incontrast,ifa
singlesynapseisactivatedrepeatedlybyaquicksuccessionofactionpotentials,the
second,third,andsubsequentactionpotentialshavemuchlesseffectatthecell
body,thanthefirst.
Therefore,wecansaythatinputstotheproximaldendritessumlinearlyatthecell
body,andthatrapidspikesarrivingatasinglesynapsewillhaveonlyaslightly
largereffectthanasinglespike.
Thefeedforwardconnectionstoaregionofneocortexpreferentiallyconnecttothe
proximaldendrites.Thishasbeenreportedatleastforlayer4neurons,theprimary
inputlayerofneuronsineachregion.
DistalDendrites
Thedendritebranchesfartherfromthecellbodyarecalleddistaldendrites.Inthe
diagramsomeofthedistaldendritesaremarkedwithbluelines.
Distaldendritesarethinnerthanproximaldendrites.Theyconnecttoother
dendritesatbranchesinthedendritictreeanddonotconnectdirectlytothecell
body.Thesedifferencesgivedistaldendritesuniqueelectricalandchemical
properties.Whenasinglesynapseisactivatedonadistaldendrite,ithasaminimal
effectatthecellbody.Thedepolarizationthatoccurslocallytothesynapse
weakensbythetimeitreachesthecellbody.Formanyyearsthiswasviewedasa
mystery.Itseemedthedistalsynapses,whicharethemajorityofsynapsesona
neuron,couldntdomuch.
Wenowknowthatsectionsofdistaldendritesactassemiindependentprocessing
regions.Ifenoughsynapsesbecomeactiveatthesametimewithinashortdistance
alongthedendrite,theycangenerateadendriticspikethatcantraveltothecell
bodywithalargeeffect.Forexample,twentyactivesynapseswithin40umofeach
otherwillgenerateadendriticspike.
Numenta2010 Page53
Therefore,wecansaythatthedistaldendritesactlikeasetofthresholdcoincidence
detectors.
Thesynapsesformedondistaldendritesarepredominantlyfromothercellsnearby
intheregion.
Theimageshowsalargedendritebranchextendingupwardswhichiscalledthe
apicaldendrite.Onetheorysaysthatthisstructureallowstheneurontolocate
severaldistaldendritesinanareawheretheycanmoreeasilymakeconnectionsto
passingaxons.Inthisinterpretation,theapicaldendriteactsasanextensionofthe
cell.
Synapses
Atypicalneuronmighthaveseveralthousandsynapses.Thelargemajority
(perhaps90%)ofthesewillbeondistaldendrites,andtherestwillbeonproximal
dendrites.
Formanyyearsitwasassumedthatlearninginvolvedstrengtheningandweakening
theeffectorweightofsynapses.Althoughthiseffecthasbeenobserved,each
synapseissomewhatstochastic.Whenactivated,itwillnotreliablyreleasea
neurotransmitter.Thereforethealgorithmsusedbythebraincannotdependon
precisionorfidelityofindividualsynapseweights.
Further,wenowknowthatentiresynapsesformandunformrapidly.This
flexibilityrepresentsapowerfulformoflearningandbetterexplainstherapid
acquisitionofknowledge.Asynapsecanonlyformifanaxonandadendriteare
withinacertaindistance,leadingtotheconceptofpotentialsynapses.Withthese
assumptions,learningoccurslargelybyformingvalidsynapsesfrompotential
synapses.
NeuronOutput
Theoutputofaneuronisaspike,oractionpotential,whichpropagatesalongthe
axon.Theaxonleavesthecellbodyandalmostalwayssplitsintwo.Onebranch
travelshorizontallymakingmanyconnectionswithothercellsnearby.Theother
branchprojectstootherlayersofcellsorelsewhereinthebrain.Intheimageofthe
neuronabove,theaxonwasnotvisible.Weaddedalineandtwoarrowsto
representthataxon.
Althoughtheactualoutputofaneuronisalwaysaspike,therearedifferentviews
onhowtointerpretthis.Thepredominantview(especiallyinregardstothe
neocortex)isthattherateofspikesiswhatmatters.Thereforetheoutputofacell
canbeviewedasascalarvalue.
Someneuronsalsoexhibitaburstingbehavior,ashortandfastseriesofafew
spikesthataredifferentthantheregularspikingpattern.
Numenta2010 Page54
Theabovedescriptionofaneuronisintendedtogiveabriefintroductionto
neurons.ItfocusesonattributesthatcorrespondtofeaturesofHTMcellsand
leavesoutmanydetails.Notallthefeaturesjustdescribedareuniversallyaccepted.
Weincludethembecausetheyarenecessaryforourmodels.Whatisknownabout
neuronscouldeasilyfillseveralbooks,andactiveresearchonneuronscontinues
today.
Simpleartificialneurons

ThemiddleimageatthebeginningofthisAppendixshowsaneuronlikeelement
usedinmanyclassicartificialneuralnetworkmodels.Theseartificialneuronshave
asetofsynapseseachwithaweight.Eachsynapsereceivesascalaractivation,
whichismultipliedbythesynapseweight.Theoutputofallthesynapsesis
summedinanonlinearfashiontoproduceanoutputoftheartificialneuron.
Learningoccursbyadjustingtheweightsofthesynapsesandperhapsthenon
linearfunction.
Thistypeofartificialneuron,andvariationsofit,hasprovenusefulinmany
applicationsasavaluablecomputationaltool.However,itdoesntcapturemuchof
thecomplexityandprocessingpowerofbiologicalneurons.Ifwewantto
understandandmodelhowanensembleofrealneuronsworksinthebrainweneed
amoresophisticatedneuronmodel.
HTMcells

Inourillustration,theimageontherightdepictsacellusedintheHTMcortical
learningalgorithms.AnHTMcellcapturesmanyoftheimportantcapabilitiesof
realneuronsbutalsomakesseveralsimplifications.
ProximalDendrite
EachHTMcellhasasingleproximaldendrite.Allfeedforwardinputstothecellare
madeviasynapses(shownasgreendots).Theactivityofsynapsesislinearly
summedtoproduceafeedforwardactivationforthecell.
Werequirethatallcellsinacolumnhavethesamefeedforwardresponse.Inreal
neuronsthiswouldlikelybedonebyatypeofinhibitorycell.InHTMswesimply
forceallthecellsinacolumntoshareasingleproximaldendrite.
Toavoidhavingcellsthatneverwininthecompetitionwithneighboringcells,an
HTMcellwillboostitsfeedforwardactivationifitisnotwinningenoughrelativeto
itsneighbors.Thusthereisaconstantcompetitionbetweencells.Again,inanHTM
wemodelthisasacompetitionbetweencolumns,notcells.Thiscompetitionisnot
illustratedinthediagram.
Numenta2010 Page55
Finally,theproximaldendritehasanassociatedsetofpotentialsynapseswhichisa
subsetofalltheinputstoaregion.Asthecelllearns,itincreasesordecreasesthe
permanencevalueofallthepotentialsynapsesontheproximaldendrite.Only
thosepotentialsynapsesthatareaboveathresholdarevalid.
Asmentionedearlier,theconceptofpotentialsynapsescomesfrombiologywhereit
referstoaxonsanddendritesthatarecloseenoughtoformasynapse.Weextend
thisconcepttoalargersetofpotentialconnectionsforanHTMcell.Dendritesand
axonsonbiologicalneuronscangrowandretractaslearningoccursandtherefore
thesetofpotentialsynapseschangeswithgrowth.Bymakingthesetofpotential
synapsesonanHTMcelllarge,weroughlyachievethesameresultasaxonand
dendritegrowth.Thesetofpotentialsynapsesisnotshown.
Thecombinationofcompetitionbetweencolumns,learningfromasetofpotential
synapses,andboostingunderutilizedcolumnsgivesaregionofHTMneuronsa
powerfulplasticityalsoseeninbrains.AnHTMregionwillautomaticallyadjust
whateachcolumnrepresents(viachangestothesynapsesontheproximal
dendrites)iftheinputchanges,orthenumberofcolumnsincreasesordecreases.
DistalDendrites
EachHTMcellmaintainsalistofdistaldendritesegments.Eachsegmentactslikea
thresholddetector.Ifthenumberofactivesynapsesonanysegment(shownasblue
dotsontheearlierdiagram)isaboveathreshold,thesegmentbecomesactive,and
theassociatedcellentersthepredictivestate.ThepredictivestateofacellistheOR
oftheactivationsofitssegments.
Adendritesegmentremembersthestateoftheregionbyformingconnectionsto
cellsthatwereactivetogetheratapointintime.Thesegmentremembersastate
thatprecedesthecellbecomingactiveduetofeedforwardinput.Thusthesegment
islookingforastatethatpredictsthatitscellwillbecomeactive.Atypicalthreshold
foradendritesegmentis15.If15validsynapsesonasegmentareactiveatonce,
thedendritebecomesactive.Theremightbehundredsorthousandsofcellsactive
nearby,butconnectingtoonly15issufficienttorecognizethelargerpattern.
Eachdistaldendritesegmentalsohasanassociatedsetofpotentialsynapses.The
setofpotentialsynapsesisasubsetofallthecellsinaregion.Asthesegment
learns,itincreasesordecreasesthepermanencevalueofallitspotentialsynapses.
Onlythosepotentialsynapsesthatareaboveathresholdarevalid.
Inoneimplementation,weuseafixednumberofdendritesegmentspercell.In
anotherimplementation,weaddanddeletesegmentswhiletraining.Bothmethods
canwork.Ifwehaveafixednumberofdendritesegmentspercell,itispossibleto
storeseveraldifferentsetsofsynapsesonthesamesegment.Forexample,saywe
have20validsynapsesonasegmentandathresholdof15.(Ingeneralwewantthe
thresholdtobelessthanthenumberofsynapsestoimprovenoiseimmunity.)The
Numenta2010 Page56
segmentcannowrecognizeoneparticularstateofthecellsnearby.Whatwould
happenifweaddedanother20synapsestothesamesegmentrepresentingan
entirelydifferentstateofcellsnearby?Itintroducesthepossibilityoferrorbecause
thesegmentcouldadd8activesynapsesfromonepatternand7activesynapses
fromtheotherandbecomeactiveincorrectly.Wehavefoundexperimentallythat
upto20differentpatternscanbestoredononesegmentbeforeerrorsoccur.
ThereforeanHTMcellwithadozendendritesegmentscanparticipateinmany
differentpredictions.
Synapses
SynapsesonanHTMcellhaveabinaryweight.ThereisnothingintheHTMmodel
thatprecludesscalarsynapseweights,butduetotheuseofsparsedistributed
patternswehavenotyethadaneedtousescalarweights.
However,synapsesonanHTMcellhaveascalarvaluecalledpermanencewhichis
adjustedduringlearning.A0.0permanencevaluerepresentsapotentialsynapse
whichisnotvalidandhasnotprogressedatalltowardsbecomingavalidsynapse.
Apermanencevalueaboveathreshold(typically0.2)representsasynapsethathas
justconnectedbutcouldeasilybeunconnected.Ahighpermanencevalue,for
example0.9,representsasynapsethatisconnectedandcannoteasilybeun
connected.
Thenumberofvalidsynapsesontheproximalanddistaldendritesegmentsofan
HTMcellisnotfixed.Itchangesasthecellisexposedtopatterns.Forexample,the
numberofvalidsynapsesonthedistaldendritesisdependentonthetemporal
structureofthedata.Iftherearenopersistenttemporalpatternsintheinputtothe
region,thenallthesynapsesondistalsegmentswouldhavelowpermanencevalues
andveryfewsynapseswouldbevalid.Ifthereisalotoftemporalstructureinthe
inputstream,thenwewillfindmanyvalidsynapseswithhighpermanence.
CellOutput
AnHTMcellhastwodifferentbinaryoutputs:1)thecellisactiveduetofeed
forwardinput(viatheproximaldendrite),and2)thecellisactiveduetolateral
connections(viathedistaldendritesegments).Theformeriscalledtheactive
stateandthelatteriscalledthepredictivestate.
Intheearlierdiagram,thetwooutputsarerepresentedbythetwolinesexitingthe
squarecellbody.Theleftlineisthefeedforwardactivestate,whiletherightlineis
thepredictivestate.
Onlythefeedforwardactivestateisconnectedtoothercellsintheregion,ensuring
thatpredictionsarealwaysbasedonthecurrentinput(pluscontext).Wedont
wanttomakepredictionsbasedonpredictions.Ifwedid,almostallthecellsinthe
regionwouldbeinthepredictivestateafterafewiterations.
Numenta2010 Page57
Theoutputoftheregionisavectorrepresentingthestateofallthecells.This
vectorbecomestheinputtothenextregionofthehierarchyifthereisone.This
outputistheORoftheactiveandpredictivestates.Bycombiningbothactiveand
predictivestates,theoutputofourregionwillbemorestable(slowerchanging)
thantheinput.Suchstabilityisanimportantpropertyofinferenceinaregion.
Suggestedreading

Weareoftenaskedtosuggestreadingmaterialstolearnmoreaboutneuroscience.
Thefieldofneuroscienceissolargethatageneralintroductionrequireslookingat
manydifferentsources.Newfindingsarepublishedinacademicjournalswhichare
bothhardtoreadandhardtogetaccesstoifyoudonthaveauniversityaffiliation.
Herearetworeadilyavailablebooksthatadedicatedreadermightwanttolookat
whicharerelevanttothetopicsinthisappendix.
Stuart,Greg,Spruston,Nelson,Hauser,Michael,Dendrites,secondedition(New
York:OxfordUniversityPress,2008)
Thisbookisagoodsourceoneverythingaboutdendrites.Chapter16discussesthe
nonlinearpropertiesofdendritesegmentsusedintheHTMcorticallearning
algorithms.ItiswrittenbyBartlettMelwhohasdonemuchofthethinkinginthis
field.
Mountcastle,VernonB.PerceptualNeuroscience:TheCerebralCortex
(Cambridge,Mass.:HarvardUniversityPress,1998)
Thisbookisagoodintroductiontoeverythingabouttheneocortex.Severalofthe
chaptersdiscusscelltypesandtheirconnections.Youcangetagoodsenseof
corticalneuronsandtheirconnections,althoughitistoooldtocoverthelatest
knowledgeofdendriteproperties.

Ierarchical Emporal Emory: HTM Cortical Learning Algorithms

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ierarchical Emporal Emory: HTM Cortical Learning Algorithms

Uploaded by

Copyright:

Available Formats

HIERARCHICALTEMPORALMEMORY

Version Date Changes PrincipalAuthors

Figure 1.2: Converging networks from different sensors

Figure 1.4: An HTM region showing sparse distributed cell activation

11. for c in columns

18. for c in activeColumns(t)

14. for c, i in cells

42. for c, i in cells

54. for c, i in cells

getSegmentActiveSynapses(c, i, t, s, newSynapses= false)

You might also like