Professional Documents
Culture Documents
Brain Emulation Roadmap Report
Brain Emulation Roadmap Report
A Roadmap
(*)Correspondingauthor:anders.sandberg@philosophy.ox.ac.uk
Contents
WholeBrainEmulation............................................................................................................................1 ARoadmap ................................................................................................................................................1 Inmemoriam:BruceH.McCormick(19302007)...........................................................................2 Contents..................................................................................................................................................3 Introduction ...........................................................................................................................................5 Thanksto ............................................................................................................................................6 Theconceptofbrainemulation..........................................................................................................7 Emulationandsimulation...............................................................................................................7 Littleneedforwholesystemunderstanding...............................................................................8 Levelsofemulationandsuccesscriteria.....................................................................................10 Scaleseparation...............................................................................................................................12 Simulationscales.............................................................................................................................13 WBEassumptions ...........................................................................................................................15 Roadmap ..............................................................................................................................................16 Requirements...................................................................................................................................16 Linkages............................................................................................................................................19 Roadmap ..........................................................................................................................................20 Technologydrivers.........................................................................................................................23 Uncertaintiesandalternatives......................................................................................................24 Alternativepathways.....................................................................................................................27 Relatedtechnologiesandspinoffs ..............................................................................................28 Issues.....................................................................................................................................................30 Emulationsystems..........................................................................................................................30 Complicationsandexotica ............................................................................................................31 Summary ..........................................................................................................................................39 Scanning ...............................................................................................................................................40 Embedding,fixationandstainingtechniques ...........................................................................52 Conclusion .......................................................................................................................................53 Imageprocessingandscaninterpretation......................................................................................55 Geometricadjustment....................................................................................................................55 Noiseremoval .................................................................................................................................56 Datainterpolation...........................................................................................................................56 Celltracing .......................................................................................................................................57 Synapseidentification....................................................................................................................59 Identificationofcelltypes .............................................................................................................60 Estimationofparametersforemulation .....................................................................................61 Connectivityidentification............................................................................................................62 Conclusion .......................................................................................................................................63 Neuralsimulation ...............................................................................................................................64 Howmuchneurondetailisneeded?...........................................................................................64 Neuralmodels .................................................................................................................................66 Simulators ........................................................................................................................................70 Parallelsimulation..........................................................................................................................70 Currentlargescalesimulations....................................................................................................71 Conclusion .......................................................................................................................................72 Bodysimulation ..................................................................................................................................74 Conclusion .......................................................................................................................................75 Environmentsimulation....................................................................................................................76
Vision ................................................................................................................................................76 Hearing .............................................................................................................................................77 SmellandTaste ...............................................................................................................................77 Haptics..............................................................................................................................................77 Conclusion .......................................................................................................................................78 Computerrequirements ....................................................................................................................79 Conclusions......................................................................................................................................81 Validation.............................................................................................................................................82 Discussion ............................................................................................................................................83 AppendixA:Estimatesofthecomputationalcapacity/demandsofthehumanbrain ..........84 AppendixB:ComputerPerformanceDevelopment ....................................................................86 ProcessingPower............................................................................................................................86 Memory ............................................................................................................................................95 Discdrives........................................................................................................................................97 Future................................................................................................................................................98 AppendixC:Largescaleneuralnetworksimulations...............................................................101 AppendixD:Historyandpreviouswork.....................................................................................105 AppendixE:Nondestructiveandgradualreplacement...........................................................107 NonDestructiveScanning ..........................................................................................................107 Gradualreplacement....................................................................................................................108 AppendixF:Glossary.......................................................................................................................110 References ..........................................................................................................................................113
Introduction
Wholebrainemulation(WBE),thepossiblefutureonetoonemodellingofthefunctionofthe humanbrain,isacademicallyinterestingandimportantforseveralreasons: Research o Brainemulationisthelogicalendpointofcomputationalneurosciences attemptstoaccuratelymodelneuronsandbrainsystems. o Brainemulationwouldhelpustounderstandthebrain,bothintheleadup tosuccessfulemulationandafterwardsbyprovidinganidealtestbedfor neuroscientificexperimentationandstudy. o Neuromorphicengineeringbasedonpartialresultswouldbeusefulina numberofapplicationssuchaspatternrecognition,AIandbraincomputer interfaces. o Asalongtermresearchgoalitmightbeastrongvisiontostimulate computationalneuroscience. o Asacaseoffuturestudiesitrepresentsacasewherearadicalfuture possibilitycanbeexaminedinthelightofcurrentknowledge. Economics o Theeconomicimpactofcopyablebrainscouldbeimmense,andcouldhave profoundsocietalconsequences(Hanson,1994,2008b).Evenlowprobability eventsofsuchmagnitudemeritinvestigation. Individually o Ifemulationofparticularbrainsispossibleandaffordable,andifconcerns aboutindividualidentitycanbemet,suchemulationwouldenablebackup copiesanddigitalimmortality. Philosophy o Brainemulationwoulditselfbeatestofmanyideasinthephilosophyof mindandphilosophyofidentity,orprovideanovelcontextforthinking aboutsuchideas. o Itmayrepresentaradicalnewformofhumanenhancement. WBErepresentsaformidableengineeringandresearchproblem,yetonewhichappearsto haveawelldefinedgoalandcould,itwouldseem,beachievedbyextrapolationsofcurrent technology.Thisisunlikemanyothersuggestedradicallytransformativetechnologieslike artificialintelligencewherewedonothaveanyclearmetricofhowfarwearefromsuccess. InordertodevelopideasaboutthefeasibilityofWBE,groundtechnologyforesightand stimulateinterdisciplinaryexchange,theFutureofHumanityInstitutehostedaworkshopon May26and27,2007,inOxford.Invitedexpertsfromareassuchascomputational neuroscience,brainscanningtechnology,computing,nanotechnology,andneurobiology presentedtheirfindingsanddiscussedthepossibilities,problemsandmilestonesthatwould havetobereachedbeforeWBEbecomesfeasible. Theworkshopavoideddealingwithsocioeconomicramificationsandwithphilosophical issuessuchastheoryofmind,identityorethics.Whileimportant,suchdiscussionswould undoubtedlybenefitfromamorecomprehensiveunderstandingofthebrainanditwasthis understandingthatwewishedtofocusonfurtheringduringtheworkshop.Suchissueswill likelybedealtwithatfutureworkshops.
Thisdocumentcombinesanearlierwhitepaperthatwascirculatedamongworkshop participants,andadditionssuggestedbythoseparticipantsbefore,duringandafterthe workshop.ItaimsatprovidingapreliminaryroadmapforWBE,sketchingoutkey technologiesthatwouldneedtobedevelopedorrefined,andidentifyingkeyproblemsor uncertainties. Brainemulationiscurrentlyonlyatheoreticaltechnology.Thismakesitvulnerableto speculation,handwavinganduntestableclaims.AsproposedbyNickSzabo,falsifiable designisawayofcurbingtheproblemswiththeoreticaltechnology: thedesignersofatheoreticaltechnologyinanybutthemostpredictableofareas shouldidentifyitsassumptionsandclaimsthathavenotalreadybeentestedina laboratory.Theyshoulddesignnotonlythetechnologybutalsoamapofthe uncertaintiesandedgecasesinthedesignandaseriesofsuchexperimentsandtests thatwouldprogressivelyreducetheseuncertainties.Aproposalthatlacksthis admissionofuncertaintiescoupledwithdesignsofexperimentsthatwillreducesuch uncertaintiesshouldnotbedeemedcredibleforthepurposesofanyimportant decision.Wemightcallthisrequirementarequirementforafalsifiabledesign.(Szabo, 2007) Inthecaseofbrainemulationthiswouldmeannotonlysketchinghowabrainemulator wouldworkifitcouldbebuiltandaroadmapoftechnologiesneededtoimplementit,but alsoalistofthemainuncertaintiesinhowitwouldfunctionandproposedexperimentsto reducetheseuncertainties. Itisimportanttoemphasizethelongtermandspeculativenatureofmanyaspectsofthis roadmap,whichinanycaseistoberegardedonlyasafirstdrafttobeupdated,refined, andcorrectedasbetterinformationbecomesavailable.Giventhedifficultiesand uncertaintiesinherentinthistypeofwork,onemayaskwhetherourstudyisnotpremature. Ourviewisthatwhenthestakesarepotentiallyextremelyhigh,itisimportanttoapplythe bestavailablemethodstotrytounderstandtheissue.Evenifthesemethodsarerelatively weak,itisthebestwecando.Thealternativewouldbetoturnablindeyetowhatcouldturn outtobeapivotaldevelopment.Withoutfirststudyingthequestion,howisonetoformany wellgroundedviewonewayortheotherastothefeasibilityandproximityofaprospectlike WBE?
Thanks to
Wewouldliketowarmlythankthemanypeoplewhohavecommentedonthepaperand helpedextendandimproveit: Workshopparticipants:JohnFiala,RobinHanson,KennethJeffreyHayworth,Todd Huffman,EugeneLeitl,BruceMcCormick,RalphMerkle,TobyOrd,PeterPassaro,Nick Shackel,RandallA.Koene,RobertA.FreitasJrandRebeccaRoache. Otherusefulcomments:StuartArmstrong.
Theconceptofbrainemulation
Wholebrainemulation,ofteninformallycalleduploadingordownloading,hasbeenthe subjectofmuchsciencefictionandalsosomepreliminarystudies(seeAppendixDforhistory andpreviouswork).Thebasicideaistotakeaparticularbrain,scanitsstructureindetail, andconstructasoftwaremodelofitthatissofaithfultotheoriginalthat,whenrunon appropriatehardware,itwillbehaveinessentiallythesamewayastheoriginalbrain.
Apersonemulationisamindemulationthatemulatesaparticularmind. Whattherelevantpropertiesareisacrucialissue.Intermsofsoftwareemulationthisis oftenthebitsstoredinmemoryandhowtheyareprocessed.Acomputeremulatormay emulatetheprocessor,memory,I/Oandsoonoftheoriginalcomputer,butdoesnotsimulate theactualelectronicworkingsofthecomponents,onlytheirqualitativefunctiononthestored information(anditsinteractionwiththeoutsideworld).Whilelowerlevelemulationof computersmaybepossibleitwouldbeinefficientandnotcontributemuchtothefunctions thatinterestus. Dependingonthedesiredsuccesscriterionemulationmayrequiredifferentlevelsofdetail.It mightalsousedifferentlevelsofdetailindifferentpartsofthesystem.Inthecomputer example,emulatingtheresultofamathematicalcalculationmaynotrequiresimulatingthe executionofalloperatingsystemcallsformathfunctions(sincethesecanbedonemore efficientlybytheemulatingcomputersprocessor)whileemulatingthebehaviourofan analoguevideoeffectmayrequireadetailedelectronicssimulation.
learningorpatternrecognizingsystemsontheirown,artificialneuralnetworks(ANNs). ANNmodelscanbeusedtoqualitativelymodel,explainandanalyzethefunctionsofbrain systems(Rumelhart,McClellandetal.,1986).Connectionistmodelsbuildmorecomplex modelsofcognitionorbrainfunctiononthesesimplerparts.Theendpointofthispursuit wouldbemodelsthatencompassafullunderstandingofthefunctionofallbrainsystems. Suchqualitativemodelsmightnotexhibitintelligenceorthecomplexityofhumanbehaviour butwouldenableaformalizedunderstandingofhowtheycomeaboutfromsimpleparts. Anotherapproachincomputationalneuroscienceinvolvescreatingmorebiologicallyrealistic models,whereinformationaboutthebiologicaldetailsofneuronssuchastheir electrochemistry,biochemistry,detailedmorphologyandconnectivityareincluded.Atits simplestwefindcompartmentmodelsofindividualneuronsandsynapses,whilemore complexmodelsincludemultiplerealisticneuronsconnectedintonetworks,possiblytaking interactionssuchaschemicalvolumetransmissionintoaccount.Thisapproachcanbeseenas aquantitativeunderstandingofthebrain,aimingforacompletelistofthebiologicalparts (chemicalspecies,neuronmorphologies,receptortypesanddistributionetc.)andmodelling asaccuratelyaspossiblethewayinwhichthesepartsinteract.Giventhisinformation increasinglylargeandcomplexsimulationsofneuralsystemscanbecreated.WBErepresents thelogicalconclusionofthiskindofquantitativemodel:a1to1modelofbrainfunction. Notethattheamountoffunctionalunderstandingneededtoachievea1to1modelissmall. Itsbehaviourisemergentfromthelowlevelproperties,andmayormaynotbeunderstood bytheexperimenters.Forexample,ifcoherentoscillationsareimportantforconceptual bindingandtheseemergefromthelowlevelpropertiesofneuronsandtheirnetworks,a correctandcompletesimulationofthesepropertieswillproducethecoherence. Inpracticecomputationalneuroscienceworksinbetweenquantitativeandqualitative models.Qualitativemodelsareusedtoabstractcomplex,uncertainandpotentiallyirrelevant biologicaldata,andoftenprovidesignificantimprovementsinsimulationprocessing demands(inturnenablinglargersimulations,whichmayenableexplorationofdomainsof moreinterest).Quantitativemodelsaremoreconstrainedbyknownbiology,chemistryand physicsbutoftensufferfromanabundanceoffreeparametersthathavetobeset(Herz, Gollischetal.,2006).Hybridmodelsmayincludepartsusingdifferentlevelsofabstraction, orexistasafamilyofmodelsrepresentingthesamesystematdifferentlevelsofabstraction. Theinterplaybetweenbiologicalrealism(attemptingtobefaithfultobiology),completeness (usingallavailableempiricaldataaboutthesystem),tractability(thepossibilityof quantitativeorqualitativesimulation)andunderstanding(producingacompressed representationofthesalientaspectsofthesysteminthemindoftheexperimenter)willoften determinewhatkindofmodelisused.Theappropriatelevelofabstractionandmethodof implementationdependsontheparticulargoalofthemodel.InthecaseofWBE,thesuccess criteriadiscussedbelowplacelittleemphasisonunderstanding,butmuchemphasison qualitativelycorrectdynamics,requiringmuchbiologicalrealism(uptoapoint,setbyscale separation)andtheneedfordatadrivenmodels.Whethersuchmodelsforwholebrain systemsaretractablefromamodellingandsimulationstandpointisthecrucialissue. Brainemulationcannotbeachievedwithoutsomefunctionalunderstanding.Itneedsmodels andtheoriesforrecognizingwhatdataisrelevant,andwouldprovidedatafordeveloping andtestingthesefurther.WhileintheorybrainemulationmighthugthelowerlineinFigure 1,inpracticeitwilllikelyoccursomewherealongtherightedgestillfarbelowafull understandingofthetoplevelphenomena,butincludingabroadunderstandingofmany
10
1b 2
Completescan Braindatabase
Functionalbrain emulation
Genericallycorrectcausalmicro dynamics.
Speciesgenericbrain emulation
Individualbrain emulation
6a
6b
Mindemulation
Longtermdynamicsand adaptation.Appropriate behaviourresponses.Fullrange learningcapacity. Correctinternalandbehaviour responses.Retainsmost memoriesandskillsofthe particularbrainthatwas emulated.(Inanemulationofan animalbrain,itshouldbe possibletorecognizethe particular(familiar)animal.) Propertiesdependonwhich (rangeof)socialrolesthe emulationwouldbeabletofit.In alimitingcase,theemulation wouldbeabletopassa personalizedTuringtest: outsidersfamiliarwiththe emulatedpersonwouldbe unabletodetectwhether responsescamefromoriginal personoremulation. Theemulationistrulyconscious inthesamewayasanormal humanbeing.
6c
Personalidentity emulation
11
Scale separation
Atfirstitmayappearunlikelythatacomplexsystemwithmanydegreesoffreedomlikethe braincouldbemodelledwiththerightcausaldynamics,butwithouttakingintoaccountthe smallestparts.Microstimulationofindividualneuronscaninfluencesensorydecisions (HouwelingandBrecht,2008),showingthatverysmalldisturbancescanundertheright circumstancesscaleuptobehaviouraldivergences. However,statevariablesofcomplexsystemscanbequantitativelypredictedwhenthereis scaleseparation:whendifferentaspectsofthesystemexistonsufficiently(ordersof magnitude)differentscales(ofsize,energy,timeetc),theycanbecomeuncoupled (Hillerbrand,2008).Atypicalexampleishowthemicroscopicdynamicsofalaser(atoms interactingwithanoscillatingelectromagneticfield)givesrisetoamacroscopicdynamics (thegrowthanddecayofdifferentlasermodes)insuchawaythatanaccuratesimulationof thesystemusingonlyelementsonthemacroscaleispossible.Anotherexampleisthescale separationbetweenelectriccurrentsandlogicoperationsinacomputer,whichenablesbit basedemulation.Whenthereisnoscaleseparation(suchasinfluidturbulence)macroscale predictionsbecomeimpossiblewithoutsimulatingtheentiremicroscale. Animportantissuetobedeterminediswhethersuchacutoffexistsinthecaseofthehuman brainand,ifitdoesexist,atwhatlevel.Whilethispaperphrasesitintermsof simulation/emulation,itisencounteredinarangeoffields(AI,cognitiveneuroscience, philosophyofmind)inotherforms:whatleveloforganisationisnecessaryforintelligent, personal,orconsciousbehaviour? AkeyassumptionofWBEisthat,atsomeintermediarylevelofsimulationresolution betweentheatomicandthemacroscopic,thereexistsatleastonecutoffsuchthatmeeting criteria1aand1batthislevelofresolutionalsoenablesthehighercriteriatobemet. Atsuchaspatial,temporal,ororganisationalscale,thedynamicsonthelarger/slowerscaleis notfunctionallysensitivetothedynamicsofthesmaller/fasterscale.Suchscaleseparation mightoccuratthesynapticscale,wherethedetailedchemicaldynamicsunderlyingsynaptic functioncouldbereplacedbyasimplifiedqualitativemodelofitseffectsonsignalsand synapticstrengths.Anotherpossiblescaleseparationlevelmightoccurbetweenindividual moleculesandmolecularconcentrationscales:moleculardynamicscouldbereplacedwith massactioninteractionsofconcentrations.Aperhapslesslikely separationcouldalsooccuronhigherlevelsifwhatmattersisthe activityofcorticalminicolumnsratherthanindividualneurons.A finallikelybutcomputationallydemandingscaleorseparation wouldbetheatomicscale,treatingthebrainemulationasaNbody systemofatoms. Conversely,ifitcouldbedemonstratedthatthereisnosuchscale,it woulddemonstratetheinfeasibilityofwholebrainemulation.Due tocausallyimportantinfluencefromsmallerscalesinthiscase,a simulationataparticularscalecannotbecomeanemulation.The causaldynamicsofthesimulationisnotinternallyconstrained,soit isnota1to1modeloftherelevantdynamics.Biologically interestingsimulationsmightstillbepossible,buttheywouldbe localtoparticularscalesandphenomena,andtheywouldnotfully reproducetheinternalcausalstructureofthewholebrain. Figure3:Sizescalesin thenervoussystem.
12
Simulation scales
Thewidelyreproduceddiagramfrom(ChurchlandandSejnowski,1992)inFigure3depicts thevariouslevelsoforganisationinthenervoussystemorderedbysizescale,runningfrom themolecularleveltotheentiresystem.Simulations(andpossiblyemulations)canoccuron alllevels: Molecularsimulation(individualmolecules) Chemistrysimulation(concentrations,lawofmassaction) Geneticexpression Compartmentmodels(subcellularvolumes) Wholecellmodels(individualneurons) Localnetworkmodels(replacesneuronswithnetworkmodulessuchas minicolumns) Systemmodels AnotherhierarchywasintroducedbyJohnFialaduringtheworkshop,andwillbeusedwith someadaptationsinthisdocument. Table2:Levelsofemulation
Level 1 2 3 Computational module Brainregion connectivity Analognetwork populationmodel ClassicAI,highlevelrepresentationsofinformationandinformation processing. Eacharearepresentsafunctionalmodule,connectedtoothersaccording toa(speciesuniversal)connectome(Sporns,Tononietal.,2005). Neuronspopulationsandtheirconnectivity.Activityandstatesof neuronsorgroupsofneuronsarerepresentedastheirtimeaverages.This issimilartoconnectionistmodelsusingANNs,ratemodelneural simulationsandcascademodels. Asabove,plusfiringproperties,firingstateanddynamicalsynapticstates. Integrateandfiremodels,reducedsinglecompartmentmodels(butalso someminicolumnmodels,e.g.(JohanssonandLansner,2007)). Asabove,plusmembranestates(ionchanneltypes,properties,state),ion concentrations,currents,voltagesandmodulationstates.Compartment modelsimulations. Asabove,plusconcentrationsofmetabolitesandneurotransmittersin compartments. Asabove,plusconcentrationsofproteinsandgeneexpressionlevels. Asabove,plusquaternaryproteinstructure. Asabove,pluslocomeinformationandinternalcellulargeometry. Asaboveplusmoleculepositions,oramolecularmechanicsmodelofthe entirebrain. Quantuminteractionsinandbetweenmolecules.
6 7 8 9 10 11
13
modelalsochangescharacterbetweenthedifferentlevels.Lowlevelsimulationsrequire massivequantitiesofsimpleinformation(molecularpositionsandtypes)whereashigher levelsrequireasmalleramountofverycomplexinformation(contentofmentalprocesses). Eachlevelhasitsowncharacteristicsizeandtimescale,restrictingtherequiredimaging resolutionandsimulationtimestep.Forexample,synapticspinenecksandthethinnestaxons canbeontheorderof50nmorsmaller,requiringimagingontheorderofthe5nanometer scaletoresolvethem. Aninformalpollamongworkshopattendeesproducedarangeofestimatesoftherequired resolutionforWBEis.Theconsensusappearedtobelevel46.Twoparticipantsweremore optimisticabouthighlevelmodels,whiletwosuggestedthatelementsonlevel89maybe necessaryatleastinitially(butthatthebulkofmatureemulation,oncethebasicswere understood,couldoccuronlevel45).Toachieveemulationonthislevel,theconsensuswas that5550nmscanningresolutionwouldbeneeded.Thisroadmapwillhencefocusonlevel 46models,whilebeingopenforthatdeeperlevelsmayturnouttobeneeded. AsnotedbyFiala,WBElikelyrequiresatleastlevel4tocapturethespecificityofindividual brains,butprobablyrequirescomplexityatlevel6orlowertofullycapturethe computationalcontributionsofionchannels,secondmessengers,proteinleveladaptation, andstochasticsynaptictransmission.Otherparticipantsthoughtthatatleastlevel5wouldbe neededforindividualbrainproperties.
Forecasting
Analysingtherequirementsforemulation(intermsofscanningmethod,numberofentities tosimulate,resultingstoragerequirementsandcomputationaldemands)ateachofthelevels providesawayofboundingprogresstowardsWBE.Giventheseestimatesandscenariosof futureprogressinscanningandcomputingitispossibletocalculatetheearliestpointintime wherethereisenoughresourcestoproduceaWBEonagivenlevelatacertainprice.As betterinformationbecomesavailablesuchestimatescanberefined. Althoughanytimeestimateswillbesubjecttostronguncertainties,theycanbehelpfulin estimatinghowfarawayWBEisfrompolicyrelevanttimescales,aswellaslikelytimeframes forearlysmallscaleemulations.Theyalsoallowcomparisonstoothertechnologyforecasts, enablingestimationofthechancesforsynergies(e.g.thedevelopmentofmolecular nanotechnology,whichwouldaccelerateWBEprogress),reliability(e.g.thefurtherintothe future,themoreunlikelyMooreslawistohold),andtheriskofothertechnologies overtakingWBE(e.g.artificialintelligence). EarlyWBEmayrequirelowerlevelsimulationthanlaterforms,astheremightnotyetbe enoughexperience(andtestsystems)todeterminewhichsimulationelementsarestrictly necessaryforsuccess.ThemainconcerninthisdocumentisestimatingwhenandhowWBE willfirstbeachievedratherthanitseventualmatureorbestform.
14
WBE assumptions
Philosophicalassumptions
Physicalism(everythingsupervenesonthephysical)isaconvenientbutnotnecessary assumption,sincesomenonphysicalisttheoriesofmentalpropertiescouldallowthemto appearinthecaseofWBE.Successcriterion6bemulationassumesmultiplerealizability (thatthesamementalproperty,state,oreventcanbeimplementedbydifferentphysical properties,states,andevents).SufficientapparentsuccesswithWBEwouldprovide persuasiveevidenceformultiplerealizability.Generally,emulationuptoandincludinglevel 6adoesnotappeartodependonanystrongmetaphysicalassumptions.
Computationalassumptions
Computability:brainactivityisTuringcomputable,orifitisuncomputable,the uncomputableaspectshavenofunctionallyrelevanteffectsonactualbehaviour. Nonorganicism:totalunderstandingofthebrainisnotrequired,justcomponentpartsand theirfunctionalinteractions. Scaleseparation:atsomeintermediarylevelofsimulationresolutionbetweentheatomicand themacroscopicthereexistsone(ormore)cutoffssuchthatmeetingcriterion2atthislevelis sufficientformeetingoneormoreofthehighercriteria. Componenttractability:theactualbraincomponentsatthelowestemulatedlevelcanbe understoodwellenoughtoenableaccuratesimulation. Simulationtractability:simulationofthelowestemulatedleveliscomputationallytractable withapracticallyrealizablecomputer.
Neuroscienceassumptions
Braincenteredness:inordertoproduceaccuratebehaviouronlythebrainandsomepartsof thebodyneedtobesimulated,nottheentirebody. WBEappearstobeawayoftestingmanyoftheseassumptionsexperimentally.Inacquiring accuratedataaboutthestructureandfunctionofthebrainandrepresentingitasemulations itshouldbepossibletofindmajordiscrepanciesif,forexample,Computabilityisnottrue.
15
Roadmap
Requirements
WBErequiresthreemaincapabilities:theabilitytophysicallyscanbrainsinordertoacquire thenecessaryinformation,theabilitytointerpretthescanneddatatobuildasoftwaremodel, andtheabilitytosimulatethisverylargemodel.Theseinturnrequireanumberof subcapabilities(Table3:CapabilitiesneededforWBE). Plausiblescanningmethodsrequirewaysofpreparingthebrains,inparticularseparation fromothertissue,fixationandpossiblydyeing.Thereisalsoaneedformethodsofphysically handlingandstoringpiecesoftissue:sincemostscanningmethodscannotimagelarge volumesthebrainswillhavetobesectionedintomanageablepieces.Thismustallow correspondingcellsanddendritestobeidentifiedonbothsides.Whilefixationand sectioningmethodsarecommonlyusedinneuroscience,thedemandsforwholebrain emulationarestricter:muchlargervolumesmustbehandledwithfarlesstolerancefor damage. Imagingmethodsarediscussedinmoredetailinthechapteronscanning.Thethreekey issuesareachievingthenecessaryresolutiontoimagethesmallestsystemsneededforan emulation,theabilitytoimage(notnecessarilysimultaneously)theentirebrain,andthe abilitytoacquirethefunctionallyrelevantinformation. Translatingthedatafromtheimagingprocessintosoftwarerequiressophisticatedimage processing,theabilitytointerprettheimageryintosimulationrelevantparameters,and havingacomputationalneurosciencemodelofsufficientprecision.Theimageprocessingwill havetodealwiththeunavoidableartefactsfromscanningsuchasdistortionsandnoise,as wellasoccasionallostdata.Itwilllikelyincludemethodsofconvertingdirectscandatainto morecompressedforms,suchastracedstructures,inordertoavoidexcessivestorageneeds. Thescaninterpretationprocessmakesuseofthisdatatoestimatetheconnectivity,andto identifysynapticconnections,celltypesandsimulationparameters.Itthenplacesthis informationinaninventorydatabasefortheemulation.Thesestepsarediscussedinthe imageprocessingchapter. Thesoftwaremodelrequiresbothamathematicalmodelofneuralactivityandwaysof efficientlyimplementingsuchmodelsoncomputers(discussedinthechapteronneural simulation).Computationalneuroscienceaimsatmodellingthebehaviourofneuralentities suchasnetworks,neurons,synapsesandlearningprocesses.ForWBE,itneedstohave sufficientlygoodmodelsofallrelevantkindsofsubsystems,alongwiththerelevant parameterssetfromscandatainordertoconstructacomputationalmodeloftheactualbrain thatwasscanned. Toemulateabrain,weneedenoughcomputingpowertorunthebasicemulationsoftware,a sufficientlyrealisticbodysimulation,andpossiblyasimulatedenvironment.Thekey demandsareformemorystoragetoholdtheinformationandprocessorpowertorunitata suitablespeed.Themassiveparallelismoftheproblemwillputsomesignificantdemandson theinternalbandwidthofthecomputingsystem. Inaddition,WBElikelyrequiresthedevelopmentofthreesupportingtechnologyareas,with whichithasasymbioticrelationship.First,validationmethodstocheckthatothersteps
16
produceaccuratedataandmodels.Thisincludesvalidationofscanning,validationofscan interpretation,validationofneurosciencemodels,validationsofimplementation,andwaysof testingthesuccessofWBE.Whileordinaryneuroscienceresearchcertainlyaimsatvalidation, itdoesnotsystematizeit.ForacomplexmultistepresearcheffortlikeWBE,integrated validationislikelynecessarytoensurethatbaddataormethodsdonotconfuselaterstepsin theprocess.Second,WBErequiressignificantlowlevelunderstandingofneurosciencein ordertoconstructthenecessarycomputationalmodelsandscaninterpretationmethods.This isessentiallyacontinuationandstrengtheningofsystemsbiologyandcomputational neuroscienceaimingataverycompletedescriptionofthebrainonsomesizeorfunctional scale.Third,WBEislargescaleneuroscience,requiringmethodsofautomating neuroscientificinformationgatheringandexperimentation.Thiswillreducecostsand increasethroughput,andisnecessaryinordertohandlethehugevolumesofdataneeded. Largescale/industrialneuroscienceisclearlyrelevantforotherneuroscienceprojectstoo.
17
Preparation
Scanning
Physical handling
Resolution
Imaging
Volume
Functional information
Noise removal
Tracing
Parameter estimation Synapse identification Scan interpretation Connectivity identification Cell type identification Databasing
Figure4:TechnologicalcapabilitiesneededforWBE.
18
Table3:CapabilitiesneededforWBE
Preprocessing/fixation Preparingbrainsappropriately, retainingrelevant microstructureandstate Methodsofmanipulatingfixed brainsandtissuepiecesbefore, during,andafterscanning Capabilitytoscanentirebrain volumesinreasonabletime andexpense. Scanningatenoughresolution toenablereconstruction Scanningisabletodetectthe functionallyrelevantproperties oftissue Handlingdistortionsdueto scanningimperfection Handlingmissingdata Improvingscanquality Detectingstructureand processingitintoaconsistent 3Dmodelofthetissue Identifyingcelltypes Identifyingsynapsesandtheir connectivity Estimatingfunctionally relevantparametersofcells, synapses,andotherentities Storingtheresultinginventory inanefficientway Modelofentitiesandtheir behaviour Implementationofmodel Storageoforiginalmodeland currentstate Efficientinterprocessor communication Processorpowertorun simulation Simulationofbodyenabling interactionwithvirtual environmentorthroughrobot Virtualenvironmentforvirtual body
Physicalhandling Scanning
Volume Imaging Resolution Functionalinformation Geometricadjustment Imageprocessing Datainterpolation Noiseremoval Tracing Celltypeidentification
Translation Scaninterpretation
Simulation
Linkages
MostofthecapabilitiesneededforWBEareindependentofeachother,orformsynergistic clusters.Clustersoftechnologiesdeveloptogether,supportingandmotivatingeachother withtheiroutput.Atypicalexampleisbettermathematicalmodelsstimulatinganeedfor betterimplementationsandcomputingcapacity,whileimprovementsinthelattertwo stimulateinterestinmodelling.Anotherkeyclusteris3Dmicroscopyandimageprocessing, whereimprovementsinonemakestheothermoreuseful. Therearefewclearcaseswhereacapabilityneedsacompletedearliercapabilityinorderto begindevelopment.Currentfixationandhandlingmethodsarelikelyunabletomeetthe
19
demandsofWBElevel3Dmicroscopy,butaregoodenoughtoenableearlydevelopmentfor certainsmallsystems.Scaninterpretationneedsenoughscandatatodevelopmethods,but givencurrentresearchthebottleneckappearstobemoreontheimageprocessingand interpretationsidethandataavailability.Achievinglargevolumescanningrequires parallelizationandscalinguppreviousscanningmethods,forexamplebyusingroboticwork andparallelmicroscopy.Thisrequiresresearchersthinkingintermsof,andhaving experiencewith,anindustrialapproachtodatacollection. Thisinterlinkednatureofthefieldavoidsanyobvioustechnologythresholdsand bottlenecks.Thereisnoonetechnologythatmustbedevelopedbeforeothertechnologiescan advance.Developmentcanoccuronabroadfrontsimultaneously,andrapidprogressina fieldcanpromotefeedbackprogressinrelatedfields.Unfortunately,italsomeansthatslow progressinoneareamayholdbackotherareas,notjustduetolackofresultsbutbyreduced demandfortheirfindings,reducedfunding,andfocusonresearchthatdoesnotleadinthe directionofWBE.
Roadmap
Basedontheseconsiderationswecansketchoutaroadmapwithmilestones,required technologies,keyuncertaintiesandexternaltechnologyinteractions. ApproachtoWBEhastwophases.Thefirstphaseconsistsofdevelopingthebasiccapabilities andsettlingkeyresearchquestionsthatdeterminethefeasibility,requiredlevelofdetailand optimaltechniques.Thisphasemainlyinvolvespartialscans,simulationsandintegrationof theresearchmodalities. Thesecondphasebeginsoncethecoremethodshavebeendevelopedandanautomated scaninterpretationsimulatepipelinehasbeenachieved.Atthispointthefirstemulations becomepossible.Ifthedevelopedmethodsprovetobescalabletheycanthenbeappliedto increasinglycomplexbrains.Herethemainissueisscalinguptechniquesthathavealready beenprovenonthesmallscale.
20
Human emulation
Validation methods
Deducing function Invertebrate emulation Scanning development Complete inventory Partial emulations Interpretation development Automated pipeline Eutelic organism emulation
Low-level neuroscience
Organism simulation
Body simulation
Simulation hardware
21
Bodysimulation:anadequatesimulationforthemodelanimalsbody(andenvironment). Ideallydemonstratedbyfoolingarealanimalconnectedtoit. Simulationhardware:specialpurposesimulation/emulationcomputerhardwaremaybe foundtobenecessaryoreffective. Organismsimulation:asimulationofanentireorganismintermsofneuralcontrol,body stateandenvironmentalinteraction.Thiswouldnotbeatrueemulationsinceitisnotbased onanyindividualbutratherknownphysiologicaldataforthespecies.Thiswouldenable morerealisticandindividualmodelsasscans,modelsandcomputerpowerimproves. Demonstrationoffunctiondeduction:demonstratingthatallrelevantfunctionalproperties onalevelcanbededucedfromscandata. Completeinventory:acompletedatabaseofentitiesatsomelevelofresolutionforaneural system,e.g.notjusttheconnectivityoftheC.elegansnervoussystembutalsothe electrophysiologyofthecellsandsynapses.Thiswouldenablefullemulationifalltheupdate rulesareknown.Itdemonstratesthatthescanningandtranslationmethodshavematured. Automatedpipeline:asystemabletoproduceasimulationbasedonaninputtissuesample, goingthroughthescan,interpretationandsimulationstepswithoutmajorhuman intervention.Theresultingsimulationwouldbebasedontheparticulartissueratherthan beingagenericmodel. Partialemulation:Acompleteemulationofneuralsystemsuchastheretina,invertebrate gangliaoraV1circuitbasedonscannedandinterpreteddatafromabrainratherthanspecies data.Thiswoulddemonstratethefeasibilityofdatadrivenbrainemulation. Eutelicorganismemulation:acompleteemulationofasimpleorganism,suchasC.elegansor anothereutelic(fixednervoussystem)organismusingdatafrompipelinescanning.Itmay turnoutthatitisunnecessarytostartwithaeutelicorganismandthefirstorganism emulationwouldbeamorecomplexinvertebrate. InvertebrateWBE:Emulationofaninvertebratesuchasasnailoraninsect,withlearning. ThiswouldtestwhethertheWBEapproachcanproduceappropriatebehaviours.Ifthe scannedindividualwastrainedbeforescanning,retentionoftrainedresponsescanbe checked. SmallmammalWBE:DemonstrationofWBEinmiceorrats,provingthattheapproachcan handlemammalianneuroanatomy. LargemammalWBE:Demonstrationinhighermammals,givingfurtherinformationabout howwellindividuality,memoryandskillsarepreservedaswellasinvestigationofsafety concerns. HumanWBE:Demonstrationofaninteractivehumanemulation.
22
Technology drivers
Preparation
Scanning
Physical handling
Resolution
Imaging
Volume
Functional information
Noise removal
Tracing
Parameter estimation Synapse identification Scan interpretation Connectivity identification Cell type identification Databasing
Commercial drivers
Research drivers
WBE specific?
Figure6:TechnologydriversforWBEnecessarytechnologies. Differentrequiredtechnologieshavedifferentsupportanddriversfordevelopment.
23
Computersaredevelopedindependentlyofanyemulationgoal,drivenbymassmarket forcesandtheneedforspecialhighperformancehardware.Mooreslawandrelated exponentialtrendsappearlikelytocontinuesomedistanceintothefuture,andthefeedback loopspoweringthemareunlikelytorapidlydisappear(seefurtherdiscussioninAppendixB: ComputerPerformanceDevelopment).Thereisindependent(andoftensizeable)investment intocomputergames,virtualreality,physicssimulationandmedicalsimulations.Like computers,thesefieldsproducetheirownrevenuestreamsanddonotrequireWBEspecific orscientificencouragement. Alargenumberoftheothertechnologies,suchasmicroscopy,imageprocessing,and computationalneurosciencearedrivenbyresearchandnicheapplications.Thismeansless funding,morevariabilityofthefunding,anddependenceonsmallergroupsdeveloping them.Scanningtechnologiesaretiedtohowmuchmoneythereisinresearch(including brainemulationresearch)unlessmedicalorotherapplicationscanbefound.Validation techniquesarenotwidelyusedinneuroscienceyet,butcould(andshould)becomestandard assystemsbiologybecomesmorecommonandwidelyapplied. FinallythereareafewareasrelativelyspecifictoWBE:largescaleneuroscience,physical handlingoflargeamountsoftissueblocks,achievinghighscanningvolumes,measuring functionalinformationfromtheimages,automatedidentificationofcelltypes,synapses, connectivityandparameters.Theseareasaretheonesthatneedmostsupportinorderto enableWBE. Thelattergroupisalsothehardesttoforecast,sinceithasweakdriversandasmallnumber ofresearchers.Thefirstgroupiseasiertoextrapolatebyusingcurrenttrends,withthe assumptionthattheyremainunbrokensufficientlyfarintothefuture.
24
Howmuchofthefunctionallyrelevantinformationcanbededucedfromscanningina particularmodality(e.g.electronmicroscopy)?Atpresent,electronmicroscopyappearsto betheonlyscanningmethodthathastherightresolutiontoreachsynapticconnectivity,but itislimitedinwhatchemicalstateinformationitcanreveal.Ifitispossibletodeducethe functionofaneuron,synapseorotherstructurethroughimageinterpretationmethods,then scanningwouldbefarsimplerthanifthisisnot(inwhichcasesomeformofhybridmethod orentirelynewscanningmodalitywouldhavetobedeveloped).Thisissueappearstoforma potentiallywelldefinedresearchquestionthatcouldbepursued.Answeringitwouldrequire findingasuitablemodelsystemforwhichgroundtruth(thecomputationalfunctionalityof targetsystem)wasknown,usingthescanningmodalitytoproduceimageryandthentesting outvariousformsofinterpretationonthedata.
Cellular data and hypotheses Qualitative modelling Quantitative modelling
Wet experiments
Large-scale scanning
Cell programming
Figure7:WBEcomputationalbiologyresearchcycle(basedon(Takahashi,Yugietal. 2002)).WBEintroduceslargescalescanningandsimulationintothecycle. Developingtherightresearchcycle.Thecomputationalbiologyresearchcycletoday involveswetexperimentsprovidingcellulardataandhypotheses,whichdrivequalitative modelling.Thismodellinginturnisusedinquantitativemodelling,whichusingsimulations generatedatathatcanbeanalysedandemployedtorefinethemodels,comparewiththe experiments,andsuggestnewexperiments(Takahashi,Yugietal.,2002).TheWBEparadigm incorporatesthisresearchcycle(especiallyinthesoftwaremodellingpart),butincludestwo newfactors.Oneislargescalescanningandprocessingofbraintissue,providingmassive amountsofdataasinputtothecycle,butalsorequiringthemodelstoguidethedevelopment ofscanningmethods.Thesecondisthelargescaledatadrivensimulationsthatdonotaimat producingjusthypothesistestingandmodelrefinement,butalsoataccuratelymimickingthe wetsystem.Bothfactorswillincreaseandchangethedemandsfordatamanagement, hypothesissearchingandsimulationanalysis/interpretation.Theywillalsointroduce sociologicalandinterdisciplinaryfactors,asdifferentacademicdisciplineswithverydifferent methodologieswillhavetolearnhowtocommunicateandcooperate.Inordertobeviable, fieldresearchmethodsoftesting,datasharing,validation,andstandardsforwhatconstitutes aresultmustbedevelopedsothattheextendedcycleprovidesincentivesforallparticipants tocooperateandpushthetechnologyforward.Thisiscloselylinkedtothelikelymove
25
towardslargescaleneuroscience,whereautomatedmethodswillplayanincreasingly prominentroleastheyhavedoneingenomics.
Figure8:Caenorhabditiselegans,apopularmodelorganismwithafullymapped302 neuronnervoussystem. Selectionofsuitablemodelsystems.Selectingtherighttargetsforscanningandmodelling willhavetotakeintoaccountexistingknowledge,existingresearchcommunities,likelihood offundingandacademicimpactaswellaspracticalfactors.WhiletheC.elegansnervous systemhasbeencompletelymapped(White,Southgateetal.,1986),westilllackdetailed electrophysiology,likelybecauseofthedifficultyofinvestigatingthesmallneurons.Animals withlargerneuronsmayprovelessrestrictiveforfunctionalandscanninginvestigationbut maylacksizeableresearchcommunities.Molluscssuchasfreshwatersnailsandinsectssuch asfruitfliesmaybepromisingtargets.Theyhavewellcharacterisedbrains,existingresearch communitiesandneuralnetworkswellwithincurrentcomputationalcapabilities. Similarly,theselectionofsubsystemsofthebraintostudyrequirescarefulconsideration. Someneuralsystemsareheavilystudied(corticalcolumns,thevisualsystem,the hippocampus)andbetterdataaboutthemwouldbewarmlyreceivedbytheresearch community,yetthelackofcharacterizationoftheirinputs,outputsandactualfunctionmay makedevelopmentofemulationmethodsveryhard.Onesystemthatmaybeverypromising istheretina,whichhasanaccessiblegeometry,iswellstudiedandsomewhatwell understood,isnotexcessivelycomplex,andbetterinsightsintowhichwouldbeusefultoa wideresearchcommunity.Buildingonretinalmodels,modelsofthelateralgeniculate nucleusandvisualcortexmaybeparticularlyuseful,sincetheywouldbothhaverelatively welldefinedinputsfromthepreviousstages. AtwhatpointwillthepotentialbeclearenoughtobringmajoreconomicactorsintoWBE? GiventhepotentiallyhugeeconomicimpactofhumanWBE(Hanson,1994,2008a,2004, 2008b),ifthefieldshowssufficientpromise,majoreconomicactorswillbecomeinterestedin fundinganddrivingtheresearchasaninvestment.Itisunclearhowfaradvancedthefield wouldneedtobeinordertogarnerthisattention.Soliddemonstrationsofkeytechnologies arelikelyrequired,aswellasaplausiblepathtowardsprofitableWBE.Theimpactoffunding onprogresswilldependonthemainremainingbottlenecksandtheirfundingelasticity.If scanningthroughputorcomputerpoweristhelimitingfactor,extrafundingcanrelatively easilyscaleupfacilities.Bycontrast,limitationsinneuroscienceunderstandingareless responsivetoinvestment.Iffundingarriveslate,whenthefundamentalproblemshave alreadybeensolved,theamplifiedresourceswouldbeusedtoscaleuppreexistingsmall scaledemonstrationprojects.
26
Alternative pathways
SpecialhardwareforWBE.ItispossiblethatWBEcanbeachievedmoreefficientlyusing dedicatedhardwareratherthangenerichardware.Suchperformancegainsarepossibleif,for example,thereisaclosemappingbetweenthehardwareandthebrain,orifthefunctionsof theemulationsoftwarecouldbeimplementedefficientlyphysically.Developingsuch dedicatedhardwarewouldbecostlyunlessotherapplicationsexisted(whichiswhyinterest indedicatedneuralnetworkchipspeakedintheearly1990s). Dedicatedneuralnetworkchipshavereachedupto1.7billionsynapticupdates(and337 millionsynapticadjustments)persecondforANNmodels(Kondo,Koshibaetal.,1996), whichisapproachingcurrentsupercomputingspeedsformorecomplexmodels.Recently, therehasbeensomedevelopmentofFPGAs(FieldProgrammableGateArrays)forrunning complexneuronsimulations,producinganorderofmagnitudefastersimulationfora motorneuronthanasoftwareimplementation(fourtimesrealtime,8Mcompartments/s) (WeinsteinandLee,2005).AFPGAimplementationhastheadvantageofbeing programmable,notrequiringWBEspecialpurposehardware.Anotheradvantageinclude thataslongasthereischipspace,morecomplexmodelsdonotrequiremoreprocessingtime andthatprecisioncanbeadjustedtosuitthemodelandreducespacerequirements. However,scalinguptolargeanddenselyinterconnectednetworkswillrequiredeveloping newtechniques(WeinsteinandLee,2006).Abetterunderstandingoftheneocortical architecturemayservetoproducehardwarearchitecturesthatfititwell(Daisyproject,2008). IthasbeensuggestedthatusingFPGAscouldincreasecomputationalspeedsinnetwork simulationsbyuptotwoordersofmagnitude,andinturnenabletestinggroundsfor developingspecialpurposeWBEchips(Markram,2006). Itmayalsobepossibletouseembeddedprocessortechnologytomanufacturelargeamounts ofdedicatedhardwarerelativelycheaply.Astudyofhighresolutionclimatemodellinginthe petafloprangefounda24to34foldreductionofcostandabouttwoordersofmagnitude smallerpowerrequirementsusingacustomvariantofembeddedprocessorchips(Wehner, Olikeretal.,2008). Thisroadmapisroughlycentredontheassumptionthatscanningtechnologywillbesimilar tocurrentmicroscopy,developedforlargescaleneuroscience,automatedsectioningof fixatedtissueandlocalimagetomodelconversion.Forreasonsdiscussedinthescanning section,nondestructivescanningoflivingbrainsappearstobehardcomparedtotheslice anddiceapproachwherewehavevarioussmallscaleexistenceproofs.However,aspointed outbyRobertFreitasJr.,nanomedicaltechniquescouldpossiblyenablenondestructive scanningbyuseofinvasivemeasurementdevices.Evenifsuchdevicesproveinfeasible, molecularnanotechnologycouldlikelyprovidemanynewscanningmethodologiesaswell asradicalimprovementofneuroscientificresearchmethodsandtheefficiencyofmany roadmaptechnologies.Evenfarmoremodestdevelopmentssuchassinglemoleculeanalysis, nanosensors,artificialantibodiesandnanoparticlesforimaging(whichareexpectedtobein useby2015(NanoroadmapProject,2006))wouldhaveanimportantimpact.Henceearlyor verysuccessfulnanotechnologywouldofferfasterandalternativeroutestoachievethe
27
roadmap.Analysingthelikelihood,timeframe,andabilitiesofsuchnanomedicineisoutside thescopeofthisdocument. Onepossiblescanningalternativenotexaminedmuchhereishighresolutionscanningof frozenbrainsusingMRI.Thismightbeacomplementtodestructivescanning,butcould possiblygainenoughinformationtoenableWBE.However,wecurrentlyhavelittle informationonthelimitsandpossibilitiesofthetechnique(seediscussioninAppendixE: Nondestructiveandgradualreplacement). Asdiscussedintheoverview,WBEdoesnotassumeanyneedforhighlevelunderstanding ofthebrainormind.Infact,shouldsuchunderstandingbereacheditislikelythatitcouldbe usedtoproduceartificialintelligence(AI).HumanlevelAI(orsuperintelligentAI)would notnecessarilyprecludeWBE,butsomeofthescientificandeconomicreasonswouldvanish, possiblymakingthefieldlessrelevant.Ontheotherhand,powerfulAIcouldgreatly accelerateneuroscienceadvancesandperhapshelpdevelopWBEforotherpurposes. Conversely,successinsomepartsoftheWBEendeavourcouldhelpAI,forexampleif corticalmicrocircuitryandlearningrulescouldbesimulatedefficientlyasageneral learning/behavioursystem. TheimpactanddevelopmentofWBEwilldependonwhichofthemaincapabilities (scanning,interpretation,simulation)developlast.Iftheydeveloprelativelyindependently itwouldbeunlikelyforallthreetomatureenoughtoenablehumanlevelemulationsatthe sametime.Ifcomputingpoweristhelimitingfactor,increasinglycomplexanimalemulations arelikelytoappear.SocietyhastimetoadapttotheprospectofhumanlevelWBEinthenear future.Ifscanningresolution,imageinterpretation,orneuralsimulationisthelimitingfactor, arelativelysuddenbreakthroughispossible:thereisenoughcomputingpower,scanning technology,andsoftwaretogorapidlyfromsimpletocomplexorganisms,usingrelatively smallcomputersandprojects.Thiscouldleadtoasurprisescenariowhereinsocietyhaslittle timetoconsiderthepossibilityofhumanlevelWBE.Ifcomputingpoweristhelimiting factor,orifscanningisthebottleneckduetolackofthroughput,thenthepaceof developmentwouldlikelybeeconomicallydetermined:ifenoughinvestmentweremade, WBEcouldbeachievedrapidly.ThiswouldplaceWBEenablementunderpoliticalor economicalcontroltoagreaterdegreethaninthealternativescenarios.
28
funding)aswellasbytradeoffsbetweenperformancewithprice,energyrequirements3,and otherconstraints.Therelatedareaofverylargescaleinformationstorageisalsoakeyissue forWBE. Anobviousrelatedtechnologyisthecreationofvirtualbodymodelsforuseinmedicine. Theycanbeusedastrainingandstudyobjects,or,atamoreadvancedstage,aspersonal medicalmodels.Suchmodelsmightenabledoctorstoinvestigatethecurrentstateofthe patient,compareittopreviousdata,testoroptimisevariousformsofsimulatedtreatment, trainparticularsurgicalapproaches,etc.Thisisatechnologyofsignificantpractical importancethatisdrivenbycurrentadvancesinmedicalimaging,thepersonalisationof healthcare,andimprovingphysiologicalmodels.Whilemostcurrentmodelsareeitherstatic datasetsorhighlevelphysiologicalmodels,personalisationrequiresdevelopingmethodsof physiologicalparameterestimationfromthedata.Simulationwillpromotethedevelopment ofmorerealisticbodymodelsusableforWBE.Conversely,theWBEfocusondatadriven bottomupsimulationappearstofitinwithdevelopingpersonalisedbiochemicalandtissue models. Virtualtestanimals,iftheycouldbedevelopedtoasufficientdegreeofrealism,wouldbe highindemandasfasterandlessexpensivetestinggroundsforbiomedicalhypotheses (MichelsonandCole,2007;Zheng,Kreuweletal.,2007).Theymayperhapsalsobeawayof avoidingtheethicalcontroversiessurroundinganimaltesting(althoughitisnot inconceivablethatconcernsaboutanimalwelfarewouldintimebeextendedtoemulated animals).Thiscouldprovideanimpetusforthedevelopmentoforganismemulation techniquesandespeciallyvalidationmethodsthathelpguaranteethatthevirtualanimals havethesamepropertiesasrealanimalswould. Overall,thereisincreasinginterestandcapabilityinquantitativemodellingofbiology(Di Ventura,Lemerleetal.,2006).Whilemuchefforthasgoneintometabolicorcontrolnetworks ofparticularinterest,thereisapushtowardscellmodelsencompassingmetabolism,the genomeandproteomics(Ortoleva,Berryetal.,2003;Tomita,2001;Schaff,Slepchenkoetal., 2001;Tyson,2001).Thereisalsointerestinbuildingsimulatorsformodellingselfassemblyin subcellularsystems(Ortoleva,Berryetal.,2003).Besidespredictingbiologicalresponsesand helpingtounderstandbiology,modellingwilllikelybecomeimportantforsyntheticbiology (Serrano,2007). Inordertoproducerealisticvirtualstimuliforabrainemulation,accuratesimulationsofthe sensoryinputtothebrainareneeded.Thesametechniquesusedtoinvestigateneural functioncanbeappliedtothesenses.Ithasalsobeenproposedthattechnologytorecord normalneuralactivityalongthebrainnervesandspinalcordcouldbehelpful.Suchdata,for examplerecordedusingmassivelyparallelelectrodearraysornanoimplants,wouldprovide goodtestdatatovalidateabodymodel.Recordedsensorydatawouldalsoberepeatable, enablingcomparisonsbetweendifferentvariantsofthesameemulation.Neuralinterfacingis alsousefulfordevelopingbetterroboticbodymodels.Currently,mostinterestinneural interfacesisfocusedonhelpingpeoplewithdisabilitiesusesensoryormotoricprosthetics. Whileneuralinterfacingforenhancementpurposesispossible,itisunlikelytobecomea significantdriveruntilprostheticsystemshavebecomecheap,safe,andveryeffective.
3
Anexascalecomputerusing2008technologywouldrequiretensofmegawatts(SandiaNationalLaboratories,2008).
29
Issues
Emulation systems
Afunctioningbrainemulationwillinclude,in additiontothebrainmodel(themainpart), somewayforthebrainmodeltoexperience bodilyinteractionswithanenvironment.There aretwodifferentwaysinwhichthiscouldbe accomplished:viaasimulatedvirtualbody inhabitingavirtualreality(whichcanbelinked totheoutsideworld);orviaahardwarebody connectedtothebrainmodelviaabody interfacemodule. EntrylevelWBEdoesnotrequirethecapacityto accommodatealloftheoriginalsensory modalitiesortoprovideafullynaturalisticbody experience.Simulatedbodiesandworlds,or hardwarebodyinterfaces,aswellas Figure9:Emulationsystemforcompletely communicationswiththeoutsideworld,arenot virtualemulation. necessaryperseforbrainemulationexcept insofartheyareneededtomaintainshorttermfunctionofthebrain.Forlongtermfunction, especiallyofhumanmindemulations,embodimentandcommunicationareimportant. Sensoryormotordeprivationappearstoproduceintellectualandperceptualdeficitswithina fewdaystime(ZubekandMacneill,1966). Thebrainemulatorperformstheactual emulationofthebrainandcloselylinked subsystemssuchasbrainchemistry.Theresult ofitsfunctionisaseriesofstatesofemulated brainactivity.Theemulationproducesand receivesneuralsignalscorrespondingtomotor actionsandsensoryinformation(inaddition, somebodystateinformationsuchasglucose levelsmaybeincluded). Thebodysimulatorcontainsamodelofthe bodyanditsinternalstate.Itproducessensory signalsbasedonthestateofthebodymodel andtheenvironment,sendingthemtothe brainemulation.Itconvertsmotorsignalsto musclecontractionsordirectmovementsinthe bodymodel.Thedegreetowhichdifferent Figure10:Emulationsystemforembodied partsofthebodyrequireaccuratesimulationis emulations. likelyvariable. Theenvironmentsimulatormaintainsamodelofthesurroundingenvironment,responding toactionsfromthebodymodelandsendingbacksimulatedsensoryinformation.Thisisalso
30
themostconvenientpointofinteractionwiththeoutsideworld.Externalinformationcanbe projectedintotheenvironmentmodel,virtualobjectswithrealworldaffordancescanbeused totriggersuitableinteractionetc. Theoverallemulationsoftwaresystem(theexoselftoborrowGregEgansterm)would regulatethefunctionofthesimulatorsandemulator,allocatecomputationalresources,collect diagnosticinformation,providesecurity(e.g.backups,firewalls,errordetection,encryption) andsoon.Itcouldprovidesoftwareservicestoemulatedminds(accessedthroughthevirtual environment)and/oroutsideexperimenters. Avariantoftheabovesystemwouldbeanembodiedbrainemulation,inwhichcasethe bodysimulatorwouldmerelycontainthetranslationfunctionsbetweenneuralactivityand physicalsignals,andthesewouldthenbeactuatedusingahardwarebody.Thebodymight becompletelyartificial(inwhichcasemotorsignalshavetobemappedontoappropriate bodybehaviours)orbiologicalbutequippedwithnervecomputerinterfacesenabling sensingandcontrol.Thecomputersystemrunningtheemulationdoesnothavetobe physicallypresentinthebody. Itiscertainlypossibletointroducesignalsfromtheoutsideonhigherlevelsthanina simulatedorrealbody.Itwouldberelativelytrivialtoaddvisualorauditoryinformation directlytothebodymodelandhavethemappearasvirtualoraugmentedreality. Introducingsignalsdirectlyintothebrainemulationwouldrequirethemtomakesenseas neuralsignals(e.g.brainstimulationorsimulateddrugs).Virtualbraincomputerinterfaces withperfectclarityandnoriskofsideeffectscouldbeimplementedasextensionsofthebody simulation/interface. Ifcomputingpowerturnsouttobeabottleneckresource,thenearlyemulationsarelikelyto runmoreslowlythanthebiologicalsystemtheyaimatemulating.Thiswouldlimitthe abilityoftheemulationstointeractinrealtimewiththephysicalworld.Indistributed emulationsdelaysbetweencomputingnodesputastronglimitonhowfasttheycanbecome. Theshortestdelayusing100m/saxonsacrossthebrainisabout1ms.Assuminglightspeed communication,processingnodescannotbefurtherawaythan300kmiflongerdelaysareto beavoided.
Spinalcord
Whiletraditionallythevertebratespinalcordisoftenregardedaslittlemorethanabundleof motorandsensoraxonstogetherwithacentralcolumnofstereotypicalreflexcircuitsand patterngenerators,thereisevidencethattheprocessingmaybemorecomplex(Berg, Alaburdaetal.,2007)andthatlearningprocessesoccuramongspinalneurons(Crown, Fergusonetal.,2002).Thenetworksresponsibleforstandingandsteppingareextremely flexibleandunlikelytobehardwired(Cai,Courtineetal.,2006).
31
Thismeansthatemulatingjustthebrainpartofthecentralnervoussystemwilllosemuch bodycontrolthathasbeenlearnedandresidesinthenonscannedcord.Ontheotherhand,it ispossiblethatagenericspinalcordnetworkwould,whenattachedtotheemulatedbrain, adapt(requiringonlyscanningandemulatingonespinalcord,aswellasfindingawayof attachingthespinalemulationtothebrainemulation).Butevenifthisistrue,thetimetaken maycorrespondtorehabilitationtimescalesof(subjective)months,duringwhichtimethe simulatedbodywouldbeessentiallyparalysed.Thismightnotbeamajorproblemfor personalidentityinmindemulations(sincepeoplesufferingspinalinjuriesdonotlose personalidentity),butitwouldbeamajorlimitationtotheirusefulnessandmightlimit developmentofanimalmodelsforbrainemulation. Asimilarconcerncouldexistforotherperipheralsystemssuchastheretinaandautonomic nervoussystemganglia. Thehumanspinalcordweighs2.5%ofthebrainandcontainsaround104ofthenumberof neuronsinthebrain(13.5millionneurons).Henceaddingthespinalcordtoanemulation wouldaddanegligibleextrascanandsimulationload.
Synapticadaptation
Synapsesareusuallycharacterizedbytheirstrength,thesizeofthepostsynapticpotential theyproduceinresponsetoagivenmagnitudeofincomingexcitation.Many(most?) synapsesintheCNSalsoexhibitdepressionand/orfacilitation:atemporarychangeinrelease probabilitycausedbyrepeatedactivity(Thomson,2000).Thisrapiddynamicslikelyplaysa roleinavarietyofbrainfunctions,suchastemporalfiltering(FortuneandRose,2001), auditoryprocessing(Macleod,Horiuchietal.,2007)andmotorcontrol(NadimandManor, 2000).Thesechangesoccurontimescaleslongerthanneuralactivity(tensofmilliseconds)but shorterthanlongtermsynapticplasticity(minutestohours).Adaptationhasalreadybeen includedinnumerouscomputationalmodels.Thecomputationalloadisusually13extra statevariablesineachsynapse.
Unknownneurotransmittersandneuromodulators
Notallneuromodulatorsareknown.Atpresentabout10majorneurotransmittersand200+ neuromodulatorsareknown,andthenumberisincreasing.(Thomas,2006)lists272 endogenousextracellularneuroactivesignaltransducerswithknownreceptors,2gases,19 substanceswithputativeorunknownbindingsitesand48endogenoussubstancesthatmay ormaynotbeneuroactivetransducers(manyofthesemaybemoreinvolvedingeneral biochemicalsignallingthanbrainspecificsignals).Plottingtheyearofdiscoveryfordifferent substances(orfamiliesofsubstances)suggestsalinearorpossiblysigmoidalgrowthover time(Figure11).
32
Ghrelin Kisspeptin Orexin NPFF Nociceptin Anandamide Apelin PACAP NO FGF ANF MCH Galanin Bombesin GHRH NPY Dynorphin PDML VIP Endorphin CCK Histamine Neurotensin Enkephalin Somatostatin Substance P GnRH TRH Glycine Dopamine Angiotensin Calcitonin Serotonin Glutamate GABA CRF ACTH Noradrenaline Acetylcholine 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
Figure11:Timeofdiscoveryoftheneurotransmitterorneuromodulatoractivityofa numberofsubstancesorfamiliesofsubstances.Datatakenfrom(vonBohlenundHalbach andDermietzel2006),likelyunderrepresentingthedevelopmentafter2000. Anupperboundonthenumberofneuromodulatorscanbefoundusinggenomics.About800 Gproteincoupledreceptorscanbefoundinthehumangenome,ofwhichabouthalfwere sensoryreceptors.Manyareorphansthatlackknownligands,andmethodsof deorphanizingreceptorsbyexpressingthemanddeterminingwhattheybindtohavebeen developed.Inthemiddle1990sabout150receptorshadbeenpairedto75transmitters, leavingaround150200orphansin2003(Wise,Jupeetal.,2004).Atpresent,78receptorsare deorphanizedeachyear(vonBohlenundHalbachandDermietzel,2006);atthisrateall orphansshouldbeadoptedwithin20years,leadingtothediscoveryofaround50more transmitters(Civelli,2005). Similarlyguanylylcyclasecoupledreceptors(fourorphans,(WedelandGarbers,1998)), tyrosinekinasecoupledreceptors(<<100,(MullerTidow,Schwableetal.,2004))andcytokine receptorswouldaddafewextratransmitters. However,thereisroomforsomesurprises.Recentlyitwasfoundthatprotonswereusedto signalinC.elegansrhythmicdefecation(Pfeiffer,Johnsonetal.,2008)mediatedusinga Na+/H+exchanger,anditisnotinconceivablethatsimilarmechanismscouldexistinthe brain.Hencetheupperboundonalltransmittersmaybesetbynotjustreceptorsbutalsoby membranetransporterproteins.
33
Unknownionchannels
Similartoreceptors,therearelikelyunknownionchannelsthataffectneurondynamics. TheLigandGatedIonChannelDatabasecurrentlycontains554entrieswith71designatedas channelsubunitsfromHomosapiens(EMBLEBI,2008;Donizelli,Djiteetal.,2006).Voltage gatedionchannelsformasuperfamilywithatleast143genes(Yu,YarovYarovoyetal., 2005).Thisdiversityisincreasedbymultimerization(combinationsofdifferentsubunits), modifiersubunitsthatdonotformchannelsontheirownbutaffectthefunctionofchannels theyareincorporatedinto,accessoryproteinsaswellasalternatemRNAsplicingandpost translationalmodification(Gutman,Chandyetal.,2005).Thiswouldenableatleastanorder ofmagnitudemorevariants. Ionchanneldiversityincreasesthediversityofpossibleneuronelectrophysiology,butnot necessarilyinalinearmanner.Seethediscussionofinferringelectrophysiologyfromgene transcriptsintheinterpretationchapter.
Volumetransmission
Surroundingthecellsofthebrainistheextracellularspace,onaverage200acrossand correspondingto20%ofbrainvolume(Nicholson,2001).Ittransportsnutrientsandbuffers ions,butmayalsoenablevolumetransmissionofsignallingmolecules. Volumetransmissionofsmallmoleculesappearsfairlywellestablished.Nitrousoxideis hydrophobicandhaslowmolecularweightandcanhencediffuserelativelyfreelythrough membranes:itcanreachupto0.10.2mmawayfromareleasepointunderphysiological conditions(Malinski,Tahaetal.,1993;SchumanandMadison,1994;WoodandGarthwaite, 1994).Whilemainlybelievedtobeimportantforautoregulationofbloodsupply,itmayalso havearoleinmemory(Ledo,Fradeetal.,2004).ThismightexplainhowLTP(LongTerm Potentiation)caninducecrosstalkthatreducesLTPinductionthresholdsoveraspanof10 mandtenminutes(HarveyandSvoboda,2007). Signalsubstancessuchasdopamineexhibitvolumetransmission(Rice,2000)andthismay haveeffectforpotentiationofnearbysynapsesduringlearning:simulationsshowthata singlesynapticreleasecanbedetectedupto20mawayandwitha100mshalflife(Cragg, Nicholsonetal.,2001).Largermoleculeshavetheirrelativediffusionspeedreducedbythe limitedgeometryoftheextracellularspace,bothintermsofitstortuosityanditsanisotropy (Nicholson,2001).AssuggestedbyRobertFreitas,theremayalsoexistactiveextracellular transportmodes.DiffusionratesarealsoaffectedbylocalflowoftheCSFandcandifferfrom regiontoregion(FenstermacherandKaye,1988);ifthisisrelevantthenlocaldiffusionand flowmeasurementsmaybeneededtodevelopatleastageneralbraindiffusionmodel.The geometricpartofsuchdatacouldberelativelyeasilygainedfromthehighresolution3D scansneededforotherWBEsubproblems.
34
Bodychemicalenvironment
Thebodyactsasaninput/outputunitthatinteractswithourperceptionandmotoractivity.It alsoactsasachemicalenvironmentthataffectsthebrainthroughnutrients,hormones, salinity,dissolvedgases,andpossiblyimmunesignals.Mostofthesechemicalsignalsoccur onasubconsciouslevelandonlybecomeapparentwhentheyinfluencee.g.hypothalamusto producehungerorthirstsensations.Forbrainemulation,someaspectsofthischemical environmenthastobesimulated. Thiswouldrequiremappingthehumanmetabolome,atleastinregardstosubstancesthat crossthebloodbrainbarrier.Themetabolomeislikelyontheorderof2,0002,500 compounds(Beecher,2003;Wishart,Tzuretal.,2007)andlargelydoesnotchangemore rapidlythanonthesecondtimescale.Thissuggeststhatcomparedtothedemandsofthe WBE,thebodychemistrymodel,whileinvolved,wouldberelativelysimple. Ifaproteininteractionmodelisneededratherthanmetabolism,thencomplexityincreases. Accordingtooneestimatethehumaninteractomeisaround650,000proteinprotein interactions(Stumpf,Thorneetal.,2008).
Neurogenesisandremodelling
Recentresultsshowthatneurogenesispersistsinsomebrainregionsinadulthood,andmight havenontrivialfunctionalconsequences(Saxe,Malleretetal.,2007).Duringneurite outgrowth,andpossiblyafterwards,celladhesionproteinscanaffectgeneexpressionand possibleneuronfunctionbyaffectingsecondmessengersystemsandcalciumlevels(Crossin andKrushel,2000).However,neurogenesisismainlyconfinedtodiscreteregionsofthebrain anddoesnotoccurtoagreatextentinadultneocortex(Bhardwaj,Curtisetal.,2006). Sinceneurogenesisoccursonfairlyslowtimescales(>1week)comparedtobrainactivityand normalplasticity,itcouldprobablybeignoredinbrainemulationifthegoalisanemulation thatisintendedtofunctionfaithfullyforonlyafewdaysandnottoexhibittrulylongterm memoryconsolidationoradaptation. Arelatedissueisremodellingofdendritesandsynapses.Overthespanofmonthsdendrites cangrow,retractandaddnewbranchtipsinacelltypespecificmanner(Lee,Huangetal., 2006).Similarlysynapticspinesintheadultbraincanchangewithinhourstodays,although themajorityremainstableovermultimonthtimespans(Grutzendler,Kasthurietal.,2002; Holtmaat,Trachtenbergetal.,2005;Zuo,Linetal.,2005).Evenifneurogenesisisignoredand theemulationisofanadultbrain,itislikelythatsuchremodellingisimportanttolearning andadaptation.
35
Gliacells
Gliacellshavetraditionallybeenregardedasmerelysupportingactorstotheneurons,but recentresultssuggestthattheymayplayafairlyactiveroleinneuralactivity.Besidethe importantroleofmyelinizationforincreasingneuraltransmissionspeed,attheveryleast theyhavestrongeffectsonthelocalchemicalenvironmentoftheextracellularspace surroundingneuronsandsynapses. Glialcellsexhibitcalciumwavesthatspreadalongglialnetworksandaffectnearbyneurons (NewmanandZahs,1998).Theycanbothexciteandinhibitnearbyneuronsthrough neurotransmitters(Kozlov,Anguloetal.,2006).Conversely,thecalciumconcentrationofglial cellsisaffectedbythepresenceofspecificneuromodulators(PereaandAraque,2005).This suggeststhattheglialcellsactsasaninformationprocessingnetworkintegratedwiththe neurons(FellinandCarmignoto,2004).Onerolecouldbeinregulatinglocalenergyand oxygensupply. Ifglialprocessingturnsouttobesignificantandfinegrained,brainemulationwouldhaveto emulatethegliacellsinthesamewayasneurons,increasingthestoragedemandsbyatleast oneorderofmagnitude.However,thetimeconstantsforglialcalciumdynamicsisgenerally farslowerthanthedynamicsofactionpotentials(ontheorderofsecondsormore), suggestingthatthetimeresolutionwouldnothavetobeasfine,makingthecomputational demandsincreasefarlesssteeply.
Ephapticeffects
Electricaleffectsmayalsoplayaroleviasocalledephaptictransmission.Inahigh resistanceenvironment,currentsfromactionpotentialsareforcedtoflowthrough neighbouringneurons,changingtheirexcitability.Ithasbeenclaimedthatthisprocess constitutesaformofcommunicationinthebrain,inparticularthehippocampus(Krnjevic, 1986).However,inmostpartsofthebrainthereisalargeextracellularspaceandblocking myelin,soevenifephapticinteractionsplayarole,theydosoonlylocally,e.g.inthe olfactorysystem(Bokil,Laarisetal.,2001),densedemyelinatednervebundles(Reutskiy, Rossonietal.,2003),ortrigeminalpainsyndromes(LoveandCoakham,2001).Itshouldbe notedthatthenervoussystemappearsrelativelyinsensitivetoeverydayexternalelectric fields(Valberg,Kavetetal.,1997;SwansonandKheifets,2006). Ifephapticeffectswereimportant,theemulationwouldneedtotakethelocallyinduced electromagneticfieldsintoaccount.Thiswouldplausiblyinvolvedividingtheextracellular space(possiblyalsotheintracellularspace)intofiniteelementswherethefieldcanbe assumedtobeconstant,linearorotherwiseeasilyapproximable.Thecorticalextracellular lengthconstantisonorderof100m(GardnerMedwin,1983),whichwouldnecessitateon theorderof1.41012suchcompartmentsifeachcompartmentis1/10ofthelengthconstant.
36
Dynamicalstate
Themethodsforcreatingthenecessarydataforbrainemulationdiscussedinthispaperdeal withjustthephysicalstructureofthebraintissue,notitsstateofactivity.Someinformation suchasworkingmemorymaybestoredjustasongoingpatternsofneuralexcitationand wouldbelost.Similarly,informationincalciumconcentrations,synapticvesicledepletion, anddiffusingneuromodulatorsmaybelostduringscanning.Alikelyconsequencewouldbe amnesiaofthetimeclosesttothescanning. However,lossofbrainactivitydoesnotseemtopreventthereturnoffunctionandpersonal identity.Thisisdemonstratedbythereawakeningofcomapatients,andbycoldwaternear drowningcasesinwhichbrainactivitytemporarilyceasedduetohypothermia(Elixson, 1991).
Quantumcomputation
Whilepracticallyallneuroscientistssubscribetothedogmathatneuralactivityisa phenomenonthatoccursonaclassicalscale,therehavebeenproposals(mainlyfrom physicists)thatquantumeffectsplayanimportantroleinthefunctionofthebrain(Penrose, 1989;Hameroff,1987).Sofarthereisnoevidenceforquantumeffectsinthebrainbeyond quantumchemistry,andnoevidencethatsucheffectsplayanimportantroleforintelligence orconsciousness(Litt,Eliasmithetal.,2006).Thereisnolackofpossiblecomputational primitivesinneurobiologynoranyphenomenathatappearunexplainableintermsof classicalcomputations(KochandHepp,2006).Quantitativeestimatesfordecoherencetimes forionsduringactionpotentialsandmicrotubulessuggestthattheydecohereonatimescale of10201013s,abouttenordersofmagnitudefasterthanthenormalneuralactivity timescales.Hencequantumeffectsareunlikelytopersistlongenoughtoaffectprocessing (Tegmark,2000).This,however,hasnotdeterredsupportersofquantumconsciousness,who arguethattheremaybemechanismsprotectingquantumsuperpositionsoversignificant periods(RosaandFaber,2004;Hagan,Hameroffetal.,2002). Ifthesequantummindhypothesesweretrue,brainemulationwouldbesignificantlymore complex,butnotimpossiblegiventheright(quantum)computer.In(Hameroff,1987)mind emulationisconsideredbasedonquantumcellularautomata,whichinturnarebasedonthe microtubulenetworkthattheauthorsuggestsunderliesconsciousness. Assuming7.1microtubulespersquaremand768.9minaveragelength(Cash,Alievetal., 2003)andthat1/30ofbrainvolumeisneurons(althoughgiventhatmicotubulinetworks occursinallcells,gliaandanyothercelltype!maycounttoo)gives1016microtubules.If eachstoresjustasinglequantumbitthiswouldcorrespondtoa1016qubitsystem,requiringa physicallyintractable210^16bitclassicalcomputertoemulate.Ifonlythemicrotubulesinsidea cellactasaquantumcomputingnetwork,theemulationwouldhavetoinclude1011 connected130,000qubitquantumcomputers.Anothercalculation,assumingmerelyclassical computationinmicrotubules,suggests1019bytesperbrainoperatingat1028FLOPS (Tuszynski,2006).Oneproblemwiththesecalculationsisthattheyimputesuchaprofoundly
37
largecomputationalcapacityatasubneurallevelthatamacroscopicbrainseemsunnecessary (especiallysinceneuronsaremetabolicallycostly).
Analogcomputation
Asurprisinglycommondoubtexpressedaboutthepossibilityofsimulatingevensimple neuralsystemsisthattheyareanalogratherthandigital.Thedoubtisbasedonthe assumptionthatthereisanimportantqualitativedifferencebetweencontinuousanddiscrete variables. Ifcomputationsinthebrainmakeuseofthefullpowerofcontinuousvariablesthebrainmay essentiallybeabletoachievehypercomputation,enablingittocalculatethingsanordinary Turingmachinecannot(Ord,2006;SiegelmannandSontag,1995).See(ZenilandHernandez Quiroz,2007)forareviewofhowdifferentbraincomputationalarchitectureswouldenable differentlevelsofcomputationalpower,andtherequirementsforneuralnetworksto simulatethese. However,brainsaremadeofimperfectstructureswhichare,inturn,madeofdiscreteatoms obeyingquantummechanicalrulesforcingthemintodiscreteenergystates,possiblyalso limitedbyaspacetimethatisdiscreteonthePlanckscale(aswellasnoise,seebelow)andso itisunlikelythatthehighprecisionrequiredofhypercomputationcanbephysicallyrealized (Eliasmith,2001).Evenifhypercomputationwerephysicallypossible,itwouldbynomeans becertainthatitisusedinthebrain,anditmightevenbedifficulttodetectifitwere(the continualandotherwisehardtoexplainfailureofWBEwouldbesomeevidenceinthis direction).However,findingclearexamplesofnonTuringcomputableabilitiesofthemind wouldbeawayofrulingoutTuringemulation. Adiscreteapproximationofananalogsystemcanbemadearbitrarilyexactbyrefiningthe resolution.IfanMbitvalueisusedtorepresentacontinuoussignal,thesignaltonoiseratio isapproximately20log10(2M)dB(assuminguniformdistributionofdiscretizationerrors, whichislikelyforlargeM).Thiscanrelativelyeasilybemadesmallerthanthenaturalnoise sourcessuchasunreliablesynapses,thermal,orelectricalnoise.Thethermalnoiseisonthe orderof4.21021J,whichsuggeststhatenergydifferencessmallerthanthiscanbeignored unlesstheyoccurinisolatedsubsystemsorontimescalesfastenoughtonotthermalize.Field potentialrecordingscommonlyhavefluctuationsontheorderofmillivoltsduetoneuron firingandabackgroundnoiseontheorderoftensofmicrovolts.Againthissuggestsalimit tothenecessaryprecisionofsimulationvariables4.
Determinism
Asomewhatrelatedcriticismistheassumeddeterminismofcomputers,whilethebrainis assumedeithertocontaintruerandomnessoraphysicallyindeterministicelement(often declaredtobefreewill). Therandomnessversionofthedeterminismcriticismcanbemetbyincludingsufficientnoise inthesimulation.Randomeventsmayplayaroleinthefunctionofthenervoussystem.In particular,thenumberofindividualmoleculesinvolvedintranscriptionregulationofsome proteinsandinthefunctionofsynapticspinesislowenoughthatindividualrandom interactionscanaffectwhetherageneisswitchedonorwhetheraphosphorylationcascade
4
Analogcomputationmaystillbeausefulhardwareparadigmundersomeconditions.
38
happens.Thisrandomnessmayhavebiologicalimportance,andissometimesincludedin biophysicalmodels.Anotherpossibleroleofnoisemightbestochasticresonance,wherenoise actstoincreasethesignaltonoiseratioinnonlinear(andinformationtheoretically suboptimal)systemsattemptingtodetectaweakperiodicinput(Gammaitoni,Hanggietal., 1998).Ithasbeendemonstratedinmechanoreceptors(DouglassandWilkens,1998)and variousotherformsofsensorysystems(DouglassandWilkens,1998)asawayofdetecting faintsignals.However,whiletheexactspectraldistributionandpowermaymatterforan emulation,itdoesnotseemthattheresonanceeffectwouldbedisruptedbyusing pseudorandomnoiseiftherecurrencetimeislongenough.Unlesstherearesomeimportant hiddenvariablesinthenoiseofthebrain,noisecanbeeasilyapproximatedusingasuitably longperiodicrandomnumbergenerator(Tegmark,2000)orevenbymeansofanattached physicalrandomnumbergeneratorusingquantummechanics(Stefanov,Gisinetal.,2000). RandomnessisthereforehighlyunlikelytoposeamajorobstacletoWBE. Hiddenvariablesorindeterministicfreewillappeartohavethesamestatusasquantum consciousness:whilenotinanyobviouswaydirectlyruledoutbycurrentobservations,there isnoevidencethattheyoccurorarenecessarytoexplainobservedphenomena.
Summary
Table4showsanoverviewofinformalestimatesofthelikeliehoodthatcertainfeaturesare neededforWBEandwhethertheywouldposeseriousimplementationproblemsifneeded. Table4:Likeliehoodestimatesofmodellingcomplications.
Spinalcord Synapticadaptation LikeliehoodneededforWBE Likely Verylikely Implementationproblems Minor.Wouldrequirescanning someextraneuraltissue. Minor.Introducesextrastate variablesandparametersthatneed tobeset. Minor.Similartoknown transmittersandmodulators. Minor.Similartoknownion channels. Medium.Requiresdiffusionmodels andmicroscalegeometry. Medium.Requiresmetabolomic modelsanddata. Medium.Requirescellmechanics andgrowthmodels. Minor.Wouldrequiremore simulationcompartments,butlikely runningonaslowertimescale. Medium.Wouldrequirerelatively finegrainedEMsimulation. Profound.Wouldprecludemost proposedscanningmethods. Profound.Wouldprecludecurrently conceivedscanningmethodsand wouldrequirequantumcomputing. Profound.Wouldrequireanalog computerhardware. Mediumtoprofound,dependingon whethermerelytruerandomnoise orhiddenvariablesareneeded.
Verylikely
Analogcomputation Truerandomness
Veryunlikely Veryunlikely
39
Scanning
Thefirststepinbrainemulationistoacquirethenecessaryinformationfromaphysicalbrain. Wewillcallthisstepscanning. Brainemulationforcompartmentmodelsofneuronactivityneedstoacquirebothgeometric dataaboutthelocalizationandmorphologyofthenervousconnectionsand functional/chemicaldataabouttheirnaturesuchaswhationchannels,receptors,and neurotransmittersarepresent,thepresenceofelectricalsynapses,electricalmembrane properties,phosphorylationstatesofsynapses,andgeneticexpressionstates.Thismustbe doneatasufficientresolution.Itmaybepossibletoinfersomefunctionalpropertiessuchas whetherasynapseisexcitatoryorinhibitorypurelyfromgeometry(e.g.,asynapsefroma smoothneuronwithaparticularmorphologyislikelyinhibitory).Yetitremainsunclearhow muchinformationaboutsynapticstrengthandneuromodulatortypecanbeinferredfrom puregeometryatagivenlevelofresolution. Thereareseveralpotentialapproachestoscanning.Herewefocusondestructivescanning,in whichthebrainisdestructivelydisassembledduringtheemulationprocess.Theprocess couldbeappliedimmediatelyafterdeathoroncryogenicallypreservedbraintissue.Thisis thetechnologicallysimplestapproach.Destructivescanninghasgreaterfreedombothin physicaleffectsused,energylevels,andfixatingthebrainthroughfreezingand/orchemical fixation.Possiblealternativesincludenondestructivescanningandgradualreplacement; thesearediscussedinAppendixE. Mostmethodsdiscussedinthissectionrequirethesectioningofthebrainintosmallerpieces thatcanbeaccuratelyscanned.Thesectioningmethodsandthehandlingofthetissuemay posesignificantproblemsandrequirethedevelopmentofspecialhandlingtechnology.In particular,sampledistortionanddriftarekeyproblems;thefirstrequiringverycareful sectioningmethods,thesecondeitherhighprecisionequipmentorembeddingoffiducial markers.
MRImicroscopy
AlthoughMRIimagingmaynotbesuitableforscanningentirebrainsatsufficientresolution, MRImicroscopymightbesuitableforscanningpartsofthemifwaterdiffusionisstoppedby freezingorfixation. 3DMRmicroscopyhasachievedresolutionsontheorderof3.73.33.3m(Ciobanu,Seeber etal.,2002)butislimitedbythesmallfieldofviewandlongexposuretimes.Onewayof improvingthefieldofviewmaybeparallelism,whereanarrayofMRIcoilsismovedover thesurface(McDougallandWright,2007). AcombinationofMRIandAFMismagneticresonanceforcemicroscopy(MRFM)wherea magneticbeadisplacedonanultrathincantileverandmovedoverthesample(orthe reverse).BygeneratingRFmagneticfieldsthespinsofnucleiwithintheresonantslice beneaththebeadcanbeflipped,producingforcesonthecantileverthatcanbedetected.This wouldenableidentificationofthemoleculespresentnearthesurface.Currentresolutions achievedare80nmvoxelsinascanvolumeof0.5m3(Chao,Doughertyetal.,2004)and singlespindetectionwith25nmresolutioninonedimension(Rugar,Budakianetal.,2004).
40
Whetherthiscanbescaleduptodetectinge.g.thepresenceofcellmembranesorparticular neurotransmittersremainstobeseen.
Opticalmethods
Figure12:Confocalmicroscopypictureofvisualcortexneurons.StainedwithGABA channelantibody(red)andcontainingneuronsexpressinggreenfluorescentprotein.The whitescalebaris100m.(Lee,Huangetal.,2006). Opticalmicroscopymethodsarelimitedbytheneedforstainingtissuestomakerelevant detailsstandoutandthediffractionlimitssetbythewavelengthoflight(0.2m).Themain benefitisthattheygowelltogetherwithvariousspectrometricmethods(seebelow)for determiningthecompositionoftissues. Subdiffractionopticalmicroscopyispossible,iflimited.Variousfluorescencebasedmethods havebeendevelopedthatcouldbeapplicableiffluorophorescouldbeattachedtothebrain tissueinawaythatprovidedtherelevantinformation.Structuredilluminationtechniques usepatternedilluminationandpostcollectionanalysisoftheinterferencefringesbetweenthe illuminationandsampleimagetogetherwithopticalnonlinearitiestobreakthediffraction limit.Thisway,50nmresolvingpowercanbeachievedinawidefield,atthepriceof photodamageduetothehighpowerlevels(Gustafsson,2005).Nearfieldscanningoptical microscopy(NSOM)usesamultinanometreopticfibertoscanthesubstrateusingnearfield optics,gainingresolution(downtothemultinanometerscale)andfreedomfromusing
41
fluorescentmarkersattheexpenseofspeedanddepthoffield.Itcanalsobeextendedinto nearfieldspectroscopy. Confocalmicroscopysuffersfromhavingtoscanthroughtheentireregionofinterestand qualitydegradesawayfromthefocalplane.Usinginversescatteringmethodsdepth independentfocuscanbeachieved(Ralston,Marksetal.,2007). Opticalhistologyanddissection Allopticalhistologyusesfemtosecondlaserpulsestoablatetissuesamples,avoidingthe needformechanicalremovalofthesurfacelayer(Tsai,Friedmanetal.,2003).Thistreatment appearstochangethetissue210mfromthesurface.However,Tsaietal.wereoptimistic aboutbeingabletoscanafluorescencelabelledentiremousebraininto2terapixelsatthe diffractionlimitofspatialresolution. Anotherinterestingapplicationoffemtosecondlaserpulsesismicrodissection(Sakakura, Kajiyamaetal.,2007;Colombelli,Grilletal.,2004).Thelaserwasabletoremove100m samplesfromplantandanimalmaterial,modifyinga~10mborder.Thisformofoptical dissectionmightbeanimportantcomplementforEMmethods,inthat,afterscanningthe geometryofthetissueatahighresolution,relevantpiecescanberemovedandanalyzed microchemically.ThiscouldenablegainingboththeEMconnectivitydataanddetailed biochemistryinformation.Platformsalreadyexistthatcanbothinjectbiomoleculesinto individualcells,performmicrodissection,isolateandcollectindividualcellsusinglaser catapulting,andsetupcomplexopticalforcepatterns(Stuhrmann,Jahnkeetal.,2006). KnifeEdgeScanningMicroscopy(KESM)
42
KESMisamethodforstainingandimaginglargevolumesathighresolutionsbyintegrating thesectioningandimagingstep.Itwasdevelopedforreconstructionandmodellingofthe threedimensionalanatomicalstructureofindividualcellsinsitu.Whilehavinglower resolutionthanSBFSEM(seebelow)itenablestheimagingofanentiremacroscopictissue volumesuchasamousebraininreasonabletime(McCormick,2002a;McCormick,2002b). TheKESMactssimultaneouslyasamicrotomeandamicroscope.Adiamondknifeisused forthecuttingandillumination,movingtogetherwiththemicroscopeacrossastationary tissuespecimenembeddedinplastic.Imagingoccursofthejustseparatedsectionalonga1D strip.Imagingat0.3mresolutionof0.5mthicksectionshasbeendemonstrated, producingdatasetswheremicrovasculatureandindividualcellscanbetracedusing contouringalgorithms.Theinstrumentscansatratesupto200Mpixels/s.Thiswouldenable scanningamousebrain(1cm3)inahundredhours,givinga15terabytedatasetof uncompresseddata(Mayerich,Abbottetal.,2008).Thewidthoffieldis2.5mmfor64m resolutionand0.625mmfor32mresolution(McCormick,Kohetal.,2004).Inorderto reducejitteringandusingkniveslargerthanthemicroscopefieldofview,thesampleiscutin astairstepmanner(KohandMcCormick,2003). Themainlimitationistheneedforstaininginsideavolume.Atpresentthenumberofsuch stainsislimited.Usingtransgenicanimalsexpressingstainsorfluorescencecouldenable mappingofdifferentneuralsystems.
XRaymicroscopy
43
couldbeidentified,andtheauthorsestimatethatthesetupcouldinprinciplescana1mm3 blockofmammaliannervetissuewith103104neurons.However,theresolutionislikelytoo crudetoidentifysynapsesandconnectivity. ByusingsoftXrayswithwavelengthsinthewaterwindow(betweentheabsorptionedges ofoxygenandcarbonat2.34.4nm)wherecarbonabsorbstentimesasstronglyaswater, organicstructurescanbeimagedwithgoodcontrastdowntoatleast30nmresolution. Advantagesincludethatitcanbedonewithoutspecialsamplepreparationproceduresand canbeappliedtohydratedcells,itpermitsthickerspecimensthanEM(upto10m),Xray absorptionspectraoflocalizedregionscanberecordedbyusingdifferentXrayenergies,and potentiallythreedimensionalimagingcouldbepossibleusingmultiplebeams(Yamamoto andShinohara,2002). AmajordisadvantageisthattheXrayscausetissuedamage.WhenthetotalXrayfluxgoes above4105photonsperm2,myofibrilslosetheircontractility;andabove106photonsper m2,yeastcellslosetheirdyeexclusionability(asignthattheyaredeadorpunctured).This islowerthanthefluxesofXraymicroscopeswith50nmresolution,makingscanningof tissuewhileretainingitsfunctionimpossible(Fujisaki,Takahashietal.,1996).Onewayof avoidingdegradationoftheimageistousearapidpulseofrays.Thisalsoavoidsmotion blurringduetodiffusioninthecaseofhydratedtissue:toimagefreelydiffusingobjectsof size10nmexposureshorterthan0.1msisneeded,whileacellularorganellewouldrequire between14ms1.4sdependingonsize(ItoandShinohara,1992).Usingvitrifiedtissuethe radiationdosecanbeincreasedthousandfoldwithoutstructuralchanges,andthediffusion issuedisappears(Methe,Springetal.,1997).TheseconcernsmakeXrayimagingunlikelyto workfornondestructivescanning. OfparticularinterestforWBEisthepossibilityofdoingspectromicrosopytodeterminethe localchemicalenvironment.Xrayabsorptionedgesareaffectedbythechemicalbindingstate oftheatomprobed.Differentaminoacidshavedistinguishablefingerprintsthatare relativelyunaffectedbypeptidebonding.Inprinciple,thespectraofproteinscouldbe predictedbasedontheirsequences.ScanningXraymicroscopeshavelongexposuretimes (minutes),althoughtheydeposit510timeslessradiationinthesample(Jacobsen,1999). WhilecurrentlyfartooslowforWBEpurposes,findingwaysofspeedingupthisprocess maymakeitrelevantforWBE.
Atomicbeammicroscopy
Insteadofphotonsorelectrons,neutralatomscouldbeusedtoimageasample.Sincethede Brogliewavelengthofthermalatomsissubnanometer,theresolutioncouldbeveryhigh.By usingunchargedandinertatomslikehelium,thebeamwouldalsobenondestructive(Holst andAllison,1997).Heliumatomscatteringhasalargecrosssectionwithhydrogen,which mightmakeitpossibletodetectmembranesinunstainedtissue. Atpresent,imagingathighresolutionusingatomicbeammicroscopyhasnotbeenachieved (lowerresolutionhasbeenachieved(Doak,Grisentietal.,1999)).Developmentofridged atomicmirrorshasenabledfocusingneutralatombeams(Oberst,Kouznetsovetal.,2005; ShimizuandFujita,2002)andwouldinprincipleallowfocusingthebeamtoaspotsizeof tensofnanometers(Kouznetsov,Oberstetal.,2006);byscanningthespotacrossthesample animagecouldbebuiltuplikeinotherformsofscanningmicroscopy.
44
Electronmicroscopy
Figure15:EMpictureofrathippocampusneuropil.Dmarksadendriteofapyramidalcell. Severalsynapsescanbeseentotheleft,noticeablebythepresenceofsmallspherical vesiclesonthepresynapticsideandadarkpostsynapticdensityonthereceivingside.The scalebaris1m.(CopyrightJSpacek,SynapseWeb) Electronmicroscopycanresolvethefinedetailsofaxonsanddendritesindenseneuraltissue. Imagescanbecreatedthroughtransmissionelectronmicroscopy(TEM),whereelectronsare sentthroughtissue,orscanningelectronmicroscopy(SEM)whereelectronsarescattered fromthesurface:bothmethodsrequirefixingthesamplebyfreezingand/orembeddingitin polymer.TEMhasachieved0.1nmimaging(Nellist,Chisholmetal.,2004).However,the mainchallengeistoautomatesectioningandacquisitionofdata.Thethreecurrentmain methodsareserialsectionelectrontomography(SSET),serialsectiontransmissionelectron microscopy(SSTEM)andserialblockfacescanningelectronmicroscopy(SBFSEM) (BriggmanandDenk,2006).TwonewmethodsthatmaybeusefulareFEIsDualBeam FocusedIonBeamSEM(FIBSEM)andautomaticultrathinsectioningandSEMimaging (ATLUM). SSET:HighresolutionTEM3Dimagescanbecreatedusingtiltseriesbasedtomography wherethepreparationistiltedrelativetotheelectronbeam,enablingtherecordingofdepth information(Frank,1992;Penczek,Markoetal.,1995).Thismethodappearssuitedmainlyfor localscanning(suchasimagingcellularorganelles)andcannotpenetrateverydeepintothe surface(around1m)(Lui,Frsteretal.,2005). SSTEM:CreatingultrathinslicesforTEMisanotherpossibility.(Tsang,2005)createdathree dimensionalmodeloftheneuromuscularjunctionthroughserialTEMof50nmsections createdusinganultramicrotome.(White,Southgateetal.,1986)usedserialsectionsto
45
reconstructtheC.elegansnervoussystem.However,sectioningisphysicallytrickyandlabor intensive. SBFSEM:Onewayofreducingtheproblemsofsectioningistoplacethemicrotomeinside themicroscopechamber(Leighton,1981);forfurthercontrast,plasmaetchingwasused (KuzirianandLeighton,1983).(DenkandHorstmann,2004)demonstratedthat backscatteringcontrastcouldbeusedinsteadinaSEM,simplifyingthetechnique.They producedstacksof5070nmthicksectionsusinganautomatedmicrotomeinthemicroscope chamber,withlateraljitterlessthan10nm.Theresolutionandfieldsizewaslimitedbythe commerciallyavailablesystem.Theyestimatedthattracingofaxonswith20nmresolution andS/Nratioofabout10withina200mcubecouldtakeaboutaday(while10nmx10nm x50nmvoxelsatS/N100wouldrequireascantimeontheorderofayear). Reconstructingvolumesfromultrathinsectionsfacesmanypracticalchallenges.Current electronmicroscopescannothandlesectionswiderthan12mm.Longseriesofsectionsare neededbuttheriskoferrorsordamageincreasewiththelength,andthenumberofspecimen holdinggridsbecomesexcessive(unlesssectioningoccursinsidethemicroscope(Kuzirian andLeighton,1983)).Currentstateoftheartforpracticalreconstructionfromtissueblocksis about0.1mm3,containingabout107108synapses(Fiala,2002). FIBSEM5:Thesemiconductorindustryhaslongusedfocusedionbeams(FIB)(usually acceleratedbeamsofgalliumions)toverypreciselycutawaypartsofintegratedcircuitchips forfailureanalysis.ResearchersatFEIhaverecentlydemonstratedthatthistechniqueand instrumentcanalsobeappliedtoplasticembeddedneuraltissueinamannersimilartothe SBFSEMabove(Mulders,Knottetal.,2006).IntheFIBSEM,thetop3050nmlayerofablock oftissue(havingblockfacedimensionsofapproximately100x100m)isablatedawayusing thefocusedionbeam.TheresultingblockfaceisthenimagedwiththeSEMusingthe backscattersignaljustasisdoneintheSBFSEM,andthisprocessisrepeatedtoimage hundredsofsuccessivelayers. BecausetheFIBSEMablatesawaytheblockstoplayerusinganionbeam(insteadofa diamondknife),thecookingoftheblocksurfacethatoccursinSBFSEMduetothehigh beamcurrentisnotaproblem.Usinglongerintegrationtimes,FIBSEMimageshave demonstratedlateralresolutionsof5nmorbetterandmuchimprovedsignaltonoiseratios. ATLUM:TheaboveSSETandSSTEMapproachesusesectionsobtainedbydiamondknife sectioningonatraditionalsemiautomatedultramicrotome.Thisprocessisonlysemi automatedbecausethecollectionofthesesectionsrequiresmanuallyscoopingfreefloating sectionsfromtheknifeswaterboatontoTEMslotgridsusinganeyelash.Thisisan inherentlyunreliableprocessforallbutthesmallestvolumes. TheAutomaticTapeCollectingLatheUltramicrotome(ATLUM)isaprototypemachine recentlydevelopedatHarvardUniversitythatfullyautomatestheprocessofultrathin sectioningandcollection(Hayworth,Kashturietal.,2006;Hayworth,2007).IntheATLUMa tissuesample(typically12mminwidth,10mmlong,and0.5mmdeep)ismountedonasteel axleandisrotatedcontinuouslyviaanultrapreciseairbearingwhileapiezodrivendiamond knifeslicesoffoneultrathinsectionperrevolution.Afeedbackloopemployingcapacitive sensorsmaintainsthepositionofthekniferelativetotheaxletowithinapproximately10nm allowingevenlargeareatissuesections(manysquaremillimeters)tobecuttothicknesses
5
TextderivedfromKennethHayworth.
46
Figure16:OverviewofATLUMoperation.(CopyrightLichtmanLab,HarvardUniversity) ThistapeissubsequentlystainedwithheavymetalsandimagedinaSEM.Imagingviathe SEMbackscattersignal(liketheSBFSEMandFIBSEMapproachesabove)allowstheATLUM todispensewithfragileTEMslotgrids(whichalsoremovestheirwidthofsection limitation).TheMylartapeprovidesasturdysubstrateforhandlingandstoragewhileits carboncoatingpreventschargingandbeamdamageduringSEMimaging. ImagesobtainedfromATLUMtissuetapesareofequivalentqualitytotraditionalTEM imagesshowinglateralresolutionbetterthan5nanometers.Recenttestshavealsoshownthat atomographictiltseries(asintheSSETtechnique)maybeperformedonATLUMcollected sectionstoobtainadditionaldepthinformation.Thisisperformedbytiltingthesectionat variousangleswithrespecttotheSEMselectronbeamthusprovidingZresolutionseven finerthanthesectionthicknessitself.Becausethesectionsarestoredforlaterimagingthey canbepoststainedwithheavymetals(producingimageswithsuperiorsignaltonoiseratios andmuchfasterimagingtimesthantheSBFSEMandFIBSEMblockfaceimagers)and,if needed,theycanbepoststainedwithaseriesofotherstainsforoverlayingchemicalanalysis mapsonthehighresolutionSEMstructuralimages.
47
Comparisonoftechniques
Resolution WiththeexceptionoftheSBFSEM,allofthesetechniqueshavealreadydemonstrated sufficientresolutiontomaptheexactpointtopointconnectivityofbraintissuedowntothe levelofcountingsynapticvesiclesinindividualsynapses.Withadditionalmodificationsit maybepossiblefortheSBFSEMtoalsoreachtheseresolutions. Reliabilityandrobustness TransmissionEMtechniquesliketheSSTEMandSSETrequiresectionstobemountedonslot grids.Theprocessofcollectingsectionsonslotgridshasnotyetbeensuccessfullyautomated, andsectionsmountedonslotgridsarethemselvesextremelyfragile(becausetheyare essentiallyfreelysupportedstructureslessthanamicronthick).Thisraisesseriousdoubtsas tothefeasibilityofscalingthesetechniquesuptoverylargevolumesofneuraltissue (althoughmachinesliketheATLUMmaybemodifiabletoallowautomatedcollectionof sectionsontapeswithprefabricatedslots,oralternativelyslotscanbecreatedinthetapeafter collectioniscompleted).SEMbackscatterimagingseemstoofferequivalentresolutionand imagequalitywhileavoidingthesepitfallsintrinsictoTEMapproaches. OneremainingpossibleadvantageofTEMapproachesisimagingspeed.Imagesincurrent SEMsarebuiltuponepixelatatimeastheelectronbeamscansacrossthesamplessurface. Incontrast,thepixelsinTEMimagesarecapturedinparallelbyaCCD+scintillatormounted belowthesection.Thisfactcould,inprinciple,allowimagingtimesordersofmagnitude fasterthanSEMimaging.However,manytechnicalissuescomplicatethiscomparison.In practice,currentTEMsimagesectionsonlyslightlyfasterthanSEMsdo.Inaddition, multibeamSEMs(describedbelow)arecurrentlybeingdevelopedthatwillmassively parallelize,andthusspeedup,SEMimageacquisition Blockfaceapproaches(SBFSEMandFIBSEM)haveinherentreliabilitysincetheyavoidthe perilousstepofcollectingultrathinsectionsbysimplydestroyingtheminsituafterhaving alreadyimagedthem.TheFIBSEMtechniqueisadditionally,atleastinprinciple,robustto smalldifferencesinembeddingquality.Alloftheothertechniquesusediamondknife sectioningwhichrequiresgooduniformityofresininfiltrationandresinhardnessthroughout thetissueblocktosectionsmoothly.Ofcoursetissuesamplesforallthetechniquesmustbe infiltratedcorrectlyforproperultrastructuralpreservation WhiletheATLUMisnecessarilylessreliablethanblockfaceapproaches(sinceitcollects ultrathinsectionsforlaterimaging)itstillhasthepotentialforextremereliabilityoverlarge volumes.Thisisbecausetheultrathinsections,withintheATLUMmechanism,aresecuredto thesturdyMylartapealmostimmediatelyaftertheyaresectionedbythediamondknife.In fact,theleadingedgeofeachsectionissecuredtothecollectiontapewhilethetrailedgeis stillbeingsectionedbytheknife.Inthiswayeachsectionisalwaysundercompletecontrolof themechanism. Imagingtime Forconcreteness,letsoutlineareasonableneartermneuronalcircuitmappinggoal necessarytosupportasetofpartialbraincircuitemulationexperiments.Wecanthen comparetheimagingtimes,andthusthefeasibility,ofthedifferentmethods.Oneofthemost interestingandbeststudiedpiecesofbraincircuitryisthatunderlyingorientationtuningin theprimaryvisualcortex(V1)inthecatandintheprimate.Thiscircuitscrudefunctional properties(orientationtunedsimpleandcomplexcells)havebeenknownfordecadesyet thecircuitryunderlyingthesefunctionalresponsesremainsatopicofheateddebate
48
generatingdozensofjournalarticleseveryyear.Moreimportantly,ourlackofadetailed understandingoftheneuronalcircuitryhasblockedprogressintoV1smoresubtle computationalpropertiessuchasitsroleinperceptualgrouping. Atrialbrainemulationexperimentmightsetasagoalthemodellingoforientationtuning responsesofafewcellsinV1(possiblycorrelatedwithprevioustwophotonrecordingof activitywithinthesametissue).Thistaskwouldnotrequireentirevolumesofcomplex neuropilbetracedintotal.Itwouldrequire,however,imagingtheconnectivityofatleast severaldozenneuronsinthevariouslayersofV1,andtheirconnectivitybothwitheachother andwithmanydozenaxonalprojectionstoV1fromthelateralgeniculatenucleus(LGN).In addition,theseaxonalprojectionsfromtheLGNwouldneedtobetracedbacktotheirorigins todeterminetheirrelativepositionsandthustoinfertheirownreceptivefieldproperties.In whole,suchastudywouldrequirenanoresolutiontracingofpartsofoveronehundred neuronsandthousandsofsynapticconnectionsspanningavolumeontheorderof10mm3 (ThisisassumingthatthemyelinatedprojectionsbetweenLGNandV1canbetracedatmuch lowerresolution,excludingitfromthevolumeestimate).Becauseofthenatureofthestudy, suchneuronscouldnotbeselectivelylabelledbeforehandandsotheelectronmicroscopic methodsdescribedinthissection(whichcanimageanyarbitraryneuronalcellorprocess)are theonlycurrenttechniqueswhichmaybecapableofperformingthetask. SSETandSSTEM:Asdiscussedabove,themanualcollectionofTEMreadysections(for techniquesliketheSSETandSSTEM)isoutofthequestionforvolumesaslargeas10mm3. SBFSEMandFIBSEM:Sofarthelargestvolumessectionedinthesedeviceshavebeenless than0.01mm3,withblockfacedimensionstypically200x200morless.Imagingrateon thesedevicesisaround100kHz(10microsecondsperpixel).Ingeneral,blockfaceimaging approachesnecessitatesuchslowimagingrates(toobtainadequatesignaltonoiseratios) sincethematerialcanonlybelightlystainedwhileinblockform. OurV1circuittracingscenariodoesnotstrictlyrequiretheentire10mm3volumebeimaged athighestresolution;however,astheneuronalcircuitstobetracedwanderrandomly throughouttheentirevolume,onedoesnotknowaprioriwhichpartstoimageathigh resolution.Because,intheseblockfaceimagingdevices,eachsectionisdestroyedinsitu, thereisonlyonechancetoimageitandthusineffectallpartsofthevolumemustbeimaged atsufficientresolutiontotracethefinestneuronalprocesses.Assuming5nmlateral resolutionand50nmsections(typicalvaluesforneuropiltracingstudies)the10mm3block consistsof81015voxelsandwouldrequireontheorderof2,000yearstoimage!Suchlong imagingtimemakesthistracingstudyinfeasibleonthesemachines. ATLUM:ThelargestvolumesectionedontheATLUMsofaris0.1mm3withblockface dimensionsof1.4mmx4.5mmandsectionthicknessof45nm(totalof400sections).ATLUM cuttingspeedisaround0.03mm/secandeachoftheseultrathinsections(6.3mm2inarea)was producedatarateofapproximately4minutespersection.Ifsucharatecouldbereliably sustaineditwouldtakeabout3monthsforanATLUMlikedevicetoreducea10mm3tissue blocktoaseriesof50nmsectionseachsecurelymountedonalongMylartapereadyfor imaging. BecausetheATLUMcollectssectionsforlaterimaging,itssectionscanbepoststainedwith heavymetals.Thisallowsadequatesignaltonoiseratioswithshorterimageacquisition times.Imagingrateisaround1MHz(1.0microsecondperpixel)sobulkimagingtheentire 10mm3wouldrequireontheorderof200years.Thisstillwouldrendertheprojectinfeasible;
49
however,unliketheblockfacetechniques,theATLUMtechniquecantakeadvantageof directedhighresolutionimagingtodramaticallyshortenthisimagingtime. Asstatedabove,atcurrentsectioningspeeds,theATLUMcouldreducea10mm3blockof braintissuetoatapeofultrathinsectionsinjustafewmonths.Suchatapewouldconstitutea permanentultrathinsectionlibraryoftheentirevolume,andanypartofitcouldbe randomlyimaged(andreimaged)atwhateverresolutiondesired.Aresearchercould coarselyimagetheentirevolumeinafewdays,findatargetneuronwithinthevolume,and thensubsequentlydirecthighresolutionimagingexactlywhereitisneededtotracethat neuronsprocessesanditsconnectionswithadditionalneurons.Inthisway,amultineuron circuit,stretchingthroughouttheentire10mm3volume,couldbemappedwithsynaptic precisionwhileonlyimagingperhapsoneonethousandthofthewholevolume.Inthis fashion,theATLUMcanpotentiallyreducethetimeneededtotraceacompleteneuralcircuit fromhundredsofyearstojustafewmonths,makingtheV1trialemulationexperiment potentiallyfeasibleinthenearterm. Parenthetically,wecanalsonotethattheneartermgoaloftracingandemulatingcircuitsofa fewhundredneuronsdrasticallyreducestheimageprocessingandcomputersimulation requirements.Insteadofrequiringmillionsofneuronswithbillionsofsynapsestobetraced andidentified,onlythelimitedsubsetunderstudyneedbetracedintheimagedata.The successfulcompletionofpartialbrainemulationstudies(suchasthehypotheticalV1study above)isprobablyanecessaryprecursortospurthetypeofresearchandinvestmentin sectioning,imaging,tracing,andemulationstechnologiesneededforeventuallyscalingupto wholebrainemulationlevels. PossibilitiesforincreasingSEMimagingspeed Fromtheabovediscussionitisclearthatlongimagingtimesconstituteamajorbarrierto wholebrainemulationusingSEMtechniques.However,thereiscurrentlyamajorresearch pushtowardmassivelyparallelmultibeamSEMswhichhasthepotentialtospeedupSEM imagingbymanyordersofmagnitude.Thisresearchpushisbeingdrivenbythe semiconductorindustryaspartofitsefforttoreducefeaturesizesoncomputerchipsbelow thelevelthattraditionalphotolithographycanproduce. Thecircuitrypatternswithincomputerchipsareproducedthroughaseriesofetchingand dopingsteps.Eachofthesestepsmustaffectonlyselectedpartsofthechip,soareastobeleft unaffectedaretemporallycoveredbyathinlayerofpolymerwhichispatternedinexquisite detailtomatchthesubmicronfeaturesofthedesiredcircuitry.Forcurrentmassproduction ofchipsthispolymerlayerispatternedbyshiningultravioletlightthroughamaskontothe surfaceofthesiliconwaferwhichhasbeencoveredwiththephotopolymerinliquidform. Thisselectivelycuresonlythedesiredpartsofthephotopolymer.Toobtainsmallerfeatures thanUVlightcanallow,electronbeams(justasinaSEM)mustinsteadbeusedtoselectively curethephotopolymer.Thisprocessiscalledebeamlithography.Becausetheelectronbeam mustberasteredacrossthewafersurface(insteadoffloodilluminatingitasinlight lithography)theprocessiscurrentlymuchtooslowforproductionlevelruns. Severalresearchgroupsandcompaniesarecurrentlyaddressingthisspeedproblemby developingmultibeamebeamlithographysystems(Kruit,1998;vanBruggen,vanSomeren etal.,2005;vanSomeren,vanBruggenetal.,2006;ArradianceInc).Inthesesystems, hundredstothousandsofelectronbeamsrasteracrossawaferssurfacesimultaneously writingthecircuitrypatterns.ThesemultibeamsystemsareessentiallySEMs,anditshould beastraightforwardtasktomodifythemtoallowmassivelyparallelscanningaswell
50
(Pickard,Grovesetal.,2003).Forbackscatterimaging(asintheSBFSEM,FIBSEM,and ATLUMtechnologies)thismightinvolvemountingascintillatorwithagridofholes(onefor eachebeam)veryclosetothesurfaceofthetissuebeingimaged.Inthiswaytheinteractions ofeachebeamwiththetissuecanbereadoffindependentlyandsimultaneously. ItisdifficulttopredicthowfasttheseSEMsmayeventuallyget.A1,000beamSEMwhere eachindividualbeammaintainsthecurrent1MHzacquisitionrateforstainedsections appearsreachablewithinthenexttenyears.WecanverytentativelyapplythisprojectedSEM speeduptoaskhowlongimagingahumanbrainwouldtake.First,assumeabrainwere slicedinto50nmsectionsonATLUMlikedevices(anenormousfeatwhichwoulditselftake approximately1,000machineseachoperatingat10xthecurrentsectioningrateatotalof 3.5yearstoaccomplish).Thismassiveultrathinsectionlibrarywouldcontaintheequivalent of1.11021voxels(at5550nmpervoxel).Assumingjudicioususeofdirectedimaging withinthisultrathinsectionlibraryonly1/10mayhavetobeimagedatthisextremelyhigh resolution(usingmuchlower,andthusfaster,imagingonwhitematertracts,cellbody interiorsetc.).Thisleavesroughly1.11020voxelstobeimagedathighresolution.If1,000 SEMseachutilizing1,000beamletsweretotacklethisimagingjobinparalleltheircombined dataacquisitionratewouldbe11012voxelspersecond.Atthisratetheentireimagingtask couldbecompletedinlessthan4years.
Nanodisassembly
Themostcompleteapproachwouldbetopickthebrainapartatombyatomormoleculeby molecule,recordingtheirpositionandtypeforfurtheranalysis.Thescenarioin(Morevec, 1988)canalsobedescribedasnanodisassembly(inanunfixatedbrain,withonthefly emulation)workingonaslightlylargersizescale.(Merkle,1994)describesarelatively detailedproposalwherethebrainisdividedinto3.210150.4mcubeswhereeachcube wouldbedisassembledatomically(andatom/moleculepositionsrecorded)byadisassembler nanodevice(Drexler,1986)overathreeyearperiod. Ithasbeenpointedoutthatmedicalnanoroboticsistheformofnanotechnologybestusedfor nondestructivescanning6.Giventhatnodetailedproposalforananodisassemblerhasbeen madeitishardtoevaluatethechancesofnanodisassembly.Itwouldhavetoactatalow temperaturetopreventmoleculesinthesamplefrommovingaround,removingsurface moleculesonebyone,identifyingthemandtransmittingtheposition,orientationandtypeto secondlinedataaggregators.Clearchallengesaretheconstructionoftooltipsthatcanextract arbitrarymoleculesordetectmoleculartypeforfurtherhandlingwithspecializedtooltips,as wellashandlingmacromoleculesandfragilemolecularstructures.Macromoleculescanlikely notbepulledoutinonepiece.Atomicdisassemblywouldavoidthecomplicationsof moleculesforthegreatersimplicityofahandfulofatomtypes,atthepriceofneedingto breakmolecularbonds,theriskofensuingrearrangementsandthepossibilityofcreating reactivefreeradicals.Matureproductivenanosystemswouldappeartobeanecessary precursortechnologyforthisapproach(probablyfortheproductionofthemassivelyparallel disassemblysystemrequiredforcontrolledatomicdisassemblyofamacroscopicobject) (ForesightNanotechInstituteandBatelleMemorialInstitute,2007).Itappearslikelythatthe kindoftechnologyneededfordisassemblywouldbesignificantlymorecomplexthantheone neededforatomicallypreciseassembly.
RobertFreitasJr.,personalcommunication.
51
Chemicalanalysis
Akeychallengeistodetectthechemicalstateandtypeofcellularcomponents.Normallythis isdonebystainingwithdyesorquantumdotsthatbindtotherighttarget,followedby readoutusingopticalmethods.Besidetheneedfordiffusingdyesthroughthesamples,each dyeisonlyselectiveforacertaintargetorgroupoftargets,necessitatingmultipledyesfor identifyingallrelevantcomponents.Ifthenumberofchemicalsthathavetobeidentifiedis large,thiswouldmakedyeingineffective. OnepossibleapproachisRamanmicrospectroscopy(Krafft,2004;Krafft,Knetschkeetal., 2003),wherenearinfraredscatteringisusedtoimagethevibrationspectrumofthechemical components(mainlymacromolecules)oftissue.Theresolutionfornearinfraredspectroscopy isabout1m(limitedbydiffraction)andconfocalmethodscanbeusedfor3Dimaging. Recordingtimesareverylong,ontheorderofminutesforindividualpixels;inordertobe usefulforWBEthishastobespeededupsignificantlyorparallelized.Usingshorter wavelengthsappearstoinducetissuedamage(Puppels,Olminkhofetal.,1991),whichmay beoflittleconcernfordestructivescanning.Ultravioletresonancemicrospectroscopyhasalso beenused,enablingselectiveprobingofcertainmacromolecules(Pajcini,Munroetal.,1997; Hanlon,Manoharanetal.,2000).Insomecasesnativefluorescencecanenableimagingby triggeringitwithUVlight,laserinducednativefluorescence,LINF,suchasinthecaseof serotonin(Tan,Parpuraetal.,1995;Parpura,Tongetal.,1998)andpossiblydopamine (Mabuchi,Shimadaetal.,2001). Anewmethodwithgreatpromiseisarraytomography,ahybridbetweenoptical fluorescenceimagingandelectronmicroscopy(MichevaandSmith,2007).Samplesarecut intoultrathin(50200nmthick)sectionsforminganorderedarrayonaglassslide.Thearray isthenlabelledwithfluorescentantibodiesorotherstainsandimaged,generatinga3D reconstructionwithadepthresolutionsetbythesectioningthickness.Afterthisimagingthe arraycanbeelutedtoremovethestain,restainedwithanewstain,imagedandsoon.Finally itcanbestainedwithmetalandimagedinascanningelectronmicroscope.Thedifferent imagestackscanthenbecombined,givingahighresolution3Dreconstructionwithchemical information. Atpresentitlooksuncertainhowmuchfunctionallyrelevantinformationcanbedetermined fromspectra.Ifthenumberofneurontypesisrelativelylowandchemicallydistinct,itmight beenoughtorecognizetheirindividualprofiles.Addingdyestailoredtodisambiguate otherwiseindistinguishablecasesmayalsohelp.
52
Conclusion
ItislikelythatthescanningmethodsusedtoproviderawdataforWBEwillbe,atleastinthe earlydays,destructivemethodsmakinguseofsectioningandscanning. Assumingthatimagingthefinestaxonalprocessesandsynapticspinesarerequired(50nm), thissetsaresolutionrequirementontheorderof5nmatleastintwodirections.Comparing themethodsmentionedandtheirresolutionwegetthefollowingcategorization: Table5:Scanningmethods
Method MRI Resolution >5.5m(nonfrozen) Doesnotrequiresectioning,may achievebetterresolutiononvitrified brains. Spectromicroscopypossible? Notimplementedyet. Requiresfluorescentmarkers, spectroscopypossible. Enablesmultiplestaining Basic2Dmethod,mustbecombined withsectioningortomographyfor 3Dimaging.Damagefromhigh energyelectronsathighresolutions.
MRImicroscopy NIRmicrospectroscopy Allopticalhistology KESM Xraymicrotomography MRFM SI Xraymicroscopy SBFSEM FIBSEM ATLUM SSET Atomicbeammicroscopy NSOM SEM Arraytomography TEM
3m 1m 0.7m 0.3mx0.5m 0.47m 80nm 50nm 30nm Requiredresolution? 5070nmx120nm 3050nmx120nm 40nmx5nm 50nmx1nm 10nm 5nm? 120nm 120nmSEM,50x200x200nm fluorescencestains <1nm
A5550nmresolutionbrainscanrequires1.41021voxels,alargeamountofrawdata(see nextsection).Destructivescanningworkingonfixatedbrainsmayavoidhavingtostorethe entiredatasetbyonlystoringthemostrecentslicesofbrainandperformimageprocessingon these.Insteadoffirstscanningtheentirebrain,andthenprocessingtheimagedatatoextract therelevantfeaturesandneuronalnetwork,theextractioncanproceedpiecemealand concurrentlyasnewimagingdataiscollected.Oncerelevantinformationhasbeenextracted fromadatabatch,therawvisualdataforthatbatchcanbediscarded. However,ideassuchasthesinglebrainphysicallibrary(Hayworth,2002)demonstratethat thescanningdoesnothavetobesingleshot:abrainissectionedandfixedinasuitable mannerforinteractivescanning(possiblyusingseveralmodalities)andretrieval.Thisalso getsaroundthedatastorageproblembyonlyneedingtoscanregionsofinterestandstore theirdata(possiblytemporarily,sincetheycanberescannedifneededlater).Inearly neuroinformaticsapplications,thismightincludemappingoutthelongrangeconnectivityof asubsetofrepresentativeneuronsinordertofindtheirmorphologyandconnectivity,only requiringimagingaverysmallsubsetofthebrainsvolume(0.01%formappingout100,000 neuronsinahumanbrain).Itmayalsobepossibletocombinethisapproachwithmethods likearraytomography(MichevaandSmith,2007)forcombinedchemicalandstructural maps.
53
Lowresolutionmethodssuchasopticalmicroscopyareanimportanttestofmanyaspectsof theWBEresearchendeavour,suchaslargescaledatamanagement,inferringneuralfunction frommorphology(possiblyusingspectroscopyorothermethodsofestimatingchemical states),andprovidingearlytestdatafortracingandconnectivityreconstructionalgorithms. Iftherequiredlevelofsimulationisatlevel5orabove(detailedcellularelectrophysiology andneuronconnectivity:seeTable2)weappeartohavealreadyachievedtheresolution requirementsandtheremainingproblemisindata/tissuemanagementandscalingup methodstohandlelargebrains.UsingKESMorATLUMitshouldbepossibleinthevery nearfuturetoconstructadetailedconnectomeofthebrain,inparticulartheexact interconnectionsofcorticalminicolumns. IftherequiredlevelforWBEisbelowlevel5,increasesofimagingresolutionareofrelatively limiteduse.Rather,weneedmodalitiesthatenablemappingtheproteins,possiblytheir statesandthepresenceofmetabolitesorRNAtranscripts.Thiswouldposeamajorresearch challenge,althoughitisinlinewithmuchresearchinteresttodayexaminingthebiophysics ofcells.
54
Imageprocessingandscaninterpretation
Thedatafromthescanningmustbepostprocessedandinterpretedinordertobecomeuseful forbrainemulation(orotherresearch).Cellmembranesmustbetraced,synapsesidentified, neuronvolumessegmented,distributionofsynapses,organelles,celltypesandother anatomicaldetails(bloodvessels,glia)identified.Currentlythisislargelydonemanually: cellularmembranescanbeidentifiedandhandtracedatarateof12hours/m3(Fialaand Harris,2001),fartooslowforevensmallcorticalvolumes. Softwareneededincludes(after(Fiala,2002)): Geometricadjustment(aligningsections,handlingshrinkage,distortions) Noiseremoval Datainterpolation(replacinglostorcorruptedscandata) Cellmembranetracing(segmentation,tracingin2Dand3D) Synapseidentification Identificationofcelltypes Estimationofparametersforemulation Connectivityidentification Databasing Datahandlingisatpresentabottleneck.0.1mm3at400pixels/mresolutionand50nm sectionthicknesswould(compressed)contain73terabytesofrawdata.Afullbrainatthis resolutionwouldrequire109terabytes(Fiala,2002).Whileextremelylarge,eventhismight onedayberegardedasfeasible(seeAppendixB).However,asmentionedintheprevious chapter,foremulationpurposesitmaybefarmorepracticaltoperformasequenceof scanningandinterpretationstepssothatonlyasmallerbufferofhighresolutiondatais necessary. Ifthebrainisdividedintosmallblocks,eachblockcouldbeanalyzedindependently,atleast intermsoflowlevelimageprocessingandpreliminarytracing.Datafromneighbouring blockswouldneedtobecomparedbutthereisnoneedfortheanalysisofmoreremote blocks(withthepossibleexceptionofinterpolatinglostdata).Thissuggeststhatitcanbe parallelizedtoagreatdegree.Asthescanprogresses,lowleveldatacanbediscardedto makeroomforthecompressedhighlevelrepresentationneededtobuildtheemulation7. ThissectiondealswiththeassumptionthatWBEisachievedusingimagebasedmethods ratherthancorrelationanalysismethodswheree.g.nanomachinesrecordlocalneuralactivity andestimateconnectivityfromthecorrelationsintheactivitypattern.Correlationmethods wouldhavecorrespondingdemandsforsignalprocessingandinference,butinthetime domainratherthanthespacedomain.
Geometric adjustment
Variousmethodsforachievingautomaticregistration(correctingdifferencesinalignment)of imagestacksarebeingdeveloped.Atitssimplest,registrationinvolvesfindingacombination oftranslation,scaling,androtationthatmakessubsequentimagesmatchbest.However, skewingandnonlineardistortionscanoccur,requiringmorecomplexmethods.Combining thiswithoptimizationmethodsandanelasticmodeltocorrectforshapedistortionproduced
7Thislossofdatamaybeofconcernsinceinitialscansarelikelytobeexpensive(andhencehardtorepeat),andfor humanbrains(whichareindividuallyvaluable).SomeformsofscanningsuchasATLUMmayproducestorable librariesofslicesthatcanberetainedforrescanningorfurtheranalysis(Hayworth,2002).
55
goodresultswithmacroscopicstacksofratandhumanbrains(Schmitt,Modersitzkietal., 2007).
Noise removal
Removingnoisefromimagesispartofstandardimageprocessing,withanextensive literatureandstrongresearchinterest.Ingeneral,noiseremovalissimplifiedwhenthekinds ofnoiseintroducedbyscanningartefactsandthenatureofthesystemareknown.Asan example,see(Mayerich,McCormicketal.,2007)wherelightvariationsandknifechatter noisewereremovedfromKESMdata.
Data interpolation
Lost/corrupteddatamustbereplacedwithprobabilisticinterpolations.Thismightrequire feedbackfromlaterstagestofindthemostlikelyinterpretationorguess,constrainedbywhat makessensegivenknowndata. Forlargelostvolumes,genericneuronsandconnectivitymighthavetobegeneratedbased onmodelsofmorphologyandconnectivity.Thegoalwouldbetoavoidchangingthe functionalityofthetotalnetwork.EarlyresultsinWBEandstatisticsfromsuccessfulimaging volumesshouldprovidemuchusefulinputindevelopingthis. Sectioningthebrainintoindividuallyscannablechunkswillintroducedatalossandpossible misalignmentalongtheedges(forexample,theKESMsuffersdamageupto5minwidth betweendifferentcolumns(Kwon,Mayerichetal.,2008)).Thismayprovetobethemajor sourceoflostdatainWBE.Sincethiscouldcausemistracingoflongaxonscrossing numerouschunkboundaries,solvingthisissueisahighpriority.Alignmentcanprobablybe achievedreliablyifthelostzoneissufficientlysmallerthanthecorrelationlengthinthe surroundingimages.
56
Cell tracing
Figure17:BloodvesselreconstructionfromKESMNissldata.(CopyrightbrainNetworks Laboratory,TexasA&MUniversity) Automatedtracingofneuronsimagedusingconfocalmicroscopyhasbeenattemptedusinga varietyofmethods.Evenifthescanningmethodusedwillbeadifferentapproachitseems likelythatknowledgegainedfromthesereconstructionmethodswillbeuseful. Oneapproachistoenhanceedgesandfindtheoptimaljoiningofedgepixels/voxelstodetect contoursofobjects.Anotherisskeletonization.Forexample,(Urban,OMalleyetal.,2006) thresholdedneuronimages(afterimageprocessingtoremovenoiseandartefacts),extracting themedialaxistree.(Dima,Scholzetal.,2002)employeda3Dwavelettransformtoperform amultiscalevalidationofdendriteboundaries,inturnproducinganestimateofaskeleton. Athirdapproachisexploratoryalgorithms,wherethealgorithmstartsatapointanduses imagecoherencytotracethecellfromthere.Thisavoidshavingtoprocessallvoxels,but riskslosingpartsoftheneuroniftheimagesaredegradedorunclear.(AlKofahi,Laseketal., 2002)usedirectionalkernelsactingontheintensitydatatofollowcylindricalobjects. (MayerichandKeyser,2008)useasimilarmethodforKESMdata,acceleratingthekernel calculationbyusinggraphicshardware.(Uehara,Colbertetal.,2004)calculatesthe probabilityofeachvoxelbelongingtoacylindricalstructure,andthenpropagatesdendrite pathsthroughit. Oneweaknessofthesemethodsisthattheyassumecylindricalshapesofdendritesandthe lackofadjoiningstructures(suchasdendriticspines).Byusingsupportvectormachinesthat aretrainedonrealdataamorerobustreconstructioncanbeachieved(SantamaraPang, Bildeaetal.,2006). Overall,tracingofbranchingtubularstructuresisamajorinterestinmedicalcomputing.A
57
surveyofvesselextractiontechniqueslisted14majorapproaches,withseveralexamplesof each(KirbasandQuek,2004).Thesuccessofdifferentmethodsismodalitydependent.
Figure18:3DvisualizationofGolgistainedcellreconstructedfromKESMdata. (CopyrightBrainNetworksLaboratory,TexasA&MUniversity
58
Synapse identification
Figure20:ReconstructedspinydendriteofCA1pyramidalcell,renderedfromdatain (HarrisandStevens,1989).(CopyrightSynapseWeb) Inelectronmicrographs,synapsesarecurrentlyrecognizedusingthecriteriathatwithina structuretherearesynapticvesiclesadjacenttoapresynapticdensity,asynapticdensitywith electrondensematerialinthecleftanddensitiesonthecytoplasmicfacesinthepreand postsynapticmembranes(Colonnier,1981;PetersandPalay,1996). OneofthemajorunresolvedissuesforWBEiswhetheritispossibletoidentifythefunctional characteristicsofsynapses,inparticularsynapticstrengthandneurotransmittercontent,from theirmorphology. Ingeneral,corticalsynapsestendtobeeitherasymmetricaltypeIsynapses(7595%)or symmetricaltypeIIsynapses(525%),basedonhavingaprominentorthinpostsynaptic density.TypeIIsynapsesappeartobeinhibitory,whiletypeIsynapsesaremainlyexcitatory (butthereareexceptions)(PetersandPalay,1996).Thisallowsatleastsomeinferenceof functionfrommorphology. Theshapeandtypeofvesiclesmayalsoprovidecluesaboutfunction.Small,clearvesicles appeartomainlycontainsmallmoleculeneurotransmitters;largevesicles(60nmdiameter) withdensecoresappeartocontainnoradrenaline,dopamineor5HT;andlargevesicles(up to100nm)with5070nmdensecorescontainneuropeptides(Hokfelt,Brobergeretal.,2000; Salio,Lossietal.,2006).Unfortunatelytheredoesnotappeartobeanyfurtherdistinctiveness ofvesiclemorphologytosignalneurotransmittertype.
59
60
Electrophysiology
Givenelectrophysiologicaldataofacellsresponsestosimplecurrentstimuliitispossibleto replicatetheresponseinacompartmentmodel(inparticularthedensityofionicchannels) throughautomatedmethodssuchasgeneticalgorithmsandsimulatedannealing (Druckmann,Banittetal.,2007;Keren,Peledetal.,2005;VanierandBower,1999;VanGeit, Achardetal.,2007).However,fittingexperimentshaveshownthatthesameneuralactivity canbeproducedbydifferentsetsofconductances(AchardandDeSchutter,2006).Thismay suggestlessneedtodetectallionicconductancesifdonebyothermeans,buttheresultsalso suggestthatthesetofgoodmodelsformrelativelyisolatedhyperplanes:relativelyprecise fittingofallparametersisneededtogetgoodperformance.
61
Geneexpression
Differentinhibitoryinterneuronsshowdifferentgeneexpressionfordifferentreceptors,ion channelsandgapjunctionformingproteins(Blatow,Caputietal.,2005).Strongcorrelations betweengeneexpressionofcertainionchannelsandconductanceshavebeenobserved suggestingthatfunctioncanbeinferredfromgeneticdata(ToledoRodriguez,Blumenfeldet al.,2004).Differentneuronsshowvariableexpressionlevels,yetdifferentneurontypeshave uniquepatterns(Schulz,Goaillardetal.,2006;Schulz,Goaillardetal.,2007).Differencesin alternatesplicinghavebeenfoundinindividualcellswithinaneuronpopulation,butthere arealsodifferencesinsplicingindifferentbrainregionssuggestingthereareoverall regulatorymechanisms(WangandGrabowski,1996). IfscanningcandetectrelevantmRNAlevels(andthelinkbetweentheseandfunctional propertieshavebeenfoundusinghighthroughputanalysisofallkindsofneurons)itwould likelybepossibletodeducefunctionaltype.
Detectingsynapticefficacies
Normallysynapticpropertiesaredeterminedelectrophysiologicallybytriggeringaction potentialsinthepresynapticneuronandmeasuringtheresponseinthepostsynapticneuron. Thiscandetectnotjustthegeneralstrengthofthesynapsebutalsoadaptationproperties. Therearecaseswhereelectrophysiologicalpropertiesappeartobelinkedtodetectable differencesinsynapticmorphology,inparticulartheappearanceofvesicleribbons(Fields andEllisman,1985;Hull,Studholmeetal.,2006).LTPlikelyaffectssynapticcurvature,size andperforationsofthepostsynapticdensityinatimedependentmanner(MarroneandPetit, 2002;Marrone,2007).Itiscommonlyassumedthatthis,aswellasmoreshortterm electrophysiologicalchanges,involvesremodellingofthecytoskeletoninresponseto plasticity(DillonandGoda,2005;Chen,Rexetal.,2007).Particularproteins,suchasprofilin, aregulatorofactinpolymerisation,areactivatedbyLTPinducingstimuliandmayindicate synapsesthatarepotentiating(AckermannandMatus,2003). Manysynapsesaresilent,lackingmembraneAMPAreceptors;stimulationcanproducea transitiontoanactivevocalstate(Kullmann,2003).Silentsynapsesdonotappeardifferent inmicrographs,suggestinganegativeanswertothequestionofwhetherfunctioncanbe inferredfromEMstructure(AtwoodandWojtowicz,1999).ThismeansthatforWBE detectingatleastAMPAmaybenecessary;thiscouldlikelybeachievedusingarray tomography.
Connectivity identification
Thisstepassignssynapticconnectionsbetweenneurons. Atpresent,statisticalconnectivityrulesareusedbasedonproximity,thePetersrule,in whichsynapticconnectionsareassumedwhereaxonswithboutonsoverlapwithdendrites (BraitenbergandSchuz,1998;Peters,1979).Thiscanbeusedtoestimatethestatisticsof synapticconnectivity(Binzegger,Douglasetal.,2004;Shepherd,Stepanyantsetal.,2005; Kalisman,Silberbergetal.,2003).However,neuralgeometrycannotpredictthestrengthof functionalconnectionsreliably,perhapsbecausesynapticplasticitychangesthestrengthof geometricallygivensynapses(Shepherd,Stepanyantsetal.,2005).
62
Itisnotpossibletorelyonsynapsesonlyoccurringfromaxonstodendrites;axosomatic, axoaxonic,anddendrodendritic(whichmaybeonewayorreciprocal)synapseshavebeen observed.Occasionally,severalsynapsescoincide,suchasserialaxoaxodendriticsynapses andsynapticglomeruliwhereanaxonsynapsesontotwodendrites,oneofwhichalso synapsesontheother. Onepossibilityisthatthereexistsenoughstereotypy(repeatingpatternsofstructuraland functionalfeatures)inthebraintosimplifyconnectivityidentification(andpossibly interpolation)(Silberberg,Guptaetal.,2002).Suchinformationwouldconstrainthe interpretationprocessbyrulingoutcertainpossibilities,whichwouldattheveryleastenable aspeedup.Acquiringthepotentiallyverycomplexinformationaboutstereotyped arrangementswouldbeoneoftheearliestapplicationsandbenefitsofdetailedscanningand massiveneuroinformaticsdatabases.However,deviationsfromstereotypymaybeof particularimportanceinachievingpersonWBEsincetheycanrepresentindividualvariations orcharacteristics. Gapjunctions,wherethepreandpostsynapticcellsareelectricallylinkedbyconnexon channelsthroughthecellmembrane,canbeidentifiedbymembranesremainingparallelwith just2nmseparationandagridofconnexons.Theyappeartoberelativelyrareinmammals, butoccuratleastintheretina,inferioroliveandlateralvestibularnucleus(PetersandPalay, 1996). AtleastinsimplenervoussystemssuchasC.elegansgeneexpressioncontainssignificant informationaboutitsconnectivitysignature(Kaufman,Droretal.,2006).Attheveryleast, theavailabilityofthiskindofinformationcouldbeusedtoconstrainneurontypesand connectivity.
Conclusion
Imageprocessingisamatureareawithmanycommercialandscientificapplications. Developmentofbasicprocessingforhandlingthescandatadoesnotappeartoposemany problemsbeyondtheneedforextremelyhighthroughputsignalandimageprocessing,and perhapsthedatamanagementissuesoftherawscans. Theneuroinformatics/WBErelatedfurtherprocessingstepsposearesearchchallenge. Identifyingcellularobjects,inparticularconnectivityandsynapses,isanontrivialimage interpretationproblemthatiscurrentlybeingstudied.Basiccontouringalgorithmsappearto workreasonablywell,anditislikely,givenotherresultsincurrentimagerecognition,that synapsedetectorscouldbeconstructed.Imageinterpretationistraditionallya computationallycostlyoperation,andmayconceivablybeabottleneckindevelopingearly WBEmodels(inthelongrun,assumingimprovementsincomputerhardwarenecessary anywayforlargescaleemulation,thisbottleneckisunlikelytoremain). Thehardestandcurrentlyleastunderstoodissueisestimatingemulationparametersfrom imagery(ordevelopingscanningmethodsthatcangaintheseparameters).Thisrepresents oneofthekeyresearchissuesWBEneedtoanswer(ifonlytentatively)inordertoassessits viabilityasaresearchprogram.
63
Neuralsimulation
TheareaofneuralsimulationbeganwiththeclassicHodgkinandHuxleymodeloftheaction potential(HodgkinandHuxley,1952).Atthetime,calculatingasingleactionpotentialusing amanuallycrankedcalculatortook8hoursofhardmanuallabour.Sincethentheabilityto computeneuralactivityacrosslargenetworkshasgrownenormously(MooreandHines, 1994).
InternalChemistry
Brainemulationneedstotakechemistrymoreintoaccountthancommonlyoccursincurrent computationalmodels(Thagard,2002).Chemicalprocessesinsideneuronshave
64
computationalpowerontheirownandoccuronavastrangeoftimescales(fromsub millisecondtoweeks).Neuromodulatorsandhormonescanchangethecausalstructureof neuralnetworks,e.g.byshiftingfiringpatternsbetweendifferentattractorstates. About200chemicalspecieshavebeenidentifiedasinvolvedinsynapticplasticity,forminga complexchemicalnetwork.However,muchofthecomplexitymayberedundantparallel implementationsofafewcorefunctionssuchasinduction,patternselectivity,expressionof change,andmaintenanceofchange(wheretheredundancyimprovesrobustnessandoffers thepossibilityoffinetuning)(AjayandBhalla,2006). Attheverylownumbersofmoleculesfoundinsynapticspines,chemicalnoisebecomesa significantfactor,makingchemicalnetworksthatarebistableatlargervolumesunstable belowthefemtoliterlevelandreducingpatternselection(Bhalla,2004b,a).Itislikelythat complexformationoractivityconstrainedbymembranesisessentialforthereliabilityof synapses.However,thismaynotnecessarilyrequiredetailedmodellingofthecomplexes themselves,justoftheirstatistics. Proteomicsmethodsarebeingappliedtosynapses,potentiallyidentifyingallpresent proteins(Li,2007).TheSynapseproteinDataBasecontainsabout3,000humansynapse relatedproteins(Zhang,Zhangetal.,2007),althoughitislikelythatmanyofthesearenot involvedintheactualsynapticprocessing. Oftheproteinscodedbythehumangenome,around3.2%(988)havebeenpredictedtobe regulatorymolecules,2.8%(868)kinases,5.0%(1543)receptors,1.2%(376)signalling molecules,and1.3%(406)ionchannels.Iftherelativeproportionsarethesameamongthe unknownproteins(41.7%,12,809),thenumbersshouldbescaledupby1.4(VenterAdamset al.,2001).Posttranslationalmodificationsproduceonaverage36differentproteinspergene (Wilkins,Sanchezetal.,1996).Thisplacesanupperlimitonthenumberofproteintypes directlyinvolvedinneuralsignallingandinternalsignaltransductionontheorderof35,000. Ifallgeneswereinvolvedweshouldexpecttheproteometobearound158,000proteins. Laterestimateshaverunupallthewayto1millionproteins(and600,000immunoglobulins varyinginepitopebinding,likelyirrelevantforWBE)(HumpherySmith,2004),although mostestimatesappeartotendtowardsthe100,000range. IftherequirementsforWBEareonthekineomelevel,thenforeachinvolvedproteina numberofspeciesmaybeneeded,dependingonthenumberofpossiblephosphorylation, dimerization,carboxylation,orotheralteredstatesthatexist.Inprinciple,thiscouldleadtoa combinatorialexplosionoftypes,requiringstoringdataforindividualproteinmoleculesif thenumberoffunctionallyrelevantkindsisverylarge. ItiscommontosimulatecellularregulatorynetworksusingmassactionlawsandMichaelis Mentenkinetics,althoughthisassumesfreediffusionandrandomcollisions.Thisassumption doesnotalwayshold,sincemolecularmobilityinsidecellsislimitedbythepropertiesofthe cytoplasm,compartmentalization,andproteinanchoringtosurfaces.Thiscanbeatleast partiallytakenintoaccountbyadjustingtheequationsforfractalkinetics.Whetherfractal kineticsormassactionisvaliddependsmainlyontheprobabilityofreactions(Grimaand Schnell,2006;SchnellandTurner,2004;XuandDing,2007). Itmightalsobenecessarytomodelgeneexpressionandotherepigeneticmechanisms.Long termplasticityinneuronsrequireschangesingeneexpression,bothasresponsetoplasticity inducinginputandinregulatingoverallplasticity.Inaddition,geneexpressionisknownto
65
Learningrulesandsynapticadaptation
AWBEwithoutanysynapticchangewouldlikelycorrespondtoaseverelyconfusedmind, trappedinanterogradeamnesia.Whileworkingmemorymaybebasedonattractorstatesof neuralactivityintheprefrontalcortexandsomeformsofprimingandhabituationplausibly arebasedonsynapticadaptationandcalciumbuildup,longtermmemoryformation requireschangesinthestrength(andpossiblyconnectivity)ofsynapses. Synapticlearninghasbeenextensivelystudiedincomputationalneuroscience,startingwith DonaldHebbs1949suggestionthatcooccurringneuralfiringinitiatedlongtermchange. Sincethenmodelsonalllevelsofabstractionhavebeenstudied,rangingfromthecompletely abstracttodetailedsynapticbiochemistry. Manynetworkmodelsincludevarioussynapticlearningrules.Thereisawidevarietyof rulesused,rangingfromsimpleHebbianruleswithnostatebeyondthecurrentweight,over theBCMruleaccountingforLTP/LTDeffectsinratecodingneurons(Bienenstock,Cooper etal.,1982)andSTDPthatcountstimedifferencesbetweenspikes(Senn,Markrametal., 2001),todetailedsignaltransductionmodelsthatinclude30+substances(Kikuchi,Fujimoto etal.,2003;BhallaandIyengar,1999). Synapsesalsoshowvariousformsofadaptationtorepeatedfiring,includingbothfacilitation anddepression,whichinturncanaffectthenetworkdynamicsinwaysthatappearlikelyto havebehaviourallyrelevantconsequences(Thomson,2000).Variousmodelshavebeen constructedofhowsynapticresourcesaredepletedandreplenished(Tsodyks,Pawelziket al.,1998).Suchmodelsusuallyincludeafewextrastatevariablesinthesynapsesandtime constantsforthem. ForWBEitisimportanttokeepthenumberofstatevariablesandlocalparameterslowin synapses,sincetheydominatethestoragedemandsforneural/compartmentmodelsofthe brain.Atpresent,wedonothaveafirmestimateofhowcomplexsynapsesneedtobein ordertoreproducethefullrangeofplasticityobserved.Itisverylikelythatsimpleweight basedmodelsaretoosimpleandthatmodelswithahandfulofstatevariablescovermost observedphenomena.Whetherthefulltransductioncascadeneedstobesimulatedisunclear. Ontheplusside,thisisanareathathasbeensubjecttointensemodellingandresearchon multiplescalesforseveraldecadesandveryaccessibletoexperimentaltesting.Progressin synapticmodellingmaybeagoodindicatorofneuroscienceunderstanding.
Neural models
ThefirstneuralmodelwastheMcCullochPittsneuron,essentiallybinaryunitssumming weightedinputsandfiring(i.e.sending1ratherthan0asoutput)ifthesumwaslargerthana threshold(Hayman,1999;McCullochandPitts,1943).Thismodelanditscontinuousstate
66
successorsformthebasisofmostartificialneuralnetworkmodels.Theydonothaveany internalstateexceptthefiringlevel.Theirlinktorealbiologyissomewhattenuous,although asanabstractiontheyhavebeenveryfruitful. Morerealisticmodelssuchasintegrateandfiresumsynapticpotentialsandproduce spikes. Singlecellmodellingcanbedoneonroughlyfivelevelsofabstraction(Herz,Gollischetal., 2006):asblackboxmodulesthatgenerateprobabilisticresponsestostimuliaccordingtosome probabilitydistribution,asaseriesoflinearornonlinearfiltersofsignals,asasingle compartmentwithionicconductances,asareducedcompartmentmodel(fewcompartments) orasadetailedcompartmentalmodel.Themoreabstractmodelsaretheoreticallyand computationallytractableandhavefewerdegreesoffreedom,whilethemoredetailed modelsareclosertobiologicalrealismandcanbelinkedtoempiricaldata.Accordingto (Herz,Gollischetal.,2006),mosttaskspecificcomputationofknowndirectbiological relevancecanbeachievedbyreducedcompartmentalmodels,withthepossibleexceptionof someformsofnonlineardendriticprocessing.
Figure21:Compartmentmodelsofneuronsusuallybeginwithanelectrophysiologically characterizedneuron(A).Thiscanberegardedasanetworkofelectricalcableswith individualproperties(B).Thesearethensubdividedintoisopotentialcompartments.The systemcanthenbemodeledasanequivalentelectronicnetwork(C).Eachionchanneltype correspondstoapairofparallelconnectedpotentialsandvariableresistancesper compartment.Compartmentsimulationsnumericallysimulatetheequivalentnetwork8. Imagefrom(BowerandBeeman,1998) Conductancebasedmodelsarethesimplestbiophysicalrepresentationofneurons, representingthecellmembraneasacapacitorandthedifferentionchannelsas(variable) resistances.Neuronsorpartsofneuronsarereplacedbytheirequivalentcircuits,whichare
Seehttp://www.brainsmindsmedia.org/archive/222foratutorialinthismethodology.
67
thensimulatedusingordinarydifferentialequations9.Besidethemembranepotentialthey have(atleast)twogatingvariablesforeachmembranecurrentasdynamicalvariables. Thecoreassumptionsinconductancebasedmodelsarethatdifferentionchannelsare independentofeachother;thegatingvariablesareindependentofeachother,depending onlyonvoltage(orotherfactorssuchascalcium),firstorderkineticsinthegatingvariables; andthattheregionbeingsimulatedisisopotential. Morecomplexionchannelmodelswithinternalstatesinthechannelshavebeendeveloped, aswellasmodelsincludingcalciumdynamics(possiblywithseveralformsofcalcium buffering). Insimpleneuronmodels,theneuronic(firingrateupdate)equationscanbeuncoupled fromthemnemonic(synapticweightupdate)equations,theadiabaticlearning hypothesis(Caianiello,1961).However,realisticmodelsoftenincludeacomplexinterplayat synapsesbetweenmembranepotential,calciumlevels,andconductancesthatmakethis uncouplingdifficulttopreserve. Acommonguidelineinmodellingistousecompartments1/10to1/20ofthelengthconstant10 ofthedendrite(oraxon),makingpotentialdifferencesbetweencompartmentsdifferjustby 5%.Typicallengthconstantsareontheorderof25mm,givingcompartmentssmallerthan 200100m.Inheavilybranchingneurons,theshortdistancebetweenbranchesforcesfiner resolution,ontheorderof10morless.Itcanthereforebeexpectedthatthemajorityof compartmentswillbeduetocorticalarborsratherthanthelongaxonsthroughthewhite matter. Thenumberofstatevariablesofaneuronatleastscaleswiththenumberofsynapsessince eachsynapsehasitsowndynamics.Thenumberofsynapsesisalsoaroughestimateofthe numberofcompartmentsneededforanaccuratemorphologicalmodel.Eachcompartment hastostorealistofneighbourcompartments,dynamicalvariables,andlocalparameters. Synapsescanbetreatedasregularcompartmentswithextrainformationaboutweight, neurotransmitters,andinternalchemicalstate.Asynapseresolutioncompartmentmodelof 1011neuronswithontheorderof104compartmentswouldrequire1015compartments. Ifstatesinextracellularspacematter(e.g.duetovolumetransmission,ephapticsignalsor neurogenesis),itmaybenecessarytodividethesimulationintoaspatialgridratherthan compartmentsfollowingtheneurons.Avolumebasedsimulationwherethebrainisdivided intosizervoxelswouldencompass1.4103/r3voxels.Eachvoxelwouldcontaininformation aboutwhichcells,compartments,andotherinformationthatexistedinside,aswellasalistof thedynamicalvariables(localelectricfields,chemicalconcentrations)andlocalparameter values.For10msidevoxelstherewouldbe1.41018voxelsinahumanbrain.
68
Reducedmodels
*OnlytheMorrisLecarandHodgkinHuxleymodelsarebiophysicallymeaningfulinthe sensethattheyattemptactuallytomodelrealbiophysics,theothersonlyaimforacorrect phenomenologyofspiking. The(Izhikevich,2003)modelisinterestingsinceitdemonstratesthatitmaybepossibleto improvetheefficiencyofcalculationssignificantly(twoordersofmagnitude)withoutlosing toomanyfeaturesoftheneuronactivity.Themodelitselfisatwovariabledynamicalsystem withtwomodelparameters.ItwasderivedfromtheHodgkinHuxleyequationsusinga bifurcationanalysismethodologykeepingthegeometryofphasespaceintact(Izhikevich, 2007).Whileitisnotdirectlybiophysicallymeaningful,itorsimilarreducedmodelsoffull biophysicsmaybepossiblecomputationalshortcutsinbrainemulation.Whethersuch reductionscanbedonedependsonwhetherornotthedetailsoninternalneuralbiophysics areimportantfornetworkrelevantpropertiessuchasexactspiketiming.Itmayalsobe
ToafirstapproximationeachcompartmentwouldrequirethesamenumberofFLOPSmakinga multicompartmentmodelabout34ordersofmagnitudemoredemanding.
11
69
possibletoapplyreductionmethodsonsubneuralmodels,buttheapproachrequiresan understandingofthegeometryofphasespaceofthesystem.
Simulators
Thereexistnumeroussimulationsystemsatpresent.SomeofthemorecommonareGENESIS (GEneralNEuralSImulationSystem)(Wilson,Bhallaetal.,1989;BowerandBeeman,1998) andNeuron(CarnevaleandHines,2006).Forareviewandcomparisons,see(Brette,Rudolph etal.,2007). Keyissuesforneuralsimulatorsarenumericalstability,extendabilityandparallelizability. Thenumericalmethodsusedtointegrateconductancebasedmodelsneedtobothproduce accurateapproximationofsolutionsofthegoverningequationsandrunfast.Thisismade moreproblematicbythestiffnessofsomeoftheequations.Mostneuralsimulatorshavebeen designedtobeeasytoextendwithnewfunctions,oftenproducingverycomplexsoftware systems.ForWBE,thismightnotbenecessaryoncethebasicneurosciencehasbeen elucidated.Neuralsimulatorsneedtobeabletorunonparallelcomputerstoreachhigh performance(seesectionbelow).ThisisakeyneedforWBE.
Parallel simulation
Networks,neurons,andcompartmentsareingeneraljustlinkedtonearbyentitiesandact simultaneously,makingbrainmodelsnaturallysuitedforparallelsimulations.Themain problemisfindingtherightgranularityofthesimulation(i.e.howmanyandwhichentities toputoneachprocessingnode)sothatcommunicationsoverheadisminimized,orfinding communicationsmethodsthatallowthenodestocommunicateefficiently. Simulationscanbetimedrivenoreventdriven.Atimedrivensimulationadvancesone timestepatatimewhileaneventdrivensimulationkeepsaqueueoffutureevents(suchas synapticspikesarriving)andadvancesdirectlytothenext.Forsomeneuralnetworkmodels, suchasintegrateandfire,thedynamicsbetweenspikescanbecalculatedexactly,allowing thesimulationefficientlyjusttojumpforwardintimetowhenthenextspikeoccurs(Mattia andGiudice,2000).However,forhighlyconnectednetworksthetimesbetweenthearrivals ofspikesbecomeveryshort,andtimedrivensimulationsareequallyefficient.Ontheother hand,thetimestepfortimedrivenmodelsmustbeshortenoughthatthediscretizationof spiketimingtoparticulartimestepsdoesnotdisrupttimingpatterns,orvarioustechniques forkeepingsubtimesteptiminginformationmustbeemployedinthesimulation(Morrison, Mehringetal.,2005). Following(Brette,Rudolphetal.,2007),thecomputationalcostpersecondofbiologicaltime isofordercUN/dt+cPFNpfortimedrivensimulationsand(cU+cS+cQ)FNpforeventdriven simulations.cUisthecostperupdate,cPthecostforonespikepropagation(assumedtobe smallerthantheupdatecost),cQthecostofenqueuing/dequeuingaspike(assumedtobe constant),Nthenumberofneurons,Ftherateoffiring,pthenumberofsynapsesperneuron anddtthetimestep.ForF=1Hz,p=10,000anddt=0.1ms,thetimestepdependenttermofthe firstequationislikelytodominate:smallertimestepscanincreasethecomputationalcost strongly.Inthesecondequationthereisnotimeconstantdependency,butthemultiplicative termislikelytobelargeandcansufferifthequeuemanagementhasanontrivialcost.The effectivetimeconstantinaneventdrivensimulationisgoingtobeontheorderof1/Fp, makingthetimeandeventdrivensimulationsaboutequallyfast.Iftherearegapjunctions
70
ordendrodendriticinteractionsthetimecomplexitybecomesmuchhigher(Brette,Rudolph etal.,2007). Wellimplementedsimulationstendstoscalelinearlywithnumberofprocessors,although variousmemoryandcommunicationsbottlenecksmayoccurandoptimaluseofcachingcan giveevensuperlinearspeedupforsomeproblemsizes(Djurfeldt,Johanssonetal.,2005; Migliore,Canniaetal.,2006).Themainproblemappearstobehighconnectivity,sinceinter processorcommunicationsareamajorbottleneck.Keepingcommunicationstoaminimum, forexamplebyonlysendinginformationaboutwhenandwhereaspikehasoccurred (JohanssonandLansner,2007)orrunningdendriticsubtreesonthesameprocessor(Hines, Markrametal.,2008),improvesperformancesignificantly.Ifbrainemulationrequiresmore informationthanthistoflowbetweenprocessingnodesperformancewillbelowerthanthese examples.
71
Anevenlargersimulationwith1011neuronsand1015synapseswasdonein2005byEugene M.IzhikevichonaBeowulfclusterwith273GHzprocessors(Izhikevich,2005).Thiswas achievedbynotstoringthesynapticconnectivitybutbygeneratingitwheneveritwas needed,makingthismodelratherillsuitedforbrainemulation.Onesecondofsimulation took50days,givingaslowdownfactorof4.2million. (JohanssonandLansner,2007)estimatethecomputationaldemandsforsimulatingthe mammalianneocortexofdifferentspecies,basedonaparticularmodelofitsfunction. Assumingthatminicolumnsorganizedintohypercolumnsarethecomputationalunitsand thatthesecommunicatethroughspiketrainsandusingaclusterarchitecture,theyshowthat itisfeasibletorunanetworkofintermediatesizebetweenratandcatonacurrent supercomputercluster.Theyclaimthatusingthiskindofhighlevelnetwork(andassuming continuedlinearscaling)amacaquesizedcortexcouldberunonaBlueGene/Linrealtime.
Conclusion
Currentneuralsimulationstendtohaveneuronnumbersandstructuresfittedtothe availablecomputers(inparticularintermsofconnectivity).Thisisimportantforthelinear scalingofspeedwithprocessornumberandgivesaperformanceimprovementatthe expenseofgenerality.WBEscalecomputingwilllikelyhaveacomputationalgranularity fittedtothestructureofthebrain,unlessthecomputationalpoweravailableissolargethat inefficienciesduetoaloosefitareirrelevant. Modelsarecurrentlynotdrivenbyscanneddata,andindividualvariationsinneuron propertiesaregeneratedbydrawingfromapresetrandomdistribution.Whilethisallows modelsthatgenerateneededparametersonlywhenrequiredandhenceavoidstoringthem, thisisrelativelyrareandmostimplementationsdousestoredparameters.Hencetheredoes notseemtobeanydifferencebetweenstoragerequirementsforWBEmodelsandrandom models,assumingthesameunderlyingparametersandstructure. Thespeedupofdifferentnetworksrangesoversixordersofmagnitude,butmostarea hundredfoldtoathousandfoldslowerthanbiology,andafewareclosetorealtime.Thisis likelymoreduetoresearchpracticethananyinherentlimitation:modelsizesareselectedso thattheyfitavailablecomputingpower.Thelengthofthesimulationissettoproducedata comparabletosomebiologicalmeasurement(which,forneuralnetworks,tendtobeafew secondslongforneurocognitivelyinterestingcases),andthelengthofsimulationtime correspondstowhatconstitutesanacceptableturnaroundtimeforacomputercentreoran officecomputer.Smallersimulationscanrunfasteruntiltheyhitbottleneckssetbyinter processorcommunicationsspeed.ItishencelikelythatevenearlyWBEwillbeclosetoreal time,withcomputationalconstraintsratherthanspeedlimitingthelengthofthe simulation. Weclearlytodayhavecomputationalcapabilitiessufficienttosimulatemanyinvertebrate brains(suchassnails,ants,andfruitflies)usingcompartmentmodelsonparallelclustersof modestsize.Smallmammalianbrainsappearcomputationallywithinreach. Giventhedifferencesbetweenmodels,implementation,computersandotherparametersitis hardtoreliablycomparecurrentlargescalemodelstoestimatetrends.Thereisalsoalackof historicaldata,asonlyrecentlylargemodelshavebecomepracticallypossible.Hencea conservativeestimateassumesthatthemodelswillgrowbasedsolelyoncomputerpower
72
73
Bodysimulation
Thebodysimulationtranslatesbetweenneuralsignalsandtheenvironment,aswellas maintainsamodelofbodystateasitaffectsthebrainemulation. Howdetailedthebodysimulationneedstobeinordertofunctiondependsonthegoal.An adequatesimulationproducesenoughandtherightkindofinformationfortheemulation tofunctionandact,whileaconvincingsimulationisnearlyorwhollyindistinguishablefrom thefeeloftheoriginalbody. Anumberofrelativelysimplebiomechanicalsimulationsofbodiesconnectedtosimulated nervoussystemshavebeencreatedtostudylocomotion.(Suzuki,Gotoetal.,2005)simulated theC.elegansbodyasamultijointrigidlinkwherethejointswerecontrolledby motorneuronsinasimulatedmotorcontrolnetwork.rjanEkeberghassimulated locomotioninlamprey(EkebergandGrillner,1999),stickinsects(Ekeberg,Blmeletal., 2004),andthehindlegsofcat(EkebergandPearson,2005)wherearigidskeletonismoved bymuscleseithermodeledasspringscontractinglinearlywithneuralsignals,orinthecase ofthecat,amodelfittingobserveddatarelatingneuralstimulation,length,andvelocitywith contractionforce(Brown,Scottetal.,1996).Thesemodelsalsoincludesensoryfeedbackfrom stretchreceptors,enablingmovementstoadapttoenvironmentalforces:locomotioninvolves aninformationloopbetweenneuralactivity,motorresponse,bodydynamics,andsensory feedback(Pearson,Ekebergetal.,2006). Todaybiomechanicalmodelsoftwareenablesfairlydetailedmodelsofmuscles,theskeleton, andthejoints,enablingcalculationofforces,torques,andinteractionwithasimulated environment(BiomechanicsResearchGroupInc,2005).Suchmodelstendtosimplifymuscles aslinesandmakeuseofprerecordedmovementsortensionstogeneratethekinematics. Adetailedmechanicalmodelofhumanwalkinghasbeenconstructedwith23degreesof freedomdrivenby54muscles.However,itwasnotcontrolledbyaneuralnetworkbutrather usedtofindanenergyoptimizinggait(AndersonandPandy,2001).Astateoftheartmodel involving200rigidboneswithover300degreesoffreedom,drivenbymuscularactuators withexcitationcontractiondynamicsandsomeneuralcontrol,hasbeendevelopedfor modellinghumanbodymotioninadynamicenvironment,e.g.forergonomicstesting (IvancevicandBeagley,2004).Thismodelrunsonanormalworkstation,suggestingthat rigidbodysimulationisnotacomputationallyhardproblemincomparisontoWBE. Otherbiomechanicalmodelsarebeingexploredforassessingmusculoskeletalfunctionin human(FernandezandPandy,2006),andcanbevalidatedorindividualizedbyuseofMRI data(Arnold,Salinasetal.,2000)orEMG(LloydandBesier,2003).Itisexpectedthatnear futuremodelswillbebasedonvolumetricmuscleandbonemodelsfoundusingMRI scanning(Blemker,Asakawaetal.,2007;BlemkerandDelp,2005),aswellasconstructionof topologicalmodels(MagnenatThalmannandCordier,2000).Therearealsovarious simulationsofsofttissue(Benham,Wrightetal.,2001),breathing(Zordan,Cellyetal.,2004) andsofttissuedeformationforsurgerysimulation(Cotin,Delingetteetal.,1999). Anothersourceofbodymodelscomesfromcomputergraphics,wheremuchefforthasgone intorenderingrealisticcharacters,includingmodellingmuscles,hairandskin.Theemphasis hasbeenonrealisticappearanceratherthanrealisticphysics(Scheepers,Parentetal.,1997), butincreasinglythemodelsarebecomingbiophysicallyrealisticandoverlappingwith
74
biophysics(ChenandZeltzer,1992;Yucesoy,Koopmanetal.,2002).Forexample,30 contact/collisioncoupledmusclesintheupperlimbwithfasciaandtendonsweregenerated fromthevisiblehumandatasetandthensimulatedusingafinitevolumemethod;this simulation(usingonemillionmeshtetrahedra)ranatarateof240secondsperframeona singleCPUXeon3.06GHz(ontheorderofafewGFLOPS)(Teran,Sifakisetal.,2005).Scaling thisup20timestoencompass600musclesimpliesacomputationalcostontheorderofa hundredTFLOPSforacompletebodysimulation. Physiologicalmodelsareincreasinglyusedinmedicineforeducation,researchandpatient evaluation.Relativelysimplemodelscanaccuratelysimulatebloodoxygenation(Hardman, Bedforthetal.,1998).Forabodysimulationthismightbeenoughtoprovidetheright feedbackbetweenexertionandbrainstate.Similarlysimplenutrientandhormonemodels couldbeusedinsofararealisticresponsetohungerandeatingweredesired.
Conclusion
Simulatingarealistichumanbodyiskinematicallypossibletoday,requiringcomputational powerrangingbetweenworkstationsandmainframes.Forsimplerorganismssuchas nematodesorinsectscorrespondinglysimplermodelscould(andhave)beenused.Sincethe needforearlyWBEismerelyadequatebodysimulation,thebodydoesnotappeartoposea majorbottleneck. However,sincemanymotoractionsinvolvearichinterplaybetweenmechanicsandnerve signalsitshouldnotbesurprisingifrelativelycomplexmodelsareneededforapparently simpleactionssuchasstandingorwalking.Newlystartedemulationsmayneedtimeand efforttolearntousetheirunfamiliarbodies.
75
Environmentsimulation
Theenvironmentsimulationprovidesasimulatedphysicalenvironmentforthebody simulation.Onecanagainmakethedistinctionbetweenanadequateenvironmentsimulation andaconvincingsimulation.Anadequateenvironmentproducesenoughinputtoactivate thebrainemulationandallowittointeractinsuchawaythatitsstateandfunctioncanbe evaluated.Aconvincingsimulationiscloseenoughtorealitythatthekindsofsignalsand interactionthatoccursishard(orimpossible)fortheemulatedbeingtodistinguishfrom reality. Itseemslikelythatwealreadyhavethetoolsformakingadequateenvironmentsintheform ofe.g.game3Drenderingengineswithphysicsmodelsorvirtualenvironmentssuchas SecondLife.Whilenotcoveringmorethansightandsound,theymightbeenoughfortesting anddevelopment.ForemulationsofsimplerbrainssuchasC.eleganssimulationswith simplifiedhydrodynamics(similarto(EkebergandGrillner,1999))maybeenough,possibly extendedwithsimulatedchemicalgradientstoguidebehaviour. Convincingenvironmentsmightbenecessaryonlyifthelongtermmentalwellbeingof emulatedhumans(orothermammals)isatstake.Whileitispossiblethatahumancould adapttoamerelyadequateenvironment,itseemslikelythatitwouldexperiencesuchan environmentasconfiningorlackinginsensorystimulation.Notethateveninaconvincing environmentsimulationnotalldetailshavetofitphysicalrealityperfectly(Bostrom,2003). Plausiblesimulationismoreimportantthanaccuratesimulationinthisdomainandmay actuallyimprovetheperceivedrealism(Barzel,Hughesetal.,1996).Inaddition,humans acceptsurprisinglylargedistortions(20%lengthchangeofobjectswhennotpayingdirect attention,3%whenpayingattention(Harrison,Rensinketal.,2004)),allowingagreatdealof leewayinconstructingaconvincingenvironment. Whatqualityofenvironmentisneededtocompletelyfoolthesenses?Inthefollowingwe willassumethatthebrainemulationrunsinrealtime,i.e.,thatonesecondofsimulationtime correspondstoonesecondofoutsidetime.Forsloweremulations,theenvironmentmodel wouldbeslowedcomparably,andallcomputationaldemandsdividedbythescalefactor. Atthecoreoftheenvironmentmodelwouldbeaphysicsenginesimulatingthemechanical interactionsbetweentheobjectsintheenvironmentandthesimulatedbody.Itwouldnot onlyupdateobjectpositionsdependingonmovementandmaintainaplausiblephysics,it wouldalsoprovidecollisionandcontactinformationneededforsimulatedtouch.Ontopof thisphysicssimulationaseriesofrenderingenginesfordifferentsenseswouldproducethe rawdataforthesensesinthebodymodel.
Vision
Visualphotorealismhasbeensoughtincomputergraphicsforabout30years,andthis appearstobeafairlymatureareaatleastforstaticimagesandscenes.Mucheffortis currentlygoingintosuchtechnology,foruseincomputergamesandmovies. (McGuigan,2006)proposesagraphicsTuringtestandestimatesthatfor30Hzinteractive visualupdates518.41036.8TFLOPSwouldbeenoughforMonteCarloglobalillumination. Thismightactuallybeanoverestimatesinceheassumesgenerationofcompletepictures. Generatingonlythesignalneededfortheretinalreceptors(withhigherresolutionforthe
76
Hearing
Thefullacousticfieldcanbesimulatedoverthefrequencyrangeofhumanhearingby solvingthedifferentialequationsforairvibration(Garriga,Spaetal.,2005).Whileaccurate, thismethodhasacomputationalcostthatscaleswiththevolumesimulated,upto16TFLOPS fora222mroom.Thiscanlikelybereducedbytheuseofadaptivemeshmethods,orray orbeamtracingofsound(Funkhouser,Tsingosetal.,2004). Soundgenerationoccursnotonlyfromsoundsourcessuchasinstruments,loudspeakers,and peoplebutalsofromnormalinteractionsbetweenobjectsintheenvironment.Bysimulating surfacevibrations,realisticsoundscanbegeneratedasobjectscollideandvibrate.Abasic modelwithNsurfacenodesrequires0.5292NGFLOPS,butthiscanbesignificantlyreduced bytakingperceptualshortcuts(RaghuvanshiandLin,2006;RaghuvanshiandLin,2007).This formofvibrationgenerationcanlikelybeusedtosynthesizerealisticvibrationsfortouch.
Haptics
Thehapticsensesoftouch,proprioception,andbalancearecrucialforperformingskilled actionsinrealandvirtualenvironments(RoblesDeLaTorre,2006). Tactilesensationrelatesbothtotheforcesaffectingtheskin(andhair)andtohowtheyare changingasobjectsorthebodyaremoved.Tosimulatetouch,stimulicollisiondetectionis neededtocalculateforcesontheskin(andpossiblydeformations)aswellasthevibrations whenitismovedoverasurfaceorexploringitwithahardobject(Klatzky,Ledermanetal., 2003).Toachieverealistichapticrendering,updatesinthekilohertzrangemaybenecessary (LinandOtaduy,2005).Inenvironmentswithdeformableobjectsvariousnonlinearitiesin responseandrestitutionhavetobetakenintoaccount(MahvashandHayward,2004).
77
Proprioception,thesenseofhowfarmusclesandtendonsarestretched(andbyinference, limblocation)isimportantformaintainingpostureandorientation.Unliketheothersenses, proprioceptivesignalswouldbegeneratedbythebodymodelinternally.SimulatedGolgi organs,musclespindles,andpulmonarystretchreceptorswouldthenconvertbodystates intonerveimpulses. Thebalancesignalsfromtheinnerearappearsrelativelysimpletosimulate,sinceitisonly dependentonthefluidvelocityandpressureinthesemicircularchannels(whichcanlikely beassumedtobelaminarandhomogeneous)andgravityeffectsontheutricleandsaccule. Comparedtoothersenses,thecomputationaldemandsareminuscule. Thermoreceptioncouldpresumablybesimulatedbygivingeachobjectinthevirtual environmentatemperature,activatingthermoreceptorsincontactwiththeobject. Nocireception(pain)wouldbesimulatedbyactivatingthereceptorsinthepresenceof excessiveforcesortemperatures;theabilitytoexperiencepainfromsimulatedinflammatory responsesmaybeunnecessaryverisimilitude.
Conclusion
Renderingaconvincingenvironmentforallsensesprobablyrequiresontheorderofseveral hundredTFLOPS.Whilesignificantbytodaysstandards,thisrepresentsaminusculefraction ofthecomputationalresourcesneededforbrainemulation,andisnotnecessaryformeeting thebasicsuccesscriteriaofemulation.
78
Computerrequirements
WBErequiressignificantcomputerpowerandstorageforimageprocessingand interpretationduringthescanningprocess,andtoholdandruntheresultingemulation.Both problemsappeartobestronglyparallelizable. Awayofestimatingthedistancebetweencurrentcapabilitiesandthoseneededforhuman WBEistoestimatethenumberofentities/statevariablesneededtospecifytheemulation,the requiredtimeresolution,andcomparethistotrendsincomputerhardware.Thiswilldepend onwhatlevelofdetailaWBEcanbeachievedat;fullmolecularsimulationwouldrequire vastlymorecomputationalpowerthanamodelusingsimplifiedneurons.Foragivenlevelof simulation,wecanthenestimatetheearliestpossibledateassumingthatMooreslaw continuesunchangedwhenthatkindofemulationwillbepossibleforagivenamountof money. AppendixBanalysescurrentcomputingtrendsindetail.Themainconclusionisthatmemory perdollarincreasesoneorderofmagnitudeper4.8yearsandprocessingpowerperdollar increasesoneorderofmagnitudeper3.7,3.9or6.4yearsdependingonwhetheronebasesthe predictiononsupercomputerprice/performance,supercomputerpowerorcommodity computers.Wewillusetheoptimisticsupercomputerestimateandthemorecautious commodityestimatestogetanoverallrange. Itshouldbenotedthattheestimatesbelowmakemerelyorderofmagnitudeestimatesofthe numberofentitiesandcomplexityoftheirstorage;theyarequitedebatable.However,given thatanorderofmagnitudecomplexityincreaseonlyaddscirca5yearstotheestimate,exact numbersarenotnecessary.Wearealsoignoringbodyandenvironmentsimulationbecause, asshowninthenextsections,theylikelyrequireonlyafractionofthebrainemulation computations. Table8:Storagedemands(emulationonly,humanbrain)
Level 1 2 Computational module Brainregion connectivity Analognetwork populationmodel #entities 1001,000? 105regions,107 connections 108populations,1013 connections. Bytesper entity ? 3?(2byte connectivity,1 byteweight) 5(3byte connectivity,1 byteweight,1 byteextrastate variable) 8(4byte connectivity,4 statevariables) 1byteperstate variable 1byteperstate variable 1byteperstate variable 1byteperstate variable 1byteperstate Memory demands(Tb) ? 3105 Earliestyear, $1million ? Present
50
Present
1011neurons,1015 connections. 1015compartmentsx10 statevariables=1016. 1016compartmentsx102 metabolites=1018. 1016compartmentsx103 proteinsandmetabolites =1019. 1016compartmentsx103 proteinsx10states=1020 1016compartmentsx103
8,000
2019
5 6 7
8 9
108 109
2038 2043
79
variable 1byteper voxel, compressed. 31(2bytes moleculetype, 14bytes position,14 bytesvelocity,1 bytestate) Qbits 109 2043
10
1025molecules
3.11014
2069
11
Quantum
ForthecaseofaManhattanprojectspending$109,subtract14.4yearsfromtheseestimates. Table9:Processingdemands(emulationonly,humanbrain)
Level #entities FLOPS per entity Timesteps persecond CPU demand (FLOPS) Earliest year,$1 million (commod ity computer estimate) ? ? Earliest year,$1 million (Super computer estimate) ? ?
1 2
Computational module Brainregion connectivity Analog network population model Spikingneural network Electrophysiol ogy
1001,000? 105regions, 107 connections 108 populations, 1013 connections 1011neurons, 1015 connections 1015 compartments x10state variables= 1016. 1016 compartments x102 metabolites= 1018. 1016 compartments x103proteins and metabolites= 1019. 1016 compartments x103proteins x10states= 1020 1016
? ?
? ?
? ?
102
1015
2023
200815
10
103
1018
2042
2019
Metabolome
104
1022
2068
2033
104
1025
2087
2044
Proteome
103
104
1026
2093
2048
103
104
1027
2100
2052
9
15
Distributionof
103
106
1030
2119
2063
RoadrunneratLosAlamosNationalLaboratoryachieved1.7petaflopsonMay25,2008.
80
complexes
10
11
103
1015
1043
2201
2111
Qbits
10131020
ForthecaseofaManhattanprojectspending$109,subtract19.1/11.1yearsfromthese estimates,respectively. Aroughestimateforsimplerbrainsisthatamacaquebrainhas14%ofthehumansynapses, catbrains3%,rat0.26%,mouse0.1%(JohanssonandLansner,2007).Assumingsimulation demandsscalebysynapsenumber(likelyforcompartmentmodels)thismeansmacaque emulationsonagivenlevelcanbeachieved5.4/3.2yearsearlier,catemulations9.7/5.6years, ratemulations16/9.6yearsandmouseemulations19/11(dependingonwhichcolumnabove isused).Emulationsofhoneybees(950,000neurons)andaplysia(20,000)appearfeasible todayatafairlyhighscale(51/30,61/36yearsearlierrespectively).Aslowerdecadetimeof computerimprovementproducesalongergapbetweenanimalemulationsandhuman emulations.
Conclusions
Itappearsfeasiblewithintheforeseeablefuturetostorethefullconnectivityoreven multistatecompartmentmodelsofallneuronsinthebrainwithintheworkingmemoryofa largecomputingsystem. Achievingtheperformanceneededforrealtimeemulationappearstobeamoreserious computationalproblem.However,theuncertaintiesinthisestimatearealsolargersinceit dependsonthecurrentlyunknownnumberofrequiredstates,thecomputationalcomplexity ofupdatingthem(whichmaybeamenabletodrasticimprovementsifalgorithmicshortcuts canbefound),thepresumedlimitationofcomputerhardwareimprovementstoaMoores lawgrowthrate,andtheinterplaybetweenimprovingprocessorsandimproving parallelism16.Aroughconclusionwouldneverthelessbethatifelectrophysiologicalmodels areenough,fullhumanbrainemulationsshouldbepossiblebeforemidcentury.Animal modelsofsimplemammalswouldbepossibleonetotwodecadesbeforethis.
81
Validation
Ascomputersoftwareincreasedincomplexity,previousmethodsofdebuggingbecame insufficient,especiallyassoftwaredevelopmentmovedfromsmallgroupstolargeprojectsin largeorganisations.Thisledtothedevelopmentofanumberofsoftwaretesting methodologiesaimingatimprovingquality(GelperinandHetzel,1988).Currently neuroscienceappearstobeinanearlydebuggingparadigmwheredataandprocedures certainlyaretestedinvariousways,butusuallyjustthroughreplicationorasanadhoc activitywhensomethingunexpectedoccurs.Forlargescaleneurosciencetestingand validationmethodsneedtobeincorporatedintheresearchprocesstoensurethattheprocess works,thatthedataprovidedtootherpartsoftheresearchisaccurateandthatthelink betweenrealityandmodelisfirm. Animportantearly/ongoingresearchgoalwouldbetoquantifytheimportanceofeachlevel ofscaleofphenomenatotheresultantobservedhigherbrainfunctionsofinterest.In particular,itisimportanttoknowtowhatlevelofdetailtheyneedtobesimulatedinorderto achievethesamekindofemergentbehaviouronthenextlevel.Insomecasesitmightbe possibletoprovethisbyperformingsimulations/emulationsatdifferentlevelsof resolutionandcomparingtheirresults.Thiswouldbeanimportantapplicationofearly smallscaleemulationsandwouldhelppavethewayforlargerones. Ideally,agoldstandardmodelatthehighestpossibleresolutioncouldbeusedtotestthe extenttowhichitispossibletodeviatefromthisbeforenoticeableeffectsoccur,andto determinewhichfactorsareirrelevantforhigherlevelphenomena.Someofthisinformation mayalreadyexistintheliterature,someneedstobediscoveredthroughnewcomputational neuroscienceresearch.Explorationofmodelfamilieswithdifferentlevelsofbiologicaldetail isalreadydoneoccasionally. Acomplementaryapproachistodevelopmanufactureddatawherethegroundtruthis known(phantomdatasets),andthenapplyreconstructionmethodsonthisdatatoseehow welltheycandeducethetruenetwork.Forexample,theNETMORPHsystemmodelsneurite outgrowthwhichcreatesdetailedneuralmorphologyandnetworkconnectivity(Koene, 2008).Thiscanthenbeusedtomakevirtualslices.Multipledatasetscanbegeneratedtotest theoverallreliabilityofreconstructionmethods.
82
Discussion
Asthisreviewshows,WBEontheneuronal/synapticlevelrequiresrelativelymodest increasesinmicroscopyresolution,alesstrivialdevelopmentofautomationforscanningand imageprocessing,aresearchpushattheproblemofinferringfunctionalpropertiesof neuronsandsynapses,andrelativelybusinessasusualdevelopmentofcomputational neurosciencemodelsandcomputerhardware.Thisassumesthatthisistheappropriatelevel ofdescriptionofthebrain,andthatwefindwaysofaccuratelysimulatingthesubsystems thatoccuronthislevel.Conversely,pursuingthisresearchagendawillalsohelpdetect whethertherearelowleveleffectsthathavesignificantinfluenceonhigherlevelsystems, requiringanincreaseinsimulationandscanningresolution. Theredonotappeartoexistanyobstaclestoattemptingtoemulateaninvertebrateorganism today.Wearestilllargelyignorantofthenetworksthatmakeupthebrainsofevenmodestly complexorganisms.Obtainingdetailedanatomicalinformationofasmallbrainappears entirelyfeasibleandusefultoneuroscience,andwouldbeacriticalfirststeptowardsWBE. Suchaprojectwouldserveasbothaproofofconceptandatestbedforfurtherdevelopment. IfWBEispursuedsuccessfully,atpresentitlooksliketheneedforrawcomputingpowerfor realtimesimulationandfundingforbuildinglargescaleautomatedscanning/processing facilitiesarethefactorsmostlikelytoholdbacklargescalesimulations.
83
AppendixA:Estimatesofthecomputational capacity/demandsofthehumanbrain
Themostcommonapproachisastraightforwardmultiplicativeestimate:giventhenumberof neurons,theaveragenumberofsynapsesandanassumedamountofinformationper synapseornumberofoperationspersecondpersynapse.Thismultiplicativemethodhas beenappliedtomicrotubuliandproteinstoo. However,itstillmightbenecessarytostoreconcentrationsofseveralchemicalspecies, neurotransmittertypesandotherdataifabiologicallyrealisticmodelisneeded(especially theidentitiesofthepreandpostsynapticneurons).Someestimatesofthestorage requirementsofbrainemulationareincludedinthetablebelow. Otherestimationmethodsarebasedonanalogyorconstraints.(Moravec,1999)suggested exploitingtheknownrequirementsofimageprocessingbyequatingthemwitha correspondingneuralstructure(theretina),andthenscalinguptheresult.(Merkle,1989a) usedenergyconstraintsonelementaryneuraloperations.(Landauer,1986)attemptedan estimationbasedonexperimentalpsychologicalmemoryandsignaltheory. Assumptionontheorderofonebitofinformationpersynapsehassomesupporton theoreticalgrounds.Modelsofassociativeneuralnetworkshaveaninformationstorage capacityslightlyunder1bitpersynapsedependingonwhatkindofinformationisencoded (Nadal,1991;NadalandToulouse,1990).Extendingthedynamicsofsynapsesforstoring sequencedatadoesnotincreasethiscapacity(RehnandLansner,2004).Geometricaland combinatorialconsiderationssuggest35bitspersynapse(Stepanyants,Hofetal.,2002; Kalisman,Silberbergetal.,2005).FittingtheoreticalmodelstoPurkinjecellssuggeststhat theycanreach0.25bits/synapse(Brunel,Hakimetal.,2004). Table10:Estimatesofcomputationalcapacityofthehumanbrain.Unitshavebeen convertedintoFLOPSandbitswheneverpossible.LevelsrefertoTable2.
Source (Leitl,1995) Assumptions Assuming1010neurons,1,000 synapsesperneuron,34bitIDper neuronand8bitrepresentationof dynamicstate,synapticweights anddelays.[Level5] Assumingmicrotubulidimerstates asbitsandoperatingon nanosecondswitchingtimes. [Level10] Basedon100billionneuronswith 1,000connectionsand200 calculationspersecond.[Level4] Arguesthatthenumberof computationalelementsinthe brainisgreaterthanthenumberof neurons,possiblyevenuptothe 1017individualproteinmolecules. [Level8] Assuming2bitslearningper secondduringconscioustime, experimentbased.[Level1] Computational demands Memory 51015bits(butnotes thatthedatacanlikely becompressed). 81019bits
(Tuszynski,2006)
1028FLOPS
(Kurzweil,1999) (Thagard,2002)
21016FLOPS
1012bits
1023FLOPS
(Landauer,1986)
1.5109bits(109bits withloss)
84
(Merkle,1989a) (Dix,2005)
Storingallimpulsesoveralifetime. Memoriesarestoredasrelations betweenneurons. 1010neurons,1,000synapses,firing 10Hz[Level4] 1011neurons,5103synapses,100 Hz,eachsignalworth5bits.[Level 5] EnergyconstraintsonRanvier nodes. Comparesinstructionsneededfor visualprocessingprimitiveswith retina,scalesuptobrainand10 timespersecond.Produces1,000 MIPSneurons.[Level3] Retinascaleup.[Level3] 10billionneurons,10,000synaptic operationspercycle,100Hzcycle time.[Level4] 1010neurons,1,000synapseseach. [Level4] 1014synapses,identitycodedby48 bitsplus2x36bitsforpreand postsynapticneuronid,1byte states.10msupdatetime.[Level4] 50200billionneurons,20,000 sharedsynapsesperneuronwith 256distinguishablelevels,40Hz firing.[Level5] 1011neurons,102104synapses,100 1,000Hzactivity.[level4] 11011neurons,eachwith104 compartmentsrunningthebasic HodgkinHuxleyequationswith 1200FLOPSeach(basedon (Izhikevich,2004)).Each compartmentwouldhave4 dynamicalvariablesand10 parametersdescribedbyonebyte each.
81014bits.
(Cherniak,1990) (Fiala,2007)
256,000terabytes/s
(Seitz)
21012synaptic operationspersecon
4101581015bits
(Malickas,1996)
1.121028bits
17ThisinformationdensityisfarlargerthantheBekensteinblackholeentropyboundontheinformation contentinmaterialsystems(Bekenstein,1981).
85
AppendixB:ComputerPerformance Development
Forecastsoffuturecomputerperformanceareneededtomakeroughestimatesofwhenbrain emulationwillbecomefeasibleintermsofcomputerstorageandprocessingcapacity.The followingestimatesare(withtwoexceptions)basedondatafromJohnC.McCallums datasetsofCPUpriceperformance(McCallum,2003),memoryperformance(McCallum, 2007b)anddiskperformance(McCallum,2007a).
Processing Power
Plottingtheperformanceofcomputersystemsgivesanestimateofavailabletotalcomputer powerofunitary,offtheshelfsystems(Figure23).
86
Figure24:Processingpowerperdollarovertime. Amorerelevantmeasureistheamountofcomputingpoweradollarcanbuy.Usingthedata andadjustingforinflationto2007dollars(thishasbeendoneinallestimatesbelow)wegeta decadetimeofjust5.6years(Figure24).Thebootstrap95%confidenceintervalis5.35.9. Presentperformanceisabout1MIPSperdollar.Thetrendappearsreliable,butpossibly accelerating. Istherateofdevelopmentchanging?Fittingcurvesto20yearintervalsoftheMIPS/$ estimatesproducesarangeofestimatesofthedevelopmentexponent,clusteringtogether intotwoorthreegroups.Thedecadetimesareroughly4.4,8.7and3.5yearsrespectively.The slowestdevelopmentspeedoccurredduringthe70sand80s,perhapsduetothe proliferationofcheaphomecomputersbuiltwithlesspowerfulprocessorsforeconomical reasons.Sincethenithaspickedupspeedagain. Fittingtocomputersinthesamepriceclasses(pricesequalwithinanorderofmagnitude) producedecadetimesrangingfrom4.3to7.1yearswiththecheapestcomputersbecoming moreeconomicalfastest.Clearly,thecommoditymarketisdrivingdevelopmentfast(but parallelisationhasthepotentialtospeeddevelopmentevenfurther,asdescribedbelow). Giventheseresultsitlookslikelythatthetrendwillcontinue(forsomeunknownduration) withadecadetimeofbetween3.5and8.7years,with5.6asamiddleestimate.
FLOPSvs.MIPS
ThesemeasurementshaveusedMIPSasameasureofcomputationalpower,butitisnota reliableindicatorofactualperformanceinnumericheavyapplications(e.g.see(Nordhaus,
87
2001)foracriticism).Forthat,theMFLOPS(MillionFLoatingPointinstructionsperSecond) ismoresuitable.Unfortunately,itisnotaswidelytestedastheMIPS.
88
89
Figure27:FittedconversionexponentfromMIPStoFLOPSfordifferent5yearintervals. Fittingtosmallertimeintervals(length5years)from1970to2000producedarangeof exponents(Figure27).Totesttheirreliabilitytheirvalueswerecalculatedforeachinterval severaltimes,eachtimewithadifferentsampleremoved,producingadistribution(shownas greydots)ofexponentsthatallowedacalculationofmeanandstandarddeviation(shownin blue).Themorereliableexponentsareallcloseto1,withastatisticallysignificantincrease since1986until1995.However,anylongtermtrendtowardshigherexponentsseemshardto support. Istheeffectdifferentinsmallercomputersthansupercomputers?PlottingMIPSvsFLOPS, colouringthesamplesbytheirprice,andthenfittingexponentialsproducesthefollowing graph(Figure28).
90
Figure28:RelationbetweenMIPSandFLOPSfordifferentpricerangecomputers. Thisgraphseemstosuggestthatthemoreexpensive(andpresumablymorepowerful computers)producefewerFLOPSperMIPSthanthecheaperones,whichhaveaslightly superlinearrelation.However,thismaybebiasedbythesmallsampleofcomputersinthe datasetforwhichinformationaboutbothFLOPS,MIPS,andpriceisavailable. Altogether,assumingthatFLOPSgrowasMIPStothepowerof0.8isprobablyasafe assumption,buttheexponentcouldbecome>1iftherewereafocusonhighperformance calculationinprocessors(orothercomputerpartssuchasgraphicscards).Thereisalsoa sizeablespreadofproportionalityconstants,atleasttwoordersofmagnitude. Puttingthepreviousconsiderationstogether,andassuminga1MIPS/$2007weshould expectFMFLOPSofperformanceachievablewithapriceofPdollarsin(T/0.8)log10(F/2.3P0.8) years,assumingadecadetimeofT.
Otherestimates
WilliamD.Nordhaushasdonehistoricalestimateofcomputerperformancesincethe1800s, includingpreelectroniccomputers(Nordhaus,2001)(partiallybasedondatafrom(Morevec, 1988)).Heusesmillionsofstandardizedoperationsperseconds(MSOPS)ashismeasure,1 MSOPSroughlycorrespondingto20million32bitadditions18.MSOPScorrelatesstrongly withadditionspersecondandcyclespersecond.
18ThecalculationusedisSOPS=0.05((6+log2(memory)+wordlength)/((7xaddtime+multiplication
time)/8))
91
92
Figure30:PeakperformanceofTop500datasetofsupercomputers. TheTop500supercomputinglist19tabulatesthe500mostpowerfulsupercomputersinthe world.Over19932003thesemachineshadadoublingtimefortheLinpacktestof13.5 months(Feitelson,2005),givingadecadetimeofjust3.82years.Usingthefulldataset1993 2008givesaslightlyshorterdecadetime,3.7years(thisoccursbothwhenusingthegeneral Linpackperformanceorthepeakperformance).Alikelyreasonforthefastergrowthrate thansingleprocessorsystemsisthatthenumberofprocessorspercomputerisalso increasingexponentially.Whilethedatadoesnotincludepriceinformation,theatpresent (June2008)topsystemachieves1petaflopsperformancefor$100million,achieving10 megaflops/$.Thisappearstobreakprevioustrendpredictions,suggestingthatextremehigh performancesystemsmayactuallybeimprovingmuchfasterthanthesmallersystems. However,assmallersystemsbegintouseparallelcomputationthesamerapidexpansion mayoccurtheretoo.
19
www.top500.org
93
Figure31:Projectedsupercomputerperformancefromwww.top500.org.
EstimatingavailablecomputingpowerforWBE
TherearethreewaysofestimatingthecomputingpoweravailablefordoingWBE.Thefirstis toextrapolatetheperformancetopriceratioofcommoditycomputers,thesecondistouse topsupercomputerperformancedirectly,thethirdistoestimatetheperformancetoprice ratioforsupercomputers.Allthreehavemeritsandproblems. Commoditycomputersrepresentalowerbound:theyarenotveryparallelsofarandthe economicforcesdrivingthemarenotallaimingformaximumperformanceoreven performanceperdollar.Wehavedatastretchingbacklongerforthemthanforhigh performancecomputers.Usingtheabovedataproducestheformula [MFLOPS/$]=2.33100.88(t2007)/D wheretistheyearandDisthedecadetimeofMIPS,5.6years(between3.5and8.7years with95%confidence). Absolutesupercomputerperformancefollowedaveryregularexponentialgrowthbetween 1993and2006,andhasenabledreliableshorttermpredictions(StrohmaierandMeuer,2004). Thegrowthisfasterthanforcommoditycomputersduetorapidlyincreasingparallelisation. However,thereisreasontobelievethisparallelisationgrowthmayconflictwiththe requirementsofenergy,coolingandspaceinthenearfuture(seebelow).SinceaWBEproject islikelytobeamajorresearchundertaking,havingatop500supercomputeravailableisvery likely(severalofthecurrenttopsupercomputershavebeenusedforcomputational neuroscience);thefullcostofthecomputermaybesharedbetweentheWBEprojectand otherprojects.
94
Assumingacomputeronparwiththe#500computer,theavailablepowergrowsas [FLOPS]=8.99610610(t2008)/D WhereDis3.92.Uptotwoordersofmagnitudemorecomputerpowerisavailablebyusing morepowerfulcomputers.Thecostofthecomputeritselfwouldbeontheorderof$10 million. Estimatingsupercomputerperformance/priceratiosishardersincetheirpricesarenot generallylisted.UsingtheRoadrunnercomputertocalibrate(~$100million)wouldgivea currentconversionfactorof1.026petaflops/$100million,increasingwithadecadetimeof 3.7years.Therangeofratiosateachtimealsolikelyspansatleastanorderofmagnitude:the personalsupercomputerMicrowulfsprice/performanceratio(2007)was$48/Gflop20 whiletheSunsSparcEnterpriceM9000mainframe(basepriceof$511,385)produced1.03 TFLOPSofmeasuredperformance,makingitsPPR>$496/Gflop.Similarlycustombuilt hardwarecanlikelyimprovetheratiobyatleast12ordersofmagnitude(Wehner,Olikeret al.,2008).Usingthiswegetanestimateof [MFLOPS/$]=10.2610(t2008)/D WhereD=3.7yearsandtheuncertaintyisatleastoneorderofmagnitudeatanypointintime.
Memory
Figure32:RAMmemoryperdollarovertime.
20
http://www.calvin.edu/~adams/research/microwulf/
95
TheamountofRAMmemoryperdollargrowsneatlyexponentially,withadecadetimeof4.8 years.
96
Disc drives
Figure34:Discstorageperdollarovertime. AsimilartrendtotheexponentialpriceperformancegrowthMooreslawincomputersis Kryderslawindiscdrives(Walter,2005).Theamountofstorageperdollarbegantogrow fasteraround1980(acurvewasfittedfor19802005),withadecadetimeofonly3.5years (Figure34).Relativelyfewdiskaccesstimesareavailable,butfittinganoverallexponential trendgivesadecadetimeof29.3years(Figure35). GiventhegrowingratiobetweenstoragecapacityandspeedofbothRAManddrives, updatingallofthestoreddatawilltakelongerandlonger:storagecapacityisoutrunningthe accessspeed.Thismayforcefinergranularityonbrainemulationssothattherearemore processorswiththeirownmemoryratherthanfewverypowerfulprocessorsbeinglimited bytheamountofdatatheycanrequest.
97
Figure35:Discaccesstimeovertime.
Future
Theissueisnotwhetherthereexistphysicalsystemsabletoperformthecomputationsdone bybrains,sincesuchsystemsalreadyexist:thebrainsthemselves.Rather,theissueiswhether brainemulatinghardwarecanbeconstructedbyhumaningenuityintheforeseeablefutureat asufficientlylowcosttomakeWBEfeasible. WhileMooreslawandotherexponentialprogresslawsincomputinghaveheldrelatively steadyoverdecades,theywillpresumablybreakatsomepoint.Attheverylimit,the quantizednatureoftheuniverseislikelytolimitthem.Beforethatpoint,limitationson molecularbuildingmaterials,lightspeed,andenergydissipationwillproveincreasingly problematic. Semiconductorsstillhavealongwaytogo(Meindl,Chenetal.,2001).Theinternational semiconductorroadmap(ITRS07,2007)appearsrelativelyconfidentwithinthe2016time horizon,andliststheresearchchallengesneededtocontinuethetrendtowards2022. However,thisisstillrelativelyclosecomparedtothetimescaleslikelyforachievingWBE. Worse,improvedindividualchipperformancemaynotnecessarilycarryoverdirectlyinto improvedsupercomputerperformance. (DeBenedictis,2004),astudyofthelimitsofcurrenttechnologytrendsfoundthatthemain probleminthenottoodistantfuture(beyond1020years)isgoingtobeheatdissipation,at leastformassivelyparallelcomputers21.Theexpectedperformanceinthisendgame
Theexceptionmaybearchitecturesdominatedbymemoryratherthanprocessing,whichcanremaindenseand cooltoagreaterextent.WBEapplications,however,arelikelytoneedtoupdatemostofthestatevariableseach
21
98
beyond2016wouldbeintheexaflopstozettaflopsrange(10181021).Whilecoolingcanbe mademoreeffectivebydispersingthecomputationunitsthisintroducescommunications delays.Reversiblelogicmayreduceheatdissipationbutthebestcurrentimplementations stilldissipatetwoordersofmagnitudemoreenergythanthetheoreticalminimum.Thestudy arguesthatacosteffectivenessof58TFLOPS/$andyear(givenrunningcostsandprorated purchaseprice)isachievable.Moreefficientcooling,orlessdissipativecomponentsare recognizedasimportantforzettaflopscomputing(DeBenedictis,Koggeetal.,2006). Exactlywhatcouldsupersedesemiconductorsremainsconjecturalbutunderintenseresearch (Welser,Bourianoffetal.,2008;Compao,Molenkampetal.,1999).Afewofthepossibilities thatarebeingpursuedorconsideredare: Ybranchswitchesarenanoelectroniccomponentsthatuseelectronsmovinginthe ballisticregimethatarenotstoppedbybarriers(ForsbergandHieke,2002).Such devicescouldbeusedtobuildreversiblelogicgates,reducingenergydissipation (Forsberg,2004). RapidSingleFluxQuantum(RSFQ)Logicusessuperconductingquantumeffectsto switch,usingthefluxquantumasabit(LikharevandSemenov,1991).Itcanswitch extremelyfast(hundredsofgigaherz),hasalowpowerconsumptionandexisting chiptechnologycanbeadaptedtomakeRSFQcircuitry(Zinoviev,1997).Simple microprocessorshavebeendemonstrated(Yoshikawa,Matsuzakietal.,2002). However,RSFQrequireslowtemperaturesforsuperconductivity. Opticalcomputing.Usingphotonsratherthanelectronswouldincreasetransmission speedsifsuitablenonlinearcomponentscanbefound.Thefieldhasbeenpursuedon andoffsincethe1960swithavarietyoftechnologies(SawchukandStrand,1984; Higgins,1995).Allopticalgatesareapproachingtheperformanceoftheirelectronic counterpartsbutstillneedfurtherdevelopment(Zhang,Wangetal.,2005).Akey problemisthatopticalcommunicationtendstorequiremorepowerovershort distancesthanelectronics,duetotheneedtoavoidshotnoise. Quantumdotcellularautomata(QCA).Electronconfigurationsinpatternsof quantumdotsactasacellularautomaton(TougawandLent,1994;Li,Wuetal.,2003; Robledo,Elzermanetal.,2008;Amlani,Orlovetal.,1999).SinceQCAdonottransmit currentsandperformreversiblecomputation(TimlerandLent,2003)heatdissipation islow,andmayprovideanidealenvironmentforzettaflopscomputing (DeBenedictis,2005).Whilecurrentdotsaresemiconductorssmallerandmore temperatureresistantsystemscouldbeconstructedoutofmolecularquantumdots (Lieberman,Chellammaetal.,2002).Arelatedtechnologyisspintronics,where magneticpolarisationisusedforcomputing(Allwood,Xiongetal.,2005;Imre,Csaba etal.,2006).Whilealsodissipatinglittleheat,speedlimitationsmaymakeit unsuitableforrapidprocessing. Helicallogic,reversiblelogicusingelectronsconstrainedtohelicalpathwaysshifted byanexternalfield(MerkleandDrexler,1996).Whilereversibleitrequireslow temperaturestofunctionandhasspeedlimitations. Molecularelectronics,wheretraditionalelectronicssuchasdiodesortransistorsare reproducedonthemolecularlevel(TsengandEllenbogen,2001).Logiccircuits (Bachtold,Hadleyetal.,2001)andnonvolatilememory(Rueckes,Kimetal.,2000) basedoncarbonnanotubetransistorshavebeendemonstrated. Intramolecularnanoelectronics,whereswitchingoccursonthemoleculelevel.This wouldinprinciplebeabletoreachtheultimatebitdensitiesof1012bits/cm2,
timestep,whichmeanstheyareunlikelytobestoragedominated.
99
Pessimisticclaimsareoftenmadetotheeffectthatlimitationsofcurrenttechnologyare foreverunsurpassable,orthattheoreticallypossibletechnologicalsystemssuchastheabove willbetoocostlyandpracticallydifficulttoeverbecomefeasible.Still,giventhatcomputer technologyhasdevelopedinarelativelystablemannerdespiteseveralchangesofbasic principles(e.g.fromflipflopsviacorememorytoseveralsemiconductorgenerations,from vacuumtubestotransistorstointegratedcircuitsetc)thereisnostrongreasontoassumethey willbreakbecausecurrenttechnologywilleventuallybereplacedbyothertechnologies. Largevestedinterestsincontinuingthegrowtharewillingtospendconsiderableresources onclosingtechnologygaps.Amorelikelyendofgrowthscenarioisthatthefeedback producingexponentialgrowthisweakenedbychangesinthemarketplacesuchaslowered demand,loweredexpectations,risingproductionfacilitycosts,longdevelopmenttimesor perhapsmoreefficientsoftware22. Iffurtherminiaturizationproveshardtoachieveforonereasonoranother,increased performancecanstillbeachievedbyusingparallelism.Ifmanyunitsaresold,priceswillgo downeveniftheprocessorsarenotmorepowerful,andaslowerexponentialgrowthofMIPS perdollarwillcontinue.Itshouldbenotedthat,sincethecommoditymarkethasdriven developmentofcomputersverystrongly,andatpresentonlyabout15%ofhumanityownsa computer,therearestilllargeuntappedmarketsthatwillovertimebecomerichenoughto affordcomputersandhencedrivefurthertechnologicaldevelopment.Asdemonstratedby variousingenioususesofGPUchipprogrammingforhighperformancescientific applications,itisentirelypossibletoprofitfromacompletelyunrelatedconsumerdemandif thesoftwareisadaptedtohighperformancehardwareevenwhenthishardwarewas developedforacompletelydifferentpurpose(inthiscasecomputergames).Themain concernforWBEisthatitisentirelypossibleforconsumercomputingtogoinadirectionless suitableforemulationsoftware(e.g.tradingprocessingpowerforlowerpowerusage).
100
AppendixC:Largescaleneuralnetwork simulations
Thesesimulationsrepresentasmallsamplingoftheliterature.Thereisabiastowards researchaimedathighperformancecomputationalneuroscienceanddevelopingnew methods. Table11:Largescaleneuralsimulations
Simulation Typeof simulation Neurons Synapses Hardware and Software Slowdown (time requiredfor 1biological second) 2 Timestep Notes
Integrate-andfire
12,500
1.56107
6-compartment neurons
2.2107
1.11010
Sun X4100 cluster, 2.4Ghz AMD Opteron, 8GB ram, MPI, NEST IBM Warson Research Blue Gene, SPLIT
0.1 ms
5,942
Supralinear scaling until 80 virtual processes for 105 neuron/109 synapses. Computing requirements: 11.5 TFLOPS. Spatial delays corresponding 16 cm2 cortex surface.
(Izhikevich, 2005)
Izhikevich neurons, Random synaptic connectivity generated on the fly. Compartment (Purkinje 4,500 comp./granule 1 comp.)
1011
1015
4.2106
16 Purkinje, 244,000 granule cells 60,000 granule cells, 300 mossy fibers, 300 Golgi cells, 300 stellate cells, 1 Purkinje cell 900 8106
4,500
79,200
(Brette, Rudolph et al., 2007; Traub, Contreras et al., 2005) (Traub, Contreras et al., 2005)
5 compartment model Low complexity spiking like Izhikevich neurons, STDP synapses 14 cell types, conductance and compartment model 14 cell types, conductance and compartment model, 11 active conductances
SPLIT 4096 processor BlueGene/L 256 Mb per CPU Cray XT3 2.4 Ghz, 800 CPUs 14 cpu Linux cluster, an IBM e1350, dualprocessor Intel P4 Xeon 10
100 s 1 ms
3,560
3,560
3,500 gap junctions, 1,122,520 synapses 3,500 gap junctions, 1,122,520 synapses
200?
Event driven 2 s
101
(Frye, 2004)
3 layer cortical column, 25% GABAergic, conductance models, synapse reversal pot., conductance, abs. str., mean prob. release, time const. recover depress facilitation, Four cell types. Km, Ka, Kahp channels
2.4-GHz, 512 Kbytes of L2 cache; or, e1350 Blade 1, at 2.8 GHz NCS, 30 dualPentium III 1GHz processor nodes, with 4 GB of RAM per node
165,000
3,000?
ANN feedforward network trained with iterative gradient descent Hopfield network Layered distribution of neural nets, detailed synaptic interconnections, spiking dynamics Point neurons, spikes, AHP, A & M potassium channels, synaptic adaptation, Spike shape and postsynaptic conductance specified by templates. Integrate and fire, event driven pulse connections with STDP plasticity. 1-8 dendritic segments. Palm binary network
1.73106
1,536 21010
2.4106 21013
Cluster 196 pentium III processors, 550 MHz, 384Mb Special purpose chips 500 nodes, 1,000 processors, 1 Tb RAM. Theoretical peak of 4,800 Gflops. Cluster, 128 Xeon 2.2 GHz processors with 256GB, 1Tb disk
256,000
106
256,000
Order of 3.21010
Special purpose chip, connected UltraSparc processor 500 Mhz, 640 Mb 512 processors, 192 GB, 250 Mhz mips 12K processorr
1012.21 timessteps/ ms
1.6106
2.01011
Execution time 4 min 7 sec, -13% MPI calls for retrieving 180K training vectors, ~O(10) iterations 9% real-time = 11.1
102
(Markram, 2006)
10,000
108
3.4GHz 8 Gb memory, peak performance 13.6 Gflop BlueGene/L, 4096 nodes, 8192 cpus, peak performances 22.4 TFLOPS IBM3090 computer Cray Y-MP 8/864 325 min/1.5 s bio time = 13,000 0.006 s (retrieval of 550,000 patterns; assuming human retrieval speed = 1 s) 0.05 ms -
(Traub and Wong, 1982) (Traub, Miles et al., 1992) (Moll and Miikkulainen, 1997)
100 19 compartment cells, ionic currents Binary units with binary weights 1,200 79,500 70,000 7.82108
Figure36:Numberofneuronsindifferentlargecomputationalneurosciencemodels.
103
Figure37:Numberofsynapsesindifferentcomputationalneurosciencemodels.
104
AppendixD:Historyandpreviouswork
TheearliestoriginsofthemindemulationideacanperhapsbetracedbacktoJ.D.BernalsThe World,TheFlesh,TheDevil(1929),wherehewrote: Menwillnotbecontenttomanufacturelife:theywillwanttoimproveonit.Forone materialoutofwhichnaturehasbeenforcedtomakelife,manwillhaveathousand; livingandorganizedmaterialwillbeasmuchatthecallofthemechanizedor compoundmanasmetalsaretoday,andgraduallythislivingmaterialwillcometo substitutemoreandmoreforsuchinferiorfunctionsofthebrainasmemory,reflex actions,etc.,inthecompoundmanhimself;forbodiesatthistimewouldbeleftfar behind.Thebrainitselfwouldbecomemoreandmoreseparatedintodifferent groupsofcellsorindividualcellswithcomplicatedconnections,andprobably occupyingconsiderablespace.Thiswouldmeanlossofmotilitywhichwouldnotbe adisadvantageowingtotheextensionofthesensefaculties.Everypartwouldnotbe accessibleforreplacingorrepairingandthiswouldinitselfensureapracticaleternity ofexistence,foreventhereplacementofapreviouslyorganicbraincellbyasynthetic apparatuswouldnotdestroythecontinuityofconsciousness. Finally,consciousnessitselfmayendorvanishinahumanitythathasbecome completelyetherealized,losingthecloseknitorganism,becomingmassesofatomsin spacecommunicatingbyradiation,andultimatelyperhapsresolvingitselfentirely intolight. Bernalsvisioncorrespondstoagradualreplacementofbiologywithartificialparts, graduallymakingitunnecessarytokeepthebraininonelocation. InthesciencefictionnovelCityandtheStars(1956)ArthurC.Clarkedescribedafarfuture citywherebodiesaremanufacturedbythecentralcomputer,mindsstoredinitsdatabanks downloadedintothem,andwhenaninhabitantdiestheirmindisstoredyetagaininthe computer,allowingcountlessreincarnations.Otherearlysciencefictiontreatmentswere RogerZelanzkysLordofLight(1968),BertilMrtenssonsDettarverkligheten(1968)and RudyRuckersSoftware(1979).Sincethen,mindemulation(uploading)hasbecomeastaple ofmuchsciencefiction23.Ofparticularnoteintermsoftechnologicalandphilosophical detailsarethenovelsandshortstoriesbyGregEgan(PermutationCity,Diaspora,Learningtobe Me,TransitionDreamsetc). Brain(andmind)emulationhasalsobeenwidelydiscussedinphilosophyofmind,although moreasGedankenexperimentethanpossibleactualpractice(e.g.(Parfit,1984;Chalmers,1995; Searle,1980)). Thefirstattemptatacarefulanalysisofbrainemulationwasatechnicalreport(Merkle, 1989b)predictingthatacompleteanalysisofthecellularconnectivityofastructureaslarge asthehumanbrainisonlyafewdecadesaway.Thereportreviewedautomatedanalysis andreconstructionmethods,goingintogreatdetailontherequirementsneededforparallel processingofbrainsamplesusingelectronmicroscopesandimageanalysissoftware.Italso clearlylistedassumptionsandrequirements,agoodexampleoffalsifiabledesign.
23
E.g.http://en.wikipedia.org/wiki/Mind_transfer_in_fiction
105
Thefirstpopularizationofatechnicaldescriptionofapossiblemindemulationscenariowas foundinHansMoravecsMindChildren(1990),wheretheauthordescribesthegradual neuronbyneuronreplacementofa(conscious)brainwithsoftware.Otherformsof emulationarealsodiscussed. (Hanson,1994)wasthefirstlookattheeconomicalimpactofcopyableminds,showingthat socialrolefitbrainemulationwouldlikelycausedramaticeconomicanddemographic changes. Onesketchofapersonemulationscenario(Leitl,1995)startsoutbythecryonicsuspensionof thebrain,whichisthendividedintocubicblocks<1mm.Theblockscanindividuallybe thawedforimmunostainingorothercontrastenhancement.Forscanning,variousmethods areproposed:Xrayfresnel/holographicdiffraction,Xrayorneutronbeamtomography(all riskingradiationdamage,mightrequirestrongstaining),transmissionEM(requiresverythin samples),UVabrasionofimmunostainedtissuewithmassspectrometry,orabrasiveatomic forcemicroscopescan.Whiledetailedintermsofthecryosuspensionmethods,thesketch becomeslessdetailedintermsofactualscanningmethodandimplementingtheemulation.
106
AppendixE:Nondestructiveandgradual replacement
Non-Destructive Scanning
Nondestructivescanningrequiresminimallyinvasivemethods.Thescanningneedsto acquiretherelevantinformationatthenecessary3Dresolution.Therearehoweverseveral seriouslimitations: Themovementofbiologicaltissue,requiringeitherimagingfasterthanitcanmove oraccuratetracking.Incats,thearterialpulseproduces110266mmovements lasting330400msandbreathinglarger(300950m)movements(BrittandRossi, 1982)24.Thestabilitytimeisasshortas520ms. Imaginghastooccuroveradistanceof>150mm(thewidthofanintactbrain)orbe invasive. Theimagingmustnotdepositenoughenergy(orusedyes,tracersorcontrast enhancers)tohurttheorganism. Themethodmustnotsignificantlyalterthementalorneuralstateofthesubjectbeing scannedinordertoavoidapossiblysignificantobservereffectanomaliesandfalse readingthatcouldproduceaflawedemulationmodel. OfthepossiblenoninvasivecandidatesonlyMRIappearsabletofulfilthelimitationsevenin principle.Opticimaging,evenusingfirstarrivinglightmethods,wouldnotworkacrosssuch alargedistance.Xraytomographyoftheintensityneededtoimagetissuewoulddeposit harmfulenergy(seediscussiononXraymicroscopy). TheresolutionofMRIdependsonthenumberofphasestepsused,gradientstrength, acquisitiontimeanddesiredsignaltonoiseratio.Torecordmicronscalefeaturesina movingbrain,veryshortacquisitiontimesareneeded,orawayofremovingthemovement artefacts.Eachdoublingofspatialresolutiondividesthesignaltonoiseratioby8,requiring longeracquisitiontimes,strongerfieldsormoresensitivedetectors.Finally,therearealso problemswithtissuewaterselfdiffusion,makingresolutionssmallerthan7.7mimpossible toachieve(GloverandMansfield,2002)25.Giventhatbrainemulationonthesynapticlevel requireshigherresolution,thisprobablyrulesoutMRIasanondestructivescanningmethod. However,ifthebrainisfrozen,waterdiffusionandmovementdonotoccurandverylong acquisitiontimescanbeused.OneproblemwithMRIoffrozenbraintissueisthatlimited protonmobilityreducesthesignal;thiscanbeamelioratedbykeepingthebrainat1oC (Longson,Hutchinsonetal.,1995;Doyle,Longsonetal.,1996).MRImightthereforebea possiblescanningmethodforfrozenorfixedbrains.Sinceitisnotdestructivetothetissueit mayalsoactasanadjuncttoother,destructive,scanningmethods.
24
Theauthorsofthepapersuggestsasasolutionusingacardiacbypasssystemtocreateanonpulsative flowofoxygenatedblood.
107
Apossiblewayaroundtheproblemsofnoninvasivescanningwouldbetouseendoscopic methodstobringmeasurementdevicesintothebloodvesselsofthebrainandimageitfrom inside.EndoscopicMRIhasbeendemonstratedinthegastrointestinalsystemwhereaRFcoil wasaffixedtothetipofanendoscope(Inui,Nakazawaetal.,1998).Thisislimitedbythe rangeofthedevices,theirsizeandtherisksofpenetratingvessels.However,itappears unlikelythatallpartsofthebrainarecloseenoughtoalargebloodvesseltoallowaccurate scanning,evenleavingouttheresolutionandmovementissues. Correlationmappingusingnanoprobes(Strout,2006)hasalsobeensuggested.Alarge number(1012)ofnanomachinessetupresidenceinoronneurons,recordingtheiractivityand functionalcorrelations.Amoredetailedanalysisofhownanomachinescouldsenseneural activitycanbefoundin(FreitasJr,1999,section4.8.6)togetherwithasketchofaninvivo fibrenetworkwith1018bit/scapacity(FreitasJr,1999,section7.3.1).Thiswouldenable essentiallyrealtimemonitoringofallneuronsandtheirchemicalenvironment(Kurzweil, 2005,pp.548549).Whetherjusttheactivitywouldbeenoughtomaptheconnectivityand memorystatesofthebrainisunclear.However,giventheassumedtechnologyboth intracellularprobingandphysicalconnectivitymappingbyhistonatingnanorobotsappears possible.Whetherthiswouldinterferewithbrainfunctionishardtoestimate(butsee(Freitas Jr.,2003)forananalysis).Thefibrenetworkwouldhaveavolumeof30cm3,whichis41%of thenormalbloodvolumeinthebrain(basedonestimatesofbloodvolumein(Leenders, Peranietal.,1990)),possiblyimpedingbloodflowifextendedinsidevessels.Althoughthe capabilitiesofnanomachinescanbeconstrainedbyknownphysicsitisnotpossibletodayto inferenoughaboutmachine/cell/tissueinteractionstoestimatewhethernondestructive scanningisfeasible. Overall,theprospectsfornondestructivescanningdonotlookgoodatpresent.Atthevery leastitwouldneedtoinvolveveryinvasiveendoscopicscanningortheuseofadvanced nanomedicine.
Gradual replacement
Scanningmightalsooccurintheformofgradualreplacement,aspieceafterpieceofthebrain isreplacedbyanartificialneuralsysteminterfacingwiththebrainandmaintainingthesame functionalinteractionsasthelostpieces.Eventuallyonlytheartificialsystemremains,and theinformationstoredcanbemovedifdesired(Morevec,1988).Whilegradualreplacement mightassuagefearsoflossofconsciousnessandidentity26itappearstechnicallyvery complexasthescanningsystemnotonlyhastoscanaliving,changingorganismbutalso interfaceseamlesslywithit(atleastonthesubmicronscale)whileworking.Thetechnology neededtoachieveitcoulddefinitelybeusedforscanningbydisassembly.Gradual replacementisthereforenotlikelyasafirstformofbrainemulationscanning(thoughin practiceitmayeventuallybecomethepreferredmethodifnondestructivescanningisnot possible). Itissometimessuggestedthatextendingthebrainthroughinterfaceswithexternalsoftware mightachieveaformoftransferwheremoreandmoreoftheentirepersonisstoredoutside thebrain,possiblyreachingthepointwherethebrainisnolongeressentialforthecomposite person.However,thiswouldnotbebrainemulationpersebutratheratransitiontoa
26Butnotnecessarily.Searlehasarguedthatreplacementwouldgraduallyremoveconsciousexperience(Searle, 1980).Parfitsphysicalspectrumthoughtexperimentinvolvesinterpolatingbetweentwodifferentpeoplethat clearlyhavedifferentidentities,andthereplacementprocesscouldhaveasimilarproperty(Parfit,1984).
108
109
AppendixF:Glossary
AFM ANN ATLUM Autoregulation Axon Blockface bouton CNS Confocalmicroscopy Connectome Dendrite Exaflop Extracellular FIBSEM FLOPS Fluorophore Atomicforcemicroscope(sometimescalledscanningforce microscope). ArtificialNeuralNetwork,amathematicalmodelbasedon biologicalneuralnetworks. AutomaticTapeCollectingLatheUltramicrotome Regulationofbloodflowtomaintainthe(cerebral)environment. Projectionfromanervcellthatconductssignalsawayfromthe neuronscellbody. Thesurfaceofanimagedsample,especiallywhencuttingtakes place. Thetypicalsynapticbumpthatgrowsbetweenaxonsand dendrites. CentralNervousSystem. Opticalimagingtechniquetoimage3Dsamplesbymakinguse ofaspatialpinhole. Thetotalsetofconnectionsbetweenregionsorneuronsina brain(Sporns,Tononietal.,2005). Branchedprojectionsofneuronsthatconductsignalsfromother neuronstothecellbody. 1018FLOPS. Theenvironmentoutsidethecells. FocusedIonBeamSEM. FloatingpointOperationsPerSecond.Ameasureofcomputing speed. Amoleculeorpartofmoleculethatcausesfluorescencewhen irradiatedbyUVlight.Usedforstainingmicroscope preparations. FieldProgrammableGateArray.Semiconductordevicethat containsprogrammablelogicandinterconnects,allowingthe systemdesignertosetupthechipfordifferentpurposes. RelatedtothetransmissionorreceptionofGABA,thechief inhibitoryneurotransmitter. Gigaflop,abillionFLOPS. Gliacellsarenonneuronalcellsthatprovidesupport,nutrition andmanyotherfunctionsinthenervoussystem. Agroupofcorticalminicolumnsorganisedintoamodulewitha fullsetofvaluesforagivensetofreceptivefieldparameters. IntheCNS,asmalllocallyprojectingneuron(unlikeneurons thatprojecttolongrangetargets). Anenzymethatphosphorylatesothermolecules. Amoleculethatbondstoanothermolecule,suchasareceptoror enzyme. Thecompletesetofsmallmoleculemetabolitesthatcanbefound inanorganism. MillionsofFloatingpointOperationsPerSecond.Ameasureof computingspeed. Acomponentofthecellskeleton,composedofsmallersubunits.
FPGA
GABAergic GFLOP Glia Hypercolumn Interneuron Kinase Ligand Metabolome MFLOPS Microtubule
110
Minicolumn
Neurite Neurogenesis Neuromodulator Neuromorphic Neuron Neuropeptide Neurotransmitter Parallelisation PCR Petaflop Phosphorylation
Skeletonization
Synapse Synapticspine
Averticalcolumnthroughthecerebralcortex;aphysiological minicolumnisacollectionofabout100interconnectedneurons,a functionalminicolumnconsistsofallneuronsthatsharethe samereceptivefield. MillionsofInstructionsPerSecond.Ameasureofcomputing speed. Aneuroninvolvedingeneratingmusclemovement. MagneticResonanceImaging. Thecerebralcortex,coveringthecerebralhemispheres. Neocortexdistinguishesitfromrelatedbutsomewhatmore primitivecortexthathavefewerthansixlayers. Aprojectionfromthecellbodyofaneuron,inparticularfroma developingneuronwhereitmaybecomeanaxonordendrite. Theprocessbywhichneuronsarecreatedfromprogenitorcells. Asubstancethataffectsthesignallingbehaviourofaneuron. Technologyaimingatmimickingneurobiologicalarchitectures. Anervecell. Aneurotransmitterthatconsistsofapeptide(aminoacidchain). Achemicalthatrelays,amplifiesormodulatessignalsfrom neuronstoatargetcell(suchasanotherneuron). Theuseofmultipleprocessorstoperformlargecomputations faster. PolymeraseChainReaction,atechniqueforamplifyingDNA fromasmallsample. 1015FLOPS. Additionofaphosphategrouptoaproteinmolecule(orother molecule),usuallydonebyakinase.Thiscanactivateor deactivatethemoleculeandplaysanimportantroleininternal cellsignalling. PeripheralNervousSystem. Theincreaseinsynapticresponsestrengthseenafterrepeated stimulation. SerialBlockFaceScanningElectronMicroscopy ScanningElectronMicroscopy. Sshaped,usuallydenotingamathematicalfunctionthatis monotonouslyincreasing,hastwohorizontalasymptotesand exactlyoneinflectionpoint. Imageprocessingmethodwhereashapeisreducedtothesetof pointsequidistantfromitsboundaries,representingits topologicalstructure. Thecellbodyofaneuron. Methodsofmakingspectrographicmeasuresofchemical compositioninmicroscopy. SerialSectionElectronTomography SerialSectionTransmissionElectronMicroscopy AsetofpropertiesAsupervenesonasetofpropertiesBifand onlyifanytwoobjectsxandythatsharealltheirBproperties mustalsosharealltheirAproperties.BeingBindiscernible impliesbeingAindiscernible. Ajunctionwhereone(ormore)neuronssignaltoeachother. Manysynapseshavetheirboutonsoffsetfromtheirparent
111
112
References
ABERDEEND,BAXTERJ,andEDWARDSR.92/mflops/s,ultralargescaleneuralnetwork trainingonapiiicluster.InSupercomputing,acm/ieee2000conference,2000,pp.4444. ACHARDPandDESCHUTTERE.Complexparameterlandscapeforacomplexneuronmodel. PlosComputationalBiology,2:794804,2006. ACKERMANNMandMATUSA.Activityinducedtargetingofprofilinandstabilizationof dendriticspinemorphology.NatureNeuroscience,6:11941200,2003. AJAYSMandBHALLAUS.Synapticplasticityinvitroandinsilico:Insightsintoan intracellularsignalingmaze.Physiology,21:289296,2006. ALKOFAHIKA,LASEKS,SZAROWSKIDH,etal.Rapidautomatedthreedimensionaltracingof neuronsfromconfocalimagestacks.IEEETransInfTechnolBiomed,6:17187,2002. ALLWOODDA,XIONGG,FAULKNERCC,etal.Magneticdomainwalllogic.Science,309:1688 1692,2005. AMLANII,ORLOVAO,TOTHG,etal.Digitallogicgateusingquantumdotcellularautomata. Science,284:289291,1999. ANDERSONFCandPANDYMG.Dynamicoptimizationofhumanwalking.JBiomechEng,123: 38190,2001. ARNOLDAS,SALINASS,ASAKAWADJ,andDELPSL.Accuracyofmusclemomentarms estimatedfrommribasedmusculoskeletalmodelsofthelowerextremity.Comput AidedSurg,5:10819,2000. ARRADIANCEINC.Massivelyparallelebeamsourcesforsemiconductorlithographyandmetrology. 2007http://arradiance.com/. ASCOLIGA.Progressandperspectivesincomputationalneuroanatomy.AnatomicalRecord, 257:195207,1999. ATWOODHLandWOJTOWICZJM.Silentsynapsesinneuralplasticity:Currentevidence. Learning&Memory,6:542571,1999. BACHTOLDA,HADLEYP,NAKANISHIT,andDEKKERC.Logiccircuitswithcarbonnanotube transistors.Science,294:13171320,2001. BAIRDRC,JOHARIH,andGY.J.Numericalsimulationofenvironmentmodulationofchemical signalstructureandodordispersalintheopenocean.ChemSenses.,21:121134,1996 BARZELR,HUGHESJF,andWOODDN.Plausiblemotionsimulationforcomputergraphics animation.InBoulicR.andHgronG.(Eds.),Computeranimationandsimulation96: Springer,1996,pp.183197. BEECHERCWW.Thehumanmetabolome.InHarriganG.G.(Ed.)Metabolicprofiling:Itsrolein biomarkerdiscoveryandgenefunctionanalysis:SpringerVerlag,2003. BEKENSTEINJD.Universalupperboundontheentropytoenergyratioforboundedsystems. PhysicalReviewD,23:287298,1981. BENHAMMP,WRIGHTDK,andBIBBR.Modellingsofttissueforkinematicanalysisofmulti segmenthumanbodymodels.BiomedSciInstrum,37:1116,2001. BERGRW,ALABURDAA,andHOUNSGAARDJ.Balancedinhibitionandexcitationdrivespike activityinspinalhalfcenters.Science,315:390393,2007. BHALLAUS.Signalinginsmallsubcellularvolumes.I.Stochasticanddiffusioneffectson individualpathways.BiophysicalJournal,87:733744,2004a. BHALLAUS.Signalinginsmallsubcellularvolumes.Ii.Stochasticanddiffusioneffectson synapticnetworkproperties.BiophysicalJournal,87:745753,2004b. BHALLAUSandIYENGARR.Emergentpropertiesofnetworksofbiologicalsignaling pathways.Science,283:381387,1999.
113
BHARDWAJRD,CURTISMA,SPALDINGKL,etal.Neocorticalneurogenesisinhumansis restrictedtodevelopment.ProceedingsoftheNationalAcademyofSciencesoftheUnited StatesofAmerica,103:1256412568,2006. BIGQandPOOMM.Synapticmodificationsinculturedhippocampalneurons:Dependence onspiketiming,synapticstrength,andpostsynapticcelltype.JournalofNeuroscience, 18:1046410472,1998. BIENENSTOCKEL,COOPERLN,andMUNROPW.Theoryforthedevelopmentofneuron selectivityorientationspecificityandbinocularinteractioninvisualcortex.Journalof Neuroscience,2:3248,1982. BINZEGGERT,DOUGLASRJ,andMARTINKAC.Aquantitativemapofthecircuitofcatprimary visualcortex.JournalofNeuroscience,24:84418453,2004. BIOMECHANICSRESEARCHGROUPINC.Brg.Lifemod2005:Biomechanicsmodelingpackage.2005 http://www.lifemodeler.com/. BLATOWM,CAPUTIA,andMONYERH.Moleculardiversityofneocorticalgabaergic interneurones.JournalofPhysiologyLondon,562:99105,2005. BLEMKERSS,ASAKAWADS,GOLDGE,andDELPSL.Imagebasedmusculoskeletalmodeling: Applications,advances,andfutureopportunities.JMagnResonImaging,25:44151, 2007. BLEMKERSSandDELPSL.Threedimensionalrepresentationofcomplexmusclearchitectures andgeometries.AnnBiomedEng,33:66173,2005. BOKILH,LAARISN,BLINDERK,ENNISM,andKELLERA.Ephapticinteractionsinthe mammalianolfactorysystem.JournalofNeuroscience,21:,2001. BOSTROMN.Howlongbeforesuperintelligence?Int.Jour.ofFutureStudies,2,1998. BOSTROMN.Areyoulivinginacomputersimulation?PhilosophicalQuarterly,53:243255, 2003. BOWERJMandBEEMAND.Thebookofgenesis:Exploringrealisticneuralmodelswiththegeneral neuralsimulationsystem.NewYork:SpringerVerlag,1998. BRAITENBERGVandSCHUZA.Cortex:Statisticsandgeometryofneuronalconnectivity.NewYork: SpringerVerlag.,1998. BRETTER,RUDOLPHM,CARNEVALET,etal.Simulationofnetworksofspikingneurons:A reviewoftoolsandstrategies.JournalofComputationalNeuroscience,23:349398,2007. BRIGGMANKLandDENKW.Towardsneuralcircuitreconstructionwithvolumeelectron microscopytechniques.CurrentOpinioninNeurobiology,16:562570,2006. BRITTRHandROSSIGT.Quantitativeanalysisofmethodsforreducingphysiologicalbrain pulsations.JournalofNeuroscienceMethods,6:219229,1982. BROWNIE,SCOTTSH,andLOEBGE.Mechanicsoffelinesoleus:Ii.Designandvalidationofa mathematicalmodel.JMuscleResCellMotil,17:22133,1996. BRUNELN,HAKIMV,ISOPEP,NADALJP,andBARBOURB.Optimalinformationstorageandthe distributionofsynapticweights:Perceptronversuspurkinjecell.Neuron,43:745757, 2004. BULLOCKTH.Revisitingtheconceptofidentifiableneurons.BrainBehaviorandEvolution,55: 236240,2000. CAILL,COURTINEG,FONGAJ,etal.Plasticityoffunctionalconnectivityintheadultspinal cord.PhilosophicalTransactionsoftheRoyalSocietyBBiologicalSciences,361:16351646, 2006. CAIANIELLOER.Outlineofatheoryofthoughtprocessandthinkingmachines.J.Theoret.Biol., 1:204235,1961. CARNEVALENTandHINESML.Theneuronbook.Cambridge,UK:CambridgeUniversityPress, 2006.
114
CASHAD,ALIEVG,SIEDLAKSL,etal.Microtubulereductioninalzheimersdiseaseandaging isindependentoftaufilamentformation.AmericanJournalofPathology,162:1623 1627,2003. CCORTEX.Artificialdevelopmenttobuildworldsbiggestspikingneuralnetwork.2003 http://www.corticaldb.com/ccortex.asp. CHALMERSDJ.Absentqualia,fadingqualia,dancingqualia.InMetzingerT.(Ed.)Conscious experience:ImprintAcademic,1995. CHANGYMandLUEBKEJI.Electrophysiologicaldiversityoflayer5pyramidalcellsinthe prefrontalcortexoftherhesusmonkey:Invitroslicestudies.Journalof Neurophysiology,98:26222632,2007. CHAOSH,DOUGHERTYWM,GARBINIJL,andSIDLESJA.Nanometerscalemagneticresonance imaging.ReviewofScientificInstruments,75:11751181,2004. CHENDTandZELTZERD.Pumpitup:Computeranimationofabiomechanicallybasedmodel ofmuscleusingthefiniteelementmethod.InThomasJ.J.(Ed.)Proceedingsofthe19th annualconferenceoncomputergraphicsandinteractivetechniques,1992,pp.8998. CHENLY,REXCS,CASALEMS,GALLCM,andLYNCHG.Changesinsynapticmorphology accompanyactinsignalingduringltp.JournalofNeuroscience,27:53635372,2007. CHERNIAKC.Theboundedbrain:Towardquantitativeneuroanatomy.JournalofCognitive Neuroscience,2:5868,1990. CHURCHLANDPandSEJNOWSKITJ.Thecomputationalbrain:MITPress,1992. CIOBANUL,SEEBERDA,andPENNINGTONCH.3dmrmicroscopywithresolution3.7mumby 3.3mumby3.3mum.JournalofMagneticResonance,158:178182,2002. CIVELLIO.Gpcrdeorphanizations:Thenovel,theknownandtheunexpectedtransmitters. TrendsinPharmacologicalSciences,26:1519,2005. COLOMBELLIJ,GRILLSW,andSTELZEREHK.Ultravioletdiffractionlimitednanosurgeryof livebiologicaltissues.ReviewofScientificInstruments,75:472478,2004. COLONNIERM.Theelectronmicroscopicanalysisoftheneuronalorganizationofthecerebral cortex.InSchmittF.,WordenF.,AdelmanG.,andDennisS.(Eds.),Theorganizationof thecerebralcortex.Cambridge:MITPress,1981,pp.125152. COMPAOR,MOLENKAMPL,andPAULDJ.Technologyroadmapfornanoelectronics. MicroelectronicsAdvancedResearchInitiative,1999 COTINS,DELINGETTEH,andAYACHEN.Realtimeelasticdeformationsofsofttissuesfor surgerysimulation.IeeeTransactionsonVisualizationandComputerGraphics,5:6273, 1999. CRAGGSJ,NICHOLSONC,KUMEKICKJ,TAOL,andRICEME.Dopaminemediatedvolume transmissioninmidbrainisregulatedbydistinctextracellulargeometryanduptake. JournalofNeurophysiology,85:17611771,2001. CROSSINKLandKRUSHELLA.Cellularsignalingbyneuralcelladhesionmoleculesofthe immunoglobulinsuperfamily.DevelopmentalDynamics,218:260279,2000. CROWNED,FERGUSONAR,JOYNESRL,andGRAUJW.Instrumentallearningwithinthespinal cordii.Evidenceforcentralmediation.Physiology&Behavior,77:259267,2002. DAISYPROJECT.Neocorticaldaisyarchitecturesandgraphicalmodelsforcontextdependent processing.2008http://daisy.ini.unizh.ch/. DEBENEDICTISE,KOGGEP,LENTC,NIEMIERM,andSTERLINGT.Thetechnologylaneontheroad toazettaflops,tr200615.UniversityofNotreDameComputerScienceand EngineeringFaculty,2006www.cse.nd.edu/Reports/2006/TR200615.pdf. DEBENEDICTISEP.Takingascisupercomputingtotheendgame,sand20040959.SandiaNational Laboratories,2004 DEBENEDICTISEP.Reversiblelogicforsupercomputing.InProceedingsofthe2ndconferenceon computingfrontiers,2005,pp.391402.
115
DENKWandHORSTMANNH.Serialblockfacescanningelectronmicroscopytoreconstruct threedimensionaltissuenanostructure.PlosBiology,2:19001909,2004. DIVENTURAB,LEMERLEC,MICHALODIMITRAKISK,andSERRANOL.Frominvivotoinsilico biologyandback.Nature,443:527533,2006. DILLONCandGODAY.Theactincytoskeleton:Integratingformandfunctionatthesynapse. AnnualReviewofNeuroscience,28:2555,2005. DIMAA,SCHOLZM,andOBERMAYERK.Automaticsegmentationandskeletonizationof neuronsfromconfocalmicroscopyimagesbasedonthe3dwavelettransform.Image Processing,IEEETransactionson,11:790801,2002. DIXA.Thebrainandtheweb:Aquickbackupincaseofaccidents.Interfaces,65:67,2005. DJURFELDTM,JOHANSSONC,EKEBERG,etal.Massivelyparallelsimulationofbrainscale neuronalnetworkmodels.Stockholm:CSC,KTH,2005http://www 03.ibm.com/servers/deepcomputing/pdf/Blue_Gene_Applications_Paper_KTH_0306. pdf. DJURFELDTM,LUNDQVISTM,JOHANSSONC,etal.Projectreportforbluegenewatsonconsortium days:Massivelyparallelsimulationofbrainscaleneuronalnetworkmodels.2006 http://wwwfp.mcs.anl.gov/bgconsortium/Past%20Results/kth%20bgwreport06.pdf. DOAKRB,GRISENTIRE,REHBEINS,etal.Towardsrealizationofanatomicdebroglie microscope:Heliumatomfocusingusingfresnelzoneplates.PhysicalReviewLetters, 83:42294232,1999. DONIZELLIM,DJITEMA,andLENOVEREN.Lgicdb:Amanuallycuratedsequencedatabase afterthegenomes.NucleicAcidsResearch,34:D267D269,2006. DOUGLASSJKandWILKENSLA.Directionalselectivitiesofnearfieldfiliformhair mechanoreceptorsonthecrayfishtailfan(crustacea:Decapoda).Journalof ComparativePhysiologyaNeuroethologySensoryNeuralandBehavioralPhysiology,183: 2334,1998. DOYLECA,LONGSOND,HUTCHINSONC,andSIMPSONMDC.Mrioffrozenbraintissue:A freshlookatmorphometryinschizophrenia.SchizophreniaResearch,18:Xg1Xg1, 1996. DREXLERKE.Enginesofcreation:Thecomingeraofnanotechnology:AnchorBooks,1986. DREXLERKE.Nanosystems:Molecularmachinery,manufacturingandcomputation:Wiley1992. DRUCKMANNS,BANITTY,GIDONAA,etal.Anovelmultipleobjectiveoptimization frameworkforconstrainingconductancebasedneuronmodelsbyexperimentaldata. Frontiersinneuroscience,1:718,2007. EKEBERG,BLMELM,andBSCHGESA.Dynamicsimulationofinsectwalking.Arthropod Structure&Development,33:287300,2004. EKEBERGOandGRILLNERS.Simulationsofneuromuscularcontrolinlampreyswimming. PhilosTransRSocLondBBiolSci,354:895902,1999. EKEBERGOandPEARSONK.Computersimulationofsteppinginthehindlegsofthecat:An examinationofmechanismsregulatingthestancetoswingtransition.JNeurophysiol, 94:425668,2005. ELIASMITHC.Attractiveandindiscreteacritiqueoftwoputativevirtuesofthedynamicist theoryofmind.MindsandMachines,11:417426,2001. ELIXSONE.Hypothermia.Coldwaterdrowning.CritCareNursClinNorthAm.,3:287292, 1991. EMBLEBI.Ligandgatedionchanneldatabase.2008http://www.ebi.ac.uk/compneur srv/LGICdb/LGICdb.php. FEITELSONDG.Thesupercomputerindustryinlightofthetop500data.ComputinginScience andEngineering,7:4247,2005. FELLINTandCARMIGNOTOG.Neuronetoastrocytesignallinginthebrainrepresentsa distinctmultifunctionalunit.JournalofPhysiologyLondon,559:315,2004.
116
FENSTERMACHERJandKAYET.Drugdiffusionwithinthebrain.AnnalsoftheNewYork AcademyofSciences,531:2939,1988. FERNANDEZJWandPANDYMG.Integratingmodellingandexperimentstoassessdynamic musculoskeletalfunctioninhumans.ExpPhysiol,91:37182,2006. FIALAJC.Threedimensionalstructureofsynapsesinthebrainandontheweb.InIjcnn02. Proceedingsofthe2002internationaljointconferenceonneuralnetworks:IEEEPress,2002, pp.14. FIALAJC.Synapses.Bu.Edu.2007,2007http://synapses.bu.edu/. FIALAJCandHARRISKM.Extendingunbiasedstereologyofbrainultrastructuretothree dimensionalvolumes.JournaloftheAmericanMedicalInformaticsAssociation,8:116, 2001. FIELDSRDandELLISMANMH.Synapticmorphologyanddifferencesinsensitivity.Science, 228:197199,1985. FORESIGHTNANOTECHINSTITUTEandBATELLEMEMORIALINSTITUTE.Productivenanosystems:A technologyroadmap.2007http://www.foresight.org/roadmaps/. FORSBERGE.Reversiblelogicbasedonelectronwaveguideybranchswitches.Nanotechnology, 15:S298S302,2004. FORSBERGEandHIEKEK.Electronwaveguideybranchswitchescontrolledbypt/gaas schottkygates.PhysicaScripta,T101:158160,2002. FORTUNEEandROSEG.Shorttermsynapticplasticityasatemporalfilter.TrendsNeuroscience, 24:381385,2001 FRANKW.Electrontomography:Threedimensionalimagingwiththetransmissionelectron microscope.NewYork:Plenum,1992. FREITASJRRA.Nanomedicine,volumei:Basiccapabilities.Georgetown,TX:LandesBioscience, 1999. FREITASJR.RA.Thefutureofcomputers.Analog,116:5773,1996. FREITASJR.RA.Nanomedicine,volumeiia:Biocompatibility.Georgetown,TX:LandesBioscience, 2003. FRYEJ.Bluegenetestreport.2004 http://brain.unr.edu/publications/NCS_BlugGene_report_07Nov04.pdf. FRYEJ,ANANTHANARAYANANR,andMODHADS.Towardsrealtime,mousescalecortical simulations.SaltLakeCity,Utah,Feb22252007,2007 http://www.modha.org/papers/rj10404.pdf. FUJISAKIH,TAKAHASHIS,OHZEKIH,etal.Softxraydamagetobiologicalsamples.Journalof MicroscopyOxford,182:7983,1996. FUNKHOUSERT,TSINGOSN,CARLBOMI,etal.Abeamtracingmethodforinteractive architecturalacoustics.JournaloftheAcousticalSocietyofAmerica,115:739756,2004. GAMMAITONIL,HANGGIP,JUNGP,andMARCHESONIF.Stochasticresonance.Reviewsof ModernPhysics,70:223287,1998. GARDNERMEDWINAR.Analysisofpotassiumdynamicsinmammalianbraintissue.Journalof PhysiologyLondon,335:393426,1983. GARRIGAA,SPAC,andLOPEZV.Computationofthecompleteacousticfieldwithfinitedifferences algorithms.Budapest,2005 http://www.tecn.upf.es/~agarriga/ARCHIVOS_ADAN/fa_2005_paper.pdf. GELPERINDandHETZELB.Thegrowthofsoftwaretesting.CommunicationsoftheAcm,31:687 695,1988. GEMMELLJ,BELLG,andLUEDERR.Mylifebits:Apersonaldatabaseforeverything. CommunicationsoftheAcm,49:8895,2006. GLOVERPandMANSFIELDP.Limitstomagneticresonancemicroscopy.ReportsonProgressin Physics,65:14891511,2002.
117
GOODMANPH,COURTENAYWILSONE,MACIOKASJB,etal.Largescaleparallelsimulationof physiologicallyrealisticmulticolumnsensorycortex.2001 http://brain.unr.edu/publications/gwm.largescalecortex_01.pdf. GRAYCM,KONIGP,ENGELAK,andSINGERW.Oscillatoryresponsesincatvisualcortex exhibitintercolumnarsynchronizationwhichreflectsglobalstimulusproperties. Nature,338:334337,1989. GRIMARandSCHNELLS.Asystematicinvestigationoftheratelawsvalidinintracellular environments.BiophysicalChemistry,124:110,2006. GRUTZENDLERJ,KASTHURIN,andGANWB.Longtermdendriticspinestabilityintheadult cortex.Nature,420:812816,2002. GUPTAA,WANGY,andMARKRAMH.Organizingprinciplesforadiversityofgabaergic interneuronsandsynapsesintheneocortex.Science,287:273278,2000. GUSTAFSSONMGL.Nonlinearstructuredilluminationmicroscopy:Widefieldfluorescence imagingwiththeoreticallyunlimitedresolution.ProceedingsoftheNationalAcademyof SciencesoftheUnitedStatesofAmerica,102:1308113086,2005. GUTMANGA,CHANDYKG,GRISSMERS,etal.Internationalunionofpharmacology.Liii. Nomenclatureandmolecularrelationshipsofvoltagegatedpotassiumchannels. PharmacologicalReviews,57:473508,2005. HAGANS,HAMEROFFSR,andTUSZYNSKIJA.Quantumcomputationinbrainmicrotubules: Decoherenceandbiologicalfeasibility.PhysicalReviewE,65:,2002. HAMEROFFSR.Ultimatecomputing:Biomolecularconsciousnessandnanotechnology:Elsevier SciencePublishers,1987. HAMMARLUNDPandEKEBERGO.Largeneuralnetworksimulationsonmultiplehardware platforms.JournalofComputationalNeuroscience,5:443459,1998. HANLONEB,MANOHARANR,KOOTW,etal.Prospectsforinvivoramanspectroscopy. PhysicsinMedicineandBiology,45:R1R59,2000. HANSONR.Ifuploadscomefirst:Thecrackofafuturedawn.Extropy,6,1994. HANSONR.Thenextreallybigenormousthing.FutureBrief,October6,2004. HANSONR.Economicsofbrainemulations.InHealeyP.(Ed.)Tomorrowspeopleproceedings ofthejamesmartininstitutesfirstworldforum:EarthScan,2008a. HANSONR.Economicsofthesingularity.IEEESpectrum:3742,2008b. HARDMANJG,BEDFORTHNM,AHMEDAB,MAHAJANRP,andAITKENHEADAR.Aphysiology simulator:Validationofitsrespiratorycomponentsanditsabilitytopredictthe patientsresponsetochangesinmechanicalventilation.BritishJournalofAnaesthesia, 81:327332,1998. HARRISFC,BAURICKJ,FRYEJ,etal.Anovelparallelhardwareandsoftwaresolutionforalargescale biologicallyrealisticcorticalsimulation.GoodmanBrainComputationLab,Universityof Nevada.,2002 HARRISKandSTEVENSJ.Dendriticspinesofca1pyramidalcellsintherathippocampus: Serialelectronmicroscopywithreferencetotheirbiophysicalcharacteristics. J.Neurosci,9:29822997,1989. HARRISONJ,RENSINKRA,andVANDEPANNEM.Obscuringlengthchangesduringanimated motion.AcmTransactionsonGraphics,23:569573,2004. HARVEYCDandSVOBODAK.Locallydynamicsynapticlearningrulesinpyramidalneuron dendrites.Nature,450:1195U3,2007. HAYMANS.Themccullochpittsmodel.NeuralNetworks,6:44384439,1999. HAYWORTHKJ.Singlebrainphysicalslicelibraryproposal:Creatingacompletehumanneural connectivitydatabaseviasparse,automaticallydirectedultramicroscopicimagingusingan automatedretrievalbrainslicestoragesystem.2002 http://www.extremeneuroanatomy.com/PhysicalSliceLibraryProposal.pdf.
118
HAYWORTHKJ.Atlumprojecthomepage.2007,2007 http://www.mcb.harvard.edu/lichtman/ATLUM/ATLUM_web.htm. HAYWORTHKJ,KASHTURIN,SCHALEKR,andLICHTMANJW.Automatingthecollectionof ultrathinserialsectionsforlargevolumetemreconstructions.Chicago,2006 HERZAVM,GOLLISCHT,MACHENSCK,andJAEGERD.Modelingsingleneurondynamicsand computations:Abalanceofdetailandabstraction.Science,314:8085,2006. HIGGINSTV.Computingwithphotonsatthespeedoflight.LaserFocusWorld,31:7277,1995. HILLERBRANDR.Scaleseparationasaconditionforquantitativemodelling.Whymathematics worksforsomeproblemsandfailsforothers.Synthese,2008. HINESM,MARKRAMH,andSCHUERMANNF.Fullyimplicitparallelsimulationofsingle neurons.JCompNeurosci,2008. HODGKINAandHUXLEYA.Aquantitativedescriptionofmembranecurrentandits applicationtoconductionandexcitationinnerve.JPhysiol.,117:500544,1952. HOKFELTT,BROBERGERC,XUZQD,etal.Neuropeptidesanoverview.Neuropharmacology, 39:13371356,2000. HOLSTBandALLISONW.Anatomfocusingmirror.Nature,390:244244,1997. HOLTMAATAJGD,TRACHTENBERGJT,WILBRECHTL,etal.Transientandpersistentdendritic spinesintheneocortexinvivo.Neuron,45:279291,2005. HOUWELINGARandBRECHTM.Behaviouralreportofsingleneuronstimulationin somatosensorycortex.Nature,451:65U8,2008. HOWELLFW,DYHRFJELDJOHNSENJ,MAEXR,GODDARDN,andDESCHUTTERE.Alargescale modelofthecerebellarcortexusingpgenesis.Neurocomputing,32:10411046,2000. HULLC,STUDHOLMEK,YAZULLAS,andVONGERSDORFFH.Diurnalchangesinexocytosisand thenumberofsynapticribbonsatactivezonesofanontypebipolarcellterminal. JournalofNeurophysiology,96:20252033,2006. HUMPHERYSMITHI.Ahumanproteomeprojectwithabeginningandanend.Proteomics,4: 25192521,2004. IMREA,CSABAG,JIL,etal.Majoritylogicgateformagneticquantumdotcellularautomata. Science,311:205208,2006. INUIK,NAKAZAWAS,YOSHINOJ,andUKAIH.Endoscopicmri.Pancreas,16:413417,1998. ITOAandSHINOHARAK.Imageblurringbythermaldiffusionintheobservationofhydrated biomoleculeswithsoftxraymicroscopy.CellStructureandFunction,17:209212, 1992. ITRS07.Internationaltechnologyroadmapforsemiconductors, http://www.Itrs.Net/links/2007itrs/home2007.Htm.2007 IVANCEVICVandBEAGLEYN.Determiningtheacceptablelimitsofheadmountedloads.Land OperationsDivision,SystemsSciencesLaboratory,AustralianDefenceScienceand TechnologyOrganisation,2004 IZHIKEVICHEM.Simplemodelofspikingneurons.IeeeTransactionsonNeuralNetworks,14: 15691572,2003. IZHIKEVICHEM.Whichmodeltouseforcorticalspikingneurons?IeeeTransactionsonNeural Networks,15:10631070,2004. IZHIKEVICHEM.Simulationoflargescalebrainmodels.2007,2005 http://vesicle.nsi.edu/users/izhikevich/human_brain_simulation/Blue_Brain.htm#Sim ulation%20of%20LargeScale%20Brain%20Models. IZHIKEVICHEM.Dynamicalsystemsinneuroscience:Thegeometryofexcitabilityandbursting. Cambridge,Massachusetts:MITPress,2007. JACOBSENC.Softxraymicroscopy.TrendsinCellBiology,9:4447,1999. JELINEKHFandFERNANDEZE.Neuronsandfractals:Howreliableandusefularecalculations offractaldimensions?JournalofNeuroscienceMethods,81:918,1998.
119
JOHANSSONCandLANSNERA.Towardscortexsizedartificialneuralsystems.Neural Networks,20:4861,2007. KALISMANN,SILBERBERGG,andMARKRAMH.Derivingphysicalconnectivityfromneuronal morphology.BiologicalCybernetics,88:210218,2003. KALISMANN,SILBERBERGG,andMARKRAMH.Theneocorticalmicrocircuitasatabularasa. ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,102:880 885,2005. KASPEREM,LARKMANAU,LUBKEJ,andBLAKEMOREC.Pyramidalneuronsinlayer5ofthe ratvisualcortex.1.Correlationamongcellmorphology,intrinsicelectrophysiological properties,andaxontargets.JournalofComparativeNeurology,339:459474,1994. KAUFMANA,DRORG,MEILIJSONI,andRUPPINE.Geneexpressionofcaenorhabditiselegans neuronscarriesinformationontheirsynapticconnectivity.PlosComputationalBiology, 2:15611567,2006. KERENN,PELEDN,andKORNGREENA.Constrainingcompartmentalmodelsusingmultiple voltagerecordingsandgeneticalgorithms.JournalofNeurophysiology,94:37303742, 2005. KEYHANIK,SCHERERPW,andMOZELLMM.Anumericalmodelofnasalodoranttransportfor theanalysisofhumanolfaction.JournalofTheoreticalBiology,186:279301,1997. KIKUCHIS,FUJIMOTOK,KITAGAWAN,etal.Kineticsimulationofsignaltransductionsystem inhippocampallongtermpotentiationwithdynamicmodelingofprotein phosphatase2a.NeuralNetworks,16:13891398,2003. KIRBASCandQUEKF.Areviewofvesselextractiontechniquesandalgorithms.Acm ComputingSurveys,36:81121,2004. KLATZKYRL,LEDERMANSJ,HAMILTONC,andGRINDLEYM.Feelingtexturesthroughaprobe: Effectsofprobeandsurfacegeometryandexploratoryfactors.Perception& Psychophysics,65:613631,2003. KOCHCandHEPPK.Quantummechanicsinthebrain.Nature,440:611612,2006. KOENERA.Netmorph.2008http://netmorph.org/. KOHWandMCCORMICKBH.Specificationsforvolumedataacquisitioninthreedimensionallight microscopy,technicalreporttr200375.CollegeStation,TX:DepartmentofComputer Science,TexasA&MUniversity,2003 KONDOY,KOSHIBAY,ARIMAY,etal.A1.2gflopsneuralnetworkchipforhighspeedneural networkservers.IeeeJournalofSolidStateCircuits,31:860864,1996. KOUZNETSOVD,OBERSTH,NEUMANNA,etal.Ridgedatomicmirrorsandatomicnanoscope. JournalofPhysicsBAtomicMolecularandOpticalPhysics,39:16051623,2006. KOZLOVAK,LANSNERA,GRILLNERS,andKOTALESKIJH.Ahemicordlocomotornetworkof excitatoryinterneurons:Asimulationstudy.BiologicalCybernetics,96:229243,2007. KOZLOVAS,ANGULOMC,AUDINATE,andCHARPAKS.Targetcellspecificmodulationof neuronalactivitybyastrocytes.ProceedingsoftheNationalAcademyofSciencesofthe UnitedStatesofAmerica,103:1005810063,2006. KRAFFTC.Bioanalyticalapplicationsoframanspectroscopy.AnalyticalandBioanalytical Chemistry,378:6062,2004. KRAFFTC,KNETSCHKET,SIEGNERA,FUNKRHW,andSALZERR.Mappingofsinglecellsby nearinfraredramanmicrospectroscopy.VibrationalSpectroscopy,32:7583,2003. KRIMERLS,ZAITSEVAV,CZANNERG,etal.Clusteranalysisbasedphysiologicalclassification andmorphologicalpropertiesofinhibitoryneuronsinlayers23ofmonkey dorsolateralprefrontalcortex.JournalofNeurophysiology,94:30093022,2005. KRNJEVICK.Ephapticinteractions:Asignificantmodeofcommunicationsinthebrain.News inPhysiologicalSciences,1:2829,1986.
120
KRUITP.Highthroughputelectronlithographywiththemultipleaperturepixelbypixel enhancementofresolutionconcept.JournalofVacuumScience&TechnologyB,16:3177 3180,1998. KULLMANNDM.Silentsynapses:Whataretheytellingusaboutlongtermpotentiation? PhilosophicalTransactionsoftheRoyalSocietyofLondonSeriesBBiologicalSciences,358: 727733,2003. KURZWEILR.Theageofspiritualmachines.NewYork:Viking,1999. KURZWEILR.Thesingularityisnear:Whenhumanstranscendbiology:VikingPress,2005. KUZIRIANAMandLEIGHTONSB.Oxygenplasmaetchingofentireblockfacesimprovesthe resolutionandusefulnessofserialscanningelectronmicroscopicimages.Scanning ElectronMicroscopy:18771885,1983. KWONJ,MAYERICHD,CHOEY,andMCCORMICKBH.Automatedlateralsectioningforknife edgescanningmicroscopy.InIeeeinternationalsymposiumonbiomedicalimaging:From nanotomacro,2008. LANDAUERTK.Howmuchdopeopleremembersomeestimatesofthequantityoflearned informationinlongtermmemory.CognitiveScience,10:477493,1986. LEDOA,FRADEJ,BARBOSARM,andLARANJINHAJ.Nitricoxideinbrain:Diffusion,targets andconcentrationdynamicsinhippocampalsubregions.MolecularAspectsof Medicine,25:7589,2004. LEEWCA,HUANGH,FENGGP,etal.Dynamicremodelingofdendriticarborsingabaergic interneuronsofadultvisualcortex.PlosBiology,4:271280,2006. LEENDERSKL,PERANID,LAMMERTSMAAA,etal.Cerebralbloodflow,bloodvolumeand oxygenutilizationnormalvaluesandeffectofage.Brain,113:2747,1990. LEIGHTONSB.Semimagesofblockfaces,cutbyaminiaturemicrotomewithinthesema technicalnote.ScanningElectronMicroscopy:7376,1981. LEITLE.Neurosuspensionanduploading.2007,1995 http://www.aleph.se/Trans/Individual/Cryonics/neurosusp_upload.txt. LESTIENNER.Determinationoftheprecisionofspiketiminginthevisualcortexof anaesthetisedcats.BiologicalCybernetics,74:5561,1996. LEVENSONJMandSWEATTJD.Epigeneticmechanismsinmemoryformation.NatureReviews Neuroscience,6:108118,2005. LIC,YUJ,andLIAOX.Chaosinathreeneuronhysteresishopfieldtypeneuralnetworks. PhysicsLettersA,285:368372,2001. LIKW.Proteomicsofsynapse.AnalyticalandBioanalyticalChemistry,387:2528,2007. LIXQ,WUYW,STEELD,etal.Anallopticalquantumgateinasemiconductorquantumdot. Science,301:809811,2003. LIEBERMANM,CHELLAMMAS,VARUGHESEB,etal.Quantumdotcellularautomataata molecularscale.MolecularElectronicsIi,960:225239,2002. LIKHAREVKKandSEMENOVVK.Rsfqlogic/memoryfamily:Anewjosephsonjunction technologyforsubterahertzclockfrequencydigitalsystems.Applied Superconductivity,IEEETransactionson,1:328,1991. LINMCandOTADUYMA.Sensationpreservinghapticrendering.IeeeComputerGraphicsand Applications,25:811,2005. LITTA,ELIASMITHC,KROONFW,WEINSTEINS,andTHAGARDP.Isthebrainaquantum computer?CognitiveScience,30:593603,2006. LLOYDDGandBESIERTF.Anemgdrivenmusculoskeletalmodeltoestimatemuscleforces andkneejointmomentsinvivo.JBiomech,36:76576,2003. LONDONMandHAUSSERM.Dendriticcomputation.AnnualReviewofNeuroscience,28:503 532,2005. LONGSOND,HUTCHINSONCE,DOYLECA,etal.Useofmriformeasuringstructuresinfrozen postmortembrain.BrainResearchBulletin,38:457460,1995.
121
LOVESandCOAKHAMHB.Trigeminalneuralgiapathologyandpathogenesis.Brain,124: 23472360,2001. LUIV,FRSTERF,andBAUMEISTERW.Structuralstudiesbyelectrontomography:From cellstomolecules.AnnualReviewofBiochemistry,74:83365,2005. MAYY,HUH,BERREBIAS,MATHERSPH,andAGMONA.Distinctsubtypesofsomatostatin containingneocorticalinterneuronsrevealedintransgenicmice.Journalof Neuroscience,26:50695082,2006. MABUCHIM,SHIMADAJ,OKAMOTOK,etal.Timeresolvedfluorescencespectroscopyof dopamineinsinglecells.InLakowiczJ.R.andThompsonR.B.(Eds.),Proceedingsof spievolume4252:Advancesinfluorescencesensingtechnologyv:SPIE,2001,pp.140148. MACLEODKM,HORIUCHITK,andCARRCE.Aroleforshorttermsynapticfacilitationand depressionintheprocessingofintensityinformationintheauditorybrainstem.J Neurophysiol,97:286374,2007. MAGNENATTHALMANNNandCORDIERF.Constructionofahumantopologicalmodelfrom medicaldata.IeeeTransactionsonInformationTechnologyinBiomedicine,4:137143, 2000. MAHVASHMandHAYWARDV.Highfidelityhapticsynthesisofcontactwithdeformable bodies.IeeeComputerGraphicsandApplications,24:4855,2004. MALICKASA.Gradualuploadingasacognitionofmind.2007,1996 http://www.aleph.se/Trans/Global/Uploading/gupload.html. MALINSKIT,TAHAZ,GRUNFELDS,etal.Diffusionofnitricoxideintheaortawallmonitored insitubyporphyrinicmicrosensors.BiochemicalandBiophysicalResearch Communications,193:10761082,1993. MARKRAMH.Neocorticalmicrocircuitdatabase.EPFL,2005http://microcircuit.epfl.ch/. MARKRAMH.Thebluebrainproject.NatureReviewsNeuroscience,7:153160,2006. MARKRAMH,LUBKEJ,FROTSCHERM,andSAKMANNB.Regulationofsynapticefficacyby coincidenceofpostsynapticapsandepsps.Science,275:213215,1997. MARRONEDF.Ultrastructuralplasticityassociatedwithhippocampaldependentlearning:A metaanalysis.NeurobiologyofLearningandMemory,87:361371,2007. MARRONEDFandPETITTL.Theroleofsynapticmorphologyinneuralplasticity:Structural interactionsunderlyingsynapticpower.BrainResearchReviews,38:291308,2002. MASONAandLARKMANA.Correlationsbetweenmorphologyandelectrophysiologyof pyramidalneuronsinslicesofratvisualcortex.2.Electrophysiology.Journalof Neuroscience,10:14151428,1990. MATTIAMandGIUDICEPD.Efficienteventdrivensimulationoflargenetworksofspiking neuronsanddynamicalsynapses.NeuralComputation,12:23052329,2000. MAYERICHD,ABBOTTL,andMCCORMICKB.Knifeedgescanningmicroscopyforimagingand reconstructionofthreedimensionalanatomicalstructuresofthemousebrain.2008 MAYERICHD,MCCORMICKBH,andKEYSERJ.Noiseandartifactremovalinknifeedge scanningmicroscopy.InPiscatawayN.(Ed.)Proceedingsof2007ieeeinternational symposiumonbiomedicalimaging:Fromnanotomacro:IEEEPress,2007. MAYERICHDMandKEYSERJ.Filamenttrackingandencodingforcomplexbiological networks.InProceedingsofacmsymposiumonsolidandphysicalmodeling,2008,pp.353 358. MCCALLUMJC.Cpupriceperformance19442003.2007,2003http://www.jcmit.com/cpu performance.htm. MCCALLUMJC.Diskdriveprices(19552007).2007,2007ahttp://www.jcmit.com/diskprice.htm. MCCALLUMJC.Memoryprices(19572007).2007,2007b http://www.jcmit.com/memoryprice.htm. MCCLUNGCAandNESTLEREJ.Neuroplasticitymediatedbyalteredgeneexpression. Neuropsychopharmacology,33:317,2008.
122
MCCORMICKBH.Braintissuescannerenablesbrainmicrostructuresurveys.Neurocomputing, 44:11131118,2002a. MCCORMICKBH.Developmentofthebraintissuescanner.DepartmentofComputerScience, TexasA&MUniversity,2002b MCCORMICKBH,KOHW,CHOEY,etal.Constructionofanatomicallycorrectmodelsof mousebrainnetworks.Neurocomputing,5860:379386,2004. MCCULLOCHWSandPITTSWH.Alogicalcalculusoftheideasimmanentinnervousactivity. .BulletinofMathematicalBiophysics,5:115133,1943. MCDOUGALLMPandWRIGHTSM.Initialresultsinwidefield3dmrmicroscopyusing parallelimaging.InBiomedicalimaging:Fromnanotomacro,2007.Isbi2007.4thieee internationalsymposiumon:IEEE,2007,pp.10721075. MCGUIGANM.Graphicsturingtest,arxiv:Cs.Gr/0603132.2007,2006 http://arxiv.org/abs/cs.GR/0603132. MEHRTASHN,JUNGD,HELLMICHHH,etal.Synapticplasticityinspikingneuralnetworks ((spinn)i2):Asystemapproach.IeeeTransactionsonNeuralNetworks,14:980992, 2003. MEINDLJD,CHENQ,andDAVISJA.Limitsonsiliconnanoelectronicsforterascaleintegration. Science,293:20442049,2001. MERKLERC.Energylimitstothecomputationalpowerofthehumanbrain.ForesightUpdate, 6,1989a. MERKLERC.Largescaleanalysisofneuralstructures.PaloAlto,California:XeroxPaloAlto ResearchCenter,1989bhttp://www.merkle.com/merkleDir/brainAnalysis.html. MERKLERC.Twotypesofmechanicalreversiblelogic.Nanotechnology,4:114131,1993. MERKLERC.Themolecularrepairofthebrain.Cryonicsmagazine,15,1994. MERKLERCandDREXLERKE.Helicallogic.Nanotechnology,7:325339,1996. METHEO,SPRINGH,GUTTMANNP,etal.Transmissionxraymicroscopyofintacthydrated ptk2cellsduringthecellcycle.JournalofMicroscopyOxford,188:125135,1997. MICHELSONSandCOLEM.Thefutureofpredictivebiosimulationindrugdiscovery.Expert OpiniononDrugDiscovery,2:515523,2007. MICHEVAKDandSMITHSJ.Arraytomography:Anewtoolforimagingthemolecular architectureandultrastructureofneuralcircuits.Neuron,55:2536,2007. MIGLIOREM,CANNIAC,LYTTONWW,MARKRAMH,andHINESML.Parallelnetwork simulationswithneuron.JournalofComputationalNeuroscience,21:119129,2006. MILLERCAandSWEATTJD.CovalentmodificationofDNAregulatesmemoryformation. Neuron,53:857869,2007. MIZUTANIR,TAKEUCHIA,HARAT,UESUGIK,andSUZUKIY.Computedtomographyimaging oftheneuronalstructureofdrosophilabrain.JournalofSynchrotronRadiation,14:282 287,2007. MOLLMandMIIKKULAINENR.Convergencezoneepisodicmemory:Analysisand simulations.NeuralNetworks,10:10171036,1997. MOOREJWandHINESML.Simulationswithneuron,1994. MORAVECH.Whenwillcomputerhardwarematchthehumanbrain?JournalofEvolutionand Technology,1,1998. MORAVECH.Robot:Meremachinetotranscendentmind.NewYork:OxfordUniversityPress, 1999. MOREVECH.Mindchildren:Thefutureofrobotandhumanintelligence:HarvardUniversityPress, 1988. MORRISONA,MEHRINGC,GEISELT,AERTSENA,andDIESMANNMA.Advancingthe boundariesofhighconnectivitynetworksimulationwithdistributedcomputing. NeuralComputation,17:17761801,2005.
123
MULDERSJL,KNOTTG,andLICHBH.Dualbeamslice&view:Practicalaspectsforcollecting3d corteximagedata.Chicago,2006 MULLERTIDOWC,SCHWABLEJ,STEFFENB,etal.Highthroughputanalysisofgenomewide receptortyrosinekinaseexpressioninhumancancersidentifiespotentialnoveldrug targets.ClinicalCancerResearch,10:12411249,2004. MUOTRIARandGAGEFH.Generationofneuronalvariabilityandcomplexity.Nature,441: 10871093,2006. NADALJPandTOULOUSEG.Informationstorageinsparselycodedmemorynets.Network: ComputationinNeuralSystems,1:6174,1990. NADALJP.Associativememoryonthe(puzzling)sparsecodinglimit.JournalofPhysicsa MathematicalandGeneral,24:10931101,1991. NADIMFandMANORY.Theroleofshorttermsynapticdynamicsinmotorcontrol.CurrOpin Neurobiol,10:68390,2000. NANOROADMAPPROJECT.Roadmapsat2015onnanotechnologyapplicationinthesectorsof: Materials,health&medicalsystems,energy.EuropeanCommission,2006 http://www.nanoroadmap.it/. NELLISTPD,CHISHOLMMF,DELLBYN,etal.Directsubangstromimagingofacrystallattice. Science,305:17411741,2004. NEUMANNJV.Thecomputerandthebrain.NewHaven:YaleUniversityPress,1958. NEWMANEAandZAHSKR.Modulationofneuronalactivitybyglialcellsintheretina.Journal ofNeuroscience,18:40224028,1998. NICHOLSONC.Diffusionandrelatedtransportmechanismsinbraintissue.ReportsonProgress inPhysics,64:815884,2001. NORDHAUSWD.Theprogressofcomputing.2001http://ssrn.com/abstract=285168. NUSBAUMMPandBEENHAKKERMP.Asmallsystemsapproachtomotorpatterngeneration. Nature,417:343350,2002. OBERSTH,KOUZNETSOVD,SHIMIZUK,FUJITAJ,andSHIMIZUF.Fresneldiffractionmirrorfor anatomicwave.PhysicalReviewLetters,94:,2005. ORDT.Themanyformsofhypercomputation.AppliedMathematicsandComputation,178:143 153,2006. ORTOLEVAP,BERRYE,BRUNY,etal.Thekaryotephysicochemicalgenomic,proteomic, metaboliccellmodelingsystem.OMICS,7:269283,2003. PAJCINIV,MUNROCH,BORMETTRW,WITKOWSKIRE,andASHERSA.Uvraman microspectroscopy:Spectralandspatialselectivitywithsensitivityandsimplicity. AppliedSpectroscopy,51:8186,1997. PALAYSL,MCGEERUSSELLSM,GORDONJR.S,andGRILLOMA.Fixationofneuraltissuesfor electronmicroscopybyperfusionwithsolutionsofosmiumtetroxide.TheJournalof CellBiology,12,1962. PARFITD.Reasonsandpersons:OxfordUniversityPress,1984. PARPURAV,TONGW,YEUNGES,andHAYDONPG.Laserinducednativefluorescence(linf) imagingofserotonindepletionindepolarizedneurons.JournalofNeuroscience Methods,82:151158,1998. PEARSONK,EKEBERG,andBUSCHGESA.Assessingsensoryfunctioninlocomotorsystems usingneuromechanicalsimulations.TRENDSinNeurosciences,29:625+631,2006. PENCZEKP,MARKOM,BUTTLEK,andFRANKJ.Doubletiltelectrontomography. Ultramicroscopy,60:393410,1995. PENROSER.TheemperorsnewmindNewYork:OxfordUniversityPress,1989. PEREAGandARAQUEA.Propertiesofsynapticallyevokedastrocytecalciumsignalreveal synapticinformationprocessingbyastrocytes.JournalofNeuroscience,25:21922203, 2005. PETERSA.Thalamicinputtothecerebralcortex.TrendsNeurosci.,2:11831185,1979.
124
PETERSAandPALAYSL.Themorphologyofsynapses.JournalofNeurocytology,25:687700, 1996. PFEIFFERJ,JOHNSOND,andNEHRKEK.Oscillatorytransepithelialh+fluxregulatesarhythmic behaviorinc.Elegans.CurrentBiology,18:297302,2008. PICKARDDS,GROVESTR,MEISBURGERWD,CRANET,andPEASERF.Distributedaxiselectron beamtechnologyformasklesslithographyanddefectinspection.JournalofVacuum Science&TechnologyB,21:28342838,2003. PLESSERHE,EPPLERJM,MORRISONA,DIESMANNM,andGEWALTIGMO.Efficientparallel simulationoflargescaleneuronalnetworksonclustersofmultiprocessorcomputers. KermarrecA.M.,BougL.,andPriolT.:672681,2007 PUPPELSGJ,OLMINKHOFJHF,SEGERSNOLTENGMJ,etal.Laserirradiationandraman spectroscopyofsinglelivingcellsandchromosomessampledegradationoccurs with514.5nmbutnotwith660nmlaserlight.ExperimentalCellResearch,195:361367, 1991. RAGHUVANSHINandLINMC.Interactivesoundsynthesisforlargescaleenvironments.Redwood City,California:ACMPress:1011082006 RAGHUVANSHINandLINMC.Physicallybasedsoundsynthesisforlargescalevirtual environments.IeeeComputerGraphicsandApplications,27:1418,2007. RALLW.Theoryphysiologicalpropertiesofdendrites.AnnalsoftheNewYorkAcademyof Sciences,96:1071&,1962. RALSTONTS,MARKSDL,CARNEYPS,andBOPPARTSA.Interferometricsyntheticaperture microscopy.NaturePhysics,3:129134,2007. REHNMandLANSNERA.Sequencememorywithdynamicalsynapses.Neurocomputing,5860: 271278,2004. REUTSKIYS,ROSSONIE,andTIROZZIB.Conductioninbundlesofdemyelinatednervefibers: Computersimulation.BiologicalCybernetics,89:439448,2003. RICEME.Distinctregionaldifferencesindopaminemediatedvolumetransmission.Volume TransmissionRevisited,125:277290,2000. RIEKEF,WARLANDD,DERUYTERVANSTEVENINCKR,andBIALEKW.Spikesexploringthe neuralcode.Cambridge,MA.:MITPress,1996. ROBLEDOL,ELZERMANJ,JUNDTG,etal.Conditionaldynamicsofinteractingquantumdots. Science,320:772775,2008. ROBLESDELATORREG.Theimportanceofthesenseoftouchinvirtualandreal environments.IeeeMultimedia,13:2430,2006. ROSALPandFABERJ.Quantummodelsofthemind:Aretheycompatiblewithenvironment decoherence?PhysicalReviewE,70:,2004. RUECKEST,KIMK,JOSELEVICHE,etal.Carbonnanotubebasednonvolatilerandomaccess memoryformolecularcomputing.Science,289:9497,2000. RUGARD,BUDAKIANR,MAMINHJ,andCHUIBW.Singlespindetectionbymagnetic resonanceforcemicroscopy.Nature,430:32932,2004. RUMELHARTDE,MCCLELLANDJL,andTHEPDPRESEARCHGROUP.Paralleldistributed processing:Explorationsinthemicrostructureofcognition.Cambridge:MITPress,1986. SAKAKURAM,KAJIYAMAS,TSUTSUMIM,etal.Femtosecondpulsedlaserasamicroscalpelfor microdissectionandisolationofspecificsectionsfrombiologicalsamples.Japanese JournalofAppliedPhysicsPart1RegularPapersBriefCommunications&ReviewPapers, 46:58595864,2007. SALIOC,LOSSIL,FERRINIF,andMERIGHIA.Neuropeptidesassynaptictransmitters.Celland TissueResearch,326:583598,2006. SANDIANATIONALLABORATORIES.Onemilliontrillionflopspersecondtargetedbynewinstitute foradvancedarchitectures.2008 http://www.sandia.gov/news/resources/releases/2008/exaflop.html.
125
SANTAMARAPANGA,BILDEATS,COLBERTC,SAGGAUP,andKAKADIARISIA.Towards segmentationofirregulartubularstructuresin3dconfocalmicroscopeimages.In Proc.Ofthemiccaiworkshopinmicroscopicimageanalysisandapplicationsinbiology (miaab),denmark,copenhangen,oct16,2006,2006. SAWCHUKAAandSTRANDTC.Digitalopticalcomputing.ProceedingsoftheIeee,72:758779, 1984. SAXEMD,MALLERETG,VRONSKAYAS,etal.Paradoxicalinfluenceofhippocampal neurogenesisonworkingmemory.PNAS,104464246462007. SCHAFFJ,SLEPCHENKOB,andMORGANF.Virtualcell.2001http://www.nrcam.uchc.edu/. SCHEEPERSF,PARENTRE,CARLSONWE,andMAYSF.Anatomybasedmodelingofthehuman musculature.InOwenG.S.,WhittedT.,andMonesHatta;B.(Eds.),Proceedingsofthe 24thannualconferenceoncomputergraphicsandinteractivetechniques,1997,pp.163172. SCHMITTO,MODERSITZKIJ,HELDMANNS,WIRTZS,andFISCHERB.Imageregistrationof sectionedbrains.InternationalJournalofComputerVision,73:539,2007. SCHNELLSandTURNERTE.Reactionkineticsinintracellularenvironmentswith macromolecularcrowding:Simulationsandratelaws.ProgressinBiophysics& MolecularBiology,85:235260,2004. SCHULZDJ,GOAILLARDJM,andMARDERE.Variablechannelexpressioninidentifiedsingle andelectricallycoupledneuronsindifferentanimals.NatureNeuroscience,9:356362, 2006. SCHULZDJ,GOAILLARDJM,andMARDEREE.Quantitativeexpressionprofilingofidentified neuronsrevealscellspecificconstraintsonhighlyvariablelevelsofgeneexpression. ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,104:13187 13191,2007. SCHUMANEMandMADISONDV.Locallydistributedsynapticpotentiationinthe hippocampus.Science,263:532536,1994. SEARLEJR.Minds,brains,andprograms.BehavioralandBrainSciences,3:417425,1980. SEITZBTJ.Thegreatgrayravelledknot.2007 http://www.geocities.com/rnseitz/The_Great_Gray_Ravelled_Knot.htm. SENNW,MARKRAMH,andTSODYKSM.Analgorithmformodifyingneurotransmitterrelease probabilitybasedonpreandpostsynapticspiketiming.NeuralComputation,13:35 67,2001. SERRANOL.Syntheticbiology:Promisesandchallenges.MolecularSystemsBiology,3:,2007. SHAOJUANZandHAMMARSTROMD.Simulationofassociativeneuralnetworks.InProceedings ofthe9thinternationalconferenceonneuralinformationprocessing,2002.Iconip02.,2002, pp.16391643. SHEPHERDGM.Thehumansenseofsmell:Arewebetterthanwethink?PlosBiology,2:572 575,2004. SHEPHERDGMG,STEPANYANTSA,BUREAUI,CHKLOVSKIID,andSVOBODAK.Geometricand functionalorganizationofcorticalcircuits.NatureNeuroscience,8:782790,2005. SHIMIZUFandFUJITAJ.Reflectiontypehologramforatoms.PhysicalReviewLetters,88:,2002. SIDIROPOULOUK,PISSADAKIEK,andPOIRAZIP.Insidethebrainofaneuron.EMBORep,7: 88692,2006. SIEGELMANNHTandSONTAGED.Onthecomputationalpowerofneuralnets.Journalof ComputerandSystemSciences,50:132150,1995. SILBERBERGG,GUPTAA,andMARKRAMH.Stereotypyinneocorticalmicrocircuits.TRENDSin Neurosciences,25:227230,2002. SINGLESandBORSTA.Dendriticintegrationanditsroleincomputingimagevelocity.Science, 281:18481850,1998. SPORNSO,TONONIG,andKTTERR.Thehumanconnectome:Astructuraldescriptionofthe humanbrain.PlosComputationalBiology,1:245251,2005.
126
STEFANOVA,GISINN,GUINNARDO,GUINNARDL,andZBINDENH.Opticalquantumrandom numbergenerator.JournalofModernOptics,47:595598,2000. STEPANYANTSA,HOFPR,andCHKLOVSKIIDB.Geometryandstructuralplasticityofsynaptic connectivity.Neuron,34:275288,2002. STROHMAIEREandMEUERHW.Supercomputing:Whathavewelearnedfromthetop500 project?ComputingandVisualizationinScience,6:227230,2004. STROUTJ.Minduploadinghomepage.2007,2006 http://www.ibiblio.org/jstrout/uploading/MUHomePage.html. STUHRMANNB,JAHNKEHG,SCHMIDTM,etal.Versatileopticalmanipulationsystemfor inspection,laserprocessing,andisolationofindividuallivingcells.ReviewofScientific Instruments,77:,2006. STUMPFMPH,THORNET,DESILVAE,etal.Estimatingthesizeofthehumaninteractome. ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,105:6959 6964,2008. SUZUKIM,GOTOT,TSUJIT,andOHTAKEH.Adynamicbodymodelofthenematodec. Eleganswithneuraloscillators.JournalofRoboticsandMechatronics,17:318326,2005. SWANSONJandKHEIFETSL.Biophysicalmechanisms:Acomponentintheweightofevidence forhealtheffectsofpowerfrequencyelectricandmagneticfields.RadiationResearch, 165:470478,2006. SZABON.Falsifiabledesign:Amethodologyforevaluatingtheoreticaltechnologies.2007 http://unenumerated.blogspot.com/2007/02/falsifiabledesignmethodologyfor.html. TAKAHASHIK,YUGIK,HASHIMOTOK,etal.Computationalchallengesincellsimulation:A softwareengineeringapproach.IeeeIntelligentSystems,17:6471,2002. TANWH,PARPURAV,HAYDONPG,andYEUNGES.Neurotransmitterimaginginlivingcells basedonnativefluorescencedetection.AnalyticalChemistry,67:25752579,1995. TEGMARKM.Importanceofquantumdecoherenceinbrainprocesses.PhysicalReviewE,61: 41944206,2000. TERANJ,SIFAKISE,BLEMKERSS,etal.Creatingandsimulatingskeletalmusclefromthevisible humandataset.IeeeTransactionsonVisualizationandComputerGraphics,11:317328, 2005. THAGARDP.Howmoleculesmattertomentalcomputation.PhilosophyofScience,69:429446, 2002. THOMASS.Endogenousneuroactiveextracellularsignaltransducers.Neurotransmitter.net,2006 http://www.neurotransmitter.net/neurosignaling.html. THOMSONA.Facilitation,augmentationandpotentiationatcentralsynapses.Trends Neuroscience,23:305312,2000. TIMLERJandLENTCS.Maxwellsdemonandquantumdotcellularautomata.Journalof AppliedPhysics,94:10501060,2003. TOLEDORODRIGUEZM,BLUMENFELDB,WUCZ,etal.Correlationmapsallowneuronal electricalpropertiestobepredictedfromsinglecellgeneexpressionprofilesinrat neocortex.CerebralCortex,14:13101327,2004. TOMITAM.Ecell.2001http://www.ecell.org/ecell/. TOUGAWPDandLENTCS.Logicaldevicesimplementedusingquantumcellularautomata. JournalofAppliedPhysics,75:18181825,1994. TRAUBRD,CONTRERASD,CUNNINGHAMMO,etal.Singlecolumnthalamocorticalnetwork modelexhibitinggammaoscillations,sleepspindles,andepileptogenicbursts.Journal ofNeurophysiology,93:21942232,2005. TRAUBRD,MILESR,andBUZSAKIG.Computersimulationofcarbacholdrivenrhythmic populationoscillationsintheca3regionoftheinvitrorathippocampus.Journalof PhysiologyLondon,451:653672,1992.
127
TRAUBRDandWONGRKS.Cellularmechanismofneuronalsynchronizationinepilepsy. Science,216:745747,1982. TSAIPS,FRIEDMANB,IFARRAGUERRIAI,etal.Allopticalhistologyusingultrashortlaser pulses.Neuron,39:2741,2003. TSANGWJ.Synapticultrastructuralreconstructionusingserialelectronmicroscopy.148149,2005 TSENGGYandELLENBOGENJC.Nanotechnologytowardnanocomputers.Science,294:1293 1294,2001. TSODYKSM,PAWELZIKK,andMARKRAMH.Neuralnetworkswithdynamicsynapses.Neural Computation,10:821835,1998. TUSZYNSKIJ.Thedynamicsofcterminiofmicrotubulesindendrites:Apossibleclueforthe roleofneuralcytoskeletoninthefunctioningofthebrain.JournalofGeoethical Nanotechnology,1,2006. TYSONJ.Jigcell.2001http://jigcell.biol.vt.edu/. UEHARAC,COLBERTC,SAGGAUP,andKAKADIARISI.Towardsautomaticreconstructionof dendritemorphologyfromliveneurons.ConfProcIEEEEngMedBiolSoc,3:1798801, 2004. URBANS,OMALLEYSM,WALSHB,etal.Automaticreconstructionofdendritemorphology fromopticalsectionstacks.InProc.2ndinternationalworkshoponcomputervision approachestomedicalimageanalysis,graz,austria,may2006.,2006,pp.190201. VALBERGPA,KAVETR,andRAFFERTYCN.Canlowlevel50/60hzelectricandmagneticfields causebiologicaleffects?RadiationResearch,148:221,1997. VANBRUGGENMJ,VANSOMERENB,andKRUITP.Developmentofamultielectronbeam sourceforsub10nmelectronbeaminduceddeposition.JournalofVacuumScience& TechnologyB,23:28332839,2005. VANGEITW,ACHARDP,andDESCHUTTERE.Neurofitter:Aparametertuningpackagefora widerangeofelectrophysiologicalneuronmodels.Frontiersinneuroinformatics,1:1 18,2007. VANSOMERENB,VANBRUGGENMJ,ZHANGY,HAGEMCW,andKRUITP.Multibeamelectron sourceusingmemselectronopticalcomponents.JournalofPhysicsaMathematicaland General,Conferenceseries34:10921097,2006. VANIERMCandBOWERJM.Acomparativesurveyofautomatedparametersearchmethods forcompartmentalneuralmodels.JournalofComputationalNeuroscience,7:149171, 1999. VENTERJCADAMSMDMYERSEW,etal.Thesequenceofthehumangenome.Science,291: 1304+,2001. VONBOHLENUNDHALBACHOandDERMIETZELR.Neurotransmittersandneuromodulators. Weinheim:WileyVCH,2006. WALTERC.Kryderslaw.ScientificAmerican:3233,2005. WANGY,LIUD,andWANGY.Discoveringthecapacityofhumanmemory.BrainandMind,4: 189198,2003. WANGZHandGRABOWSKIPJ.Cellandstagespecificsplicingeventsresolvedinspecialized neuronsoftheratcerebellum.RnaaPublicationoftheRnaSociety,2:12411253,1996. WEDELBJandGARBERSDL.Guanylylcyclases:Approachingyearthirty.Trendsin EndocrinologyandMetabolism,9:213219,1998. WEHNERM,OLIKERL,andSHALFJ.Towardsultrahighresolutionmodelsofclimateand weather.InternationalJournalofHighPerformanceComputingApplications,22:149165, 2008. WEINSTEINRKandLEERH.Designofhighperformancephysiologicallycomplexmotoneuronmodels infpgas.Arlington,Virginia,2005 WEINSTEINRKandLEERH.Architecturesforhighperformancefpgaimplementationsof neuralmodels.JournalofNeuralEngineering,3:2134,2006.
128
WELSERJJ,BOURIANOFFGI,ZHIRNOVVV,andCAVINRK.Thequestforthenextinformation processingtechnology.JournalofNanoparticleResearch,10:110,2008. WHITEJG,SOUTHGATEE,THOMSONJN,andBRENNERS.Thestructureofthenervoussystem ofthenematodecaenorhabditiselegans.PhilosophicalTransactionsoftheRoyalSociety ofLondonSeriesBBiologicalSciences,314:1340,1986. WILKINSMR,SANCHEZJC,WILLIAMSKL,andHOCHSTRASSERDF.Currentchallengesand futureapplicationsforproteinmapsandposttranslationalvectormapsinproteome projects.Electrophoresis,17:830838,1996. WILSONMA,BHALLAUS,UHLEYJD,andBOWERJM.Genesis:Asystemforsimulatingneural networks.InTouretzkyD.(Ed.)Advancesinneuralinformationprocessingsystems.San Mateo:MorganKaufmann,1989,pp.485492. WISEA,JUPESC,andREESS.Theidentificationofligandsatorphangproteincoupled receptors.AnnualReviewofPharmacologyandToxicology,44:4366,2004. WISHARTDS,TZURD,KNOXC,etal.Hmdb:Thehumanmetabolomedatabase.NucleicAcids Research,35:D521D526,2007. WOODJandGARTHWAITEJ.Modelsofthediffusionalspreadofnitricoxideimplicationsfor neuralnitricoxidesignalinganditspharmacologicalproperties.Neuropharmacology, 33:12351244,1994. XUFandDINGHS.Anewkineticmodelforheterogeneous(orspatiallyconfined)enzymatic catalysis:Contributionsfromthefractalandjamming(overcrowding)effects.Applied CatalysisaGeneral,317:7081,2007. YAMAMOTOYandSHINOHARAK.Applicationofxraymicroscopyinanalysisofliving hydratedcells.TheAnatomicalRecord,269:217223,2002. YOSHIKAWAN,MATSUZAKIF,NAKAJIMAN,andYODAK.Designandcomponenttestofa1bit rsfqmicroprocessor.PhysicaCSuperconductivityandItsApplications,378:14541460, 2002. YUFH,YAROVYAROVOYV,GUTMANGA,andCATTERALLWA.Overviewofmolecular relationshipsinthevoltagegatedionchannelsuperfamily.PharmacologicalReviews, 57:387395,2005. YUCESOYCA,KOOPMANBHFJM,HUIJINGPA,andGROOTENBOERHJ.Threedimensionalfinite elementmodelingofskeletalmuscleusingatwodomainapproach:Linkedfiber matrixmeshmodel.JBiomech,35:12531262,2002. ZENILHandHERNANDEZQUIROZF.Onthepossiblecomputationalpowerofthehuman mind.InGershensonC.,AersD.,andEdmondsB.(Eds.),Worldviews,scienceandus: Philosophyandcomplexity:WorldScientific,2007,pp.315334. ZHANGM,WANGL,andYEP.Allopticalxorlogicgates:Technologiesandexperiment demonstrations.CommunicationsMagazine,IEEE,43:S19S24,2005. ZHANGWX,ZHANGY,ZHENGH,etal.Syndb:Asynapseproteindatabasebasedonsynapse ontology.NucleicAcidsResearch,35:D737D741,2007. ZHAOK,DALTONP,YANGGC,andSCHERERPW.Numericalmodelingofturbulentand laminarairflowandodoranttransportduringsniffinginthehumanandratnose. ChemicalSenses,31:107118,2006. ZHENGY,KREUWELHTC,YOUNGDL,etal.Thevirtualnodmouseapplyingpredictive biosimulationtoresearchintype1diabetes.HowDoWeBestEmployAnimalModelsfor Type1DiabetesandMultipleSclerosis?,1103:4562,2007. ZINOVIEVDY.Designissuesinultrafastultralowpowersuperconductorbatcherbanyan switchingfabricbasedonrsfqlogic/memoryfamily.AppliedSuperconductivity,5:235 239,1997. ZORDANVB,CELLYB,CHIUB,andDILORENZOYPC.Breatheeasy:Modelandcontrolof simulatedrespirationforanimation.InBoulicR.andPaiD.K.(Eds.),Eurographics/acm siggraphsymposiumoncomputeranimation(2004),2004,pp.2937.
129
130