AnintroductiontoVMEbus

Overview
Whatyoualreadyshouldknow
VMEbus
Introduction
Addressing
Singlecycles
Blocktransfers
Interrupts
VME64x
Systemassembly
SingleBoardComputer
Software
Tools

1
 Whatyoualreadyshouldknow
C(++)programming
useofpointers
signals
datatypes(char,short,int)
useofC++methods
Linux
CVS/CMT
Makefiles
gdb
Sharedlibraries
UsuallytheenvironmentvariableLD_LIBRARY_PATHtellsthe
linkerwheretofindsharedlibraries
Handlingdrivers
Generaloperation(cd,ls,mkdir,etc.)

2
 VMEbusmechanics

VMEbuscardsexistin3standardheights:3U,6Uand9U
Definition:1U=1.75inch

In6Uand9Usystemstherecanbetransitionmodulesinstalledontherearsideof
thebackplane.TransitionmodulesdonotconnecttoVMEbusbutjusttothe
VMEbusmoduleontheoppositesideofthebackplaneviatheuserdefinedpinsof
theJ0,J2andJ3connectors
3
VMEbusmechanics(2)
Backplane
Backplane
P1 J1
3Umodule
P1 J1

6Utransition
P0 J0

module
Backplane
P2 J2
6Umodule
P1 J1

9Utransitionmodule
P0 J0

P2 J2

P3 J3
9Umodule

4
 VMEbusmechanics(3)
Example:6UVME64xmodule

Alignmentpin
Incompatiblewithcertainoldcrates

5rowP1connector
160pinsusedforVMEbus
Insertionforce
40kg (415pins*1N)

P0connector
UsedforPMCI/O
Incompatiblewith
certaincrates
(Jaux,VME64xP)
Injector/extractorhandles
5rowP2connector
Pushredbutton
32pinsusedforVMEbus
Otherpinsuserdefined(e.g.
fortransitionmodules)
Dischargestrip
5
 VMEbuscrates
21slot9Ucrate
(with6Usection)
for19racks

21slot6Ucrate
for19racks

Therearedifferenttypesofpowersupplies(5V,+/12V,3.3V,48V)
mountedlocallyorremote
Thefantrayunitallowstomonitorparameterslikevoltages,currents,fan
speed,temperature
(Some)cratescanbecontrolledbyafieldbus(CAN)
ATTENTION:TheEMCgaskettotheleftofslot1maydamageyour
VMEbuscards
6
 VMEbusBackplane
BG0
J1 BG1
Daisychain BG2
jumpers(only
onsomeold BG3
backplanes)
J0
IACK

J2

J3(9U
crates
only)
Slot1

Frontview 6UVME64xbackplane 7
 VMEbusbasics
Electricalproperties
AlllinesuseTTLlevels
Low=0...0.6V
High=2.4...5V
Address,addressmodifieranddatalinesareactivehigh
Protocollinesareactivelow
Protocol
Asynchronouswith4edgehandshaking.
ThedurationofaVMEbuscycledependsonthespeedofthemasterand
theslave
Byteordering
VMEbusisbigendian.Itstoresthemostsignificantbyteofawordatthe
lowestbyteaddress(0x0)
PCIandIntelCPUsarelittleendian.Theystorethemostsignificantbyte
ofawordatthehighestbyteaddress(0x3)
MostVMEbusmasters(e.g.VP110)haveautomaticbyteswappinglogic

8
 VMEbusbasics(2)
Typesofcommonmodules(physicalandlogical)
Master
Amodulethatcaninitiatedatatransfers
Slave
Amodulethatrespondstoamaster
Interrupter
Amodulethatcansendaninterrupt(usuallyaslave)
Interrupthandler
Amodulethatcanreceive(andhandle)interrupts(usuallyaSingle
BoardComputer)
Arbiter
Apieceofelectronics(usuallyincludedintheSBC)thatarbitrates
busaccessandmonitorsthestatusofthebus.Itshouldalwaysbe
installedinslot1oftheVMEbuscrate

9
 VMEbusbasics(3)
Maintypesofdatatransfers
Singlecycles
Transfer8,16or32bitsofdata(typically)underthecontroloftheCPUonthemaster
Typicalduration:1us
Blocktransfer(DMA)
Transferanyamountofdata(usually32or64bitatatime)underthecontrolofaDMA
controller(CPUindependent)
Dataistransferredinburstsofupto256(D32)or2048(D64)bytes
Typicalduration:150nsperdataword
Interrupts
Usedtypicallybyslavestosignalacondition(e.g.dataavailable,internalerror,etc.)
Can(inprinciple)have7priorities
Theinterrupterprovidesan8bitvectoronrequestoftheinterrupthandlertoidentify
itself
ROAK(ReleaseonAcknowledge)orRORA(ReleaseOnRegisterAccess)

10
 VMEbusprotocol

WhydoIhavetounderstandhowtheprotocol
worksifIamnotdesigningcards?

SomedesignersmakemistakesandtheirVMEbuscardsdonotworkatall
orfailincombinationwithcertainothercards
DebuggingVMEbustrafficbyS/W(printf(),gdb,etc.)isdifficultoreven
impossible
AgreathelpforfixingsuchproblemsareVMEbusanalyzers
AVMEbusanalyzeralsotellsyouifyouarereallyexecutingthedesired
typesofcycles(D8,D16,D32,etc.)
Inordertounderstandtheoutputofsuchananalyzeryouhavetohave
someknowledgeoftheprotocol

11
 Importantsignals
Name Description
BBSY* BusBusy.OnceamasterhasbeengrantedthebusitdrivesBBSY*.Aslongas
BBSY*isassertednoothermastercangetthebus
A[31..1] Addresslines(cancarrydatainD64multiplexedtransfers)
D[31..0] Datalines
AM[5..0] Addressmodifier.Definesthenumberofvalidaddressbitsandcycletype
DS0*andDS1* Datastrobes.Telltheslavewhenthemasterisready.Alsoencodethenumber
ofbytestobetransferred
LWORD* ContributestothedefinitionofthetransfersizeandcarriesdatainMBLT
cycles
AS* AddressStrobe.Tellstheslaveswhentheaddressonthebusisvalid
WRITE* Definesthedirectionofthedatatransfer
DTACK* Dataacknowledge.Usedbyaslavetotellthemasterthatithasread/written
thedata
BERR* Buserror.Usedbyslavesorarbiterstosignalerrors
IRQ1*..IRQ7* Interruptrequestlines.Assertedbytheinterrupter
IACK* Interruptacknowledge.Usedbytheinterrupthandlertoretrieveaninterrupt
vectorfromtheinterrupter 12
 Arbitration
Beforeamastercantransferdataithastorequestthebus.Itdoesthisby
assertingoneofthefourbusrequestlines
Thelines(BR0,BR1,BR2andBR3)canbeusedtoprioritizerequestsinmulti
mastersystems
Thearbiter(usuallyinslot1)knows(bylookingattheBBSYline)ifthebusis
busyoridle.OnceitisidleitassertsoneofthefourBusGrantoutlines
(BGOUT0..3)
Ifamasterdetectsa1ontheBGINlinecorrespondingtoitsBRitclaimsthe
busbyassertingBBSY(otherwiseitpassesBGINontoBGOUTtoclosethe
daisychain)
SlotN SlotN+1
BR*
BGIN

BG* BGOUT

BBSY*

Colorcode:ArbiterMaster

13
 Arbitration(2)
Thearbitercanusedifferentschemes:PRI(priority
based),RRS(roundrobin)
Notanissueforsinglemastersystems
Iftwomastersusethesamebusrequestleveltheone
closertoslot1inherentlyhasahigherpriority(becauseit
detectsBGINfirst)
Modernmasterssupportfairarbitration.I.e.theydelay
theirbusrequestifothermastersarerequestingthebusat
thesamelevel
AmastermaygetstuckiftheBGdaisychainisnotclosed

14
 Addressing
TheVMEbusbackplanehas31addresslines:A01..A31
ThereisnoA00addresslineonthebackplane.Thisinformationis
encodedintheDS0/1protocollines
Aslaveisselectedbytwocriteria:
Address(usually16,24or32validbits)
Addressmodifier(6bits).Itdefines:
Thenumberofvalidaddressbits
Theaccessmode(user/supervisor,program/data,CR/CSR)
Thetransfertype(singlecycleorblocktransfer)
Typicallyslavesrespondtoonlyoneaddresswidth(A16,A24orA32;
readthemanualoftheslave)butmayallowbothsinglecyclesandblock
transfers
Thebaseaddressofaslavecanbeset:
Mechanically:onboardJumpers,DIPswitches
ByS/W:VME64xgeographicaladdressing,CR/CSR

15
 Addressingprotocol
FirstthemasterdrivesAM,AddressandLWORD*.Thenitwaits35nsand
finallydrivesAS*tovalidatetheinformation
Theslavehastodecodetheaddressinformationwithin40ns(eventhough
mostmasterskeepAS*assertedmuchlonger)
Themasterdoesnotknowifaslavehasacceptedtheaddressinformation.It
continueswiththedatatransferuntiliteitherreceivesaDTACK*oraBERR*
Iftwoormoreslavesbelievetobeaddressedyouhaveaproblem
Thetimingparameters
mentionedherearetwoof
about50intheVMEbus
AM[5..0] invalid valid standard.Thestandard
LWORD* alsodistinguishesmaster
invalid valid andslavetiming(bus
A[31..1] invalid valid skew)
AS*
35ns 40ns
Colorcode:Master
16
 Singlecycles
Example:(Simplified)writecycle

BR*
Arbitration
BG*

AS*
1:MasterdrivesaddressandAM
1 6
code.ThenitassertsAS
Address/AM undefined defined undefined 2:Masterputsdataonthebus.Thenit
assertsDS
Data undefined defined undefined 3:Slavelatchesdataanddrives
2 DTACK
DS* 4:MasterremovesDS
3 5 5:SlaveremovesDTACK
4
DTACK* 6:MasterreleasesAddress,AMand
datalines.ThenitreleasesAS
BERR*

Colorcode:MasterSlaveArbiter
17
 Singlecycles(2)
Thenumberofbytestobetransferred(1,2or4)isencodedinthe
DS0,DS1andLWORDprotocollines
Rememberthatsomeslavessupportonlycertaindatawidths(e.g.D8
andD16butnotD32)
TheVMEbusaddressshouldbealignedtothedatasize
ReadingaD32worde.g.fromaddress0x000003maynotbeagoodidea
VMEbusalsosupports(rarelyused)readmodifywritecycles(useful
forsemaphores)
RememberthatVMEbusisbigendian.Example:

Address Action Result

0x00000000 D32write0x11223344
0x00000000 D32read 0x11223344
0x00000000 D8read 0x11
0x00000003 D8read 0x44
18
Example:D32write Blocktransfers
AS

Address/AM undefined defined undefined

undefined
Data defined defined defined defined undefined

DS

DTACK

TheBlocktransferprotocolisbasedonthesinglecycleprotocol
Theaddresslinesonthebackplanedonotchangestateduringthetransfer.Both
masterandslaveuseinternalcounterstokeeptrackoftheaddress
Astheaddresslinesarenotusedtheycancarrydata:64bitmultiplexedDMA.
InthiscasetheslaveusesDTACKfortwopurposes:
DirectlyafterASbeingassertedtoacknowledgetheaddress
AftereachassertionofDStoacknowledgethedata

Colorcode:Singlecycleprotocolblocktransfer
19
 Blocktransfers(2)
Amastermustnotcrossa256bytes(D32)or2048bytes(D64)address
boundaryrespectivelywithoutreleasingAS(transparenttotheuser)
Thisistogiveothermastersachancetoacquirethebusbeforetoolong
ReadingoutsingleaddressFIFOsisnotforeseenbythestandardandrequires
specialmasters
DesigningaslavethatterminatesablocktransferfromaFIFOwithabus
errorislegalbutbadpractice.ItdoesnotalwaysworkwiththeTundra
Universe
VMEbusinterfacechipsmayrequirearelativealignmentoftheVMEbusand
local(PCI)addresses
IncaseoftheTundraUniversetheVMEbusandPCIaddressesmustbe8byte
alignedwithrespecttoeachother
Contiguousbuffers
Memoryobtainedwithmalloc()maybefragmented.MostDMAcontrollers,
however,needcontiguousbuffers
Contiguousbuffersareprovidedbyspecialdrivers(e.g.ATLAS:cmem_rcc)
basedonkernelfunctions(get_free_pages)orextensionsoftheLinuxkernel
(BigPhysArea)

20
 VMEbustypicalperformance
Beingahandshaked,asynchronousprotocolthereisnofixedtransferratesuch
asfore.g.RS232.Thetimingparameters(seeVMEbusstandard)howeverset
anupperlimit.
Singlecycles:Typicalperformance=1spertransfer
D8=1MB/s
D16=2MB/s
D32=4MB/s
WritepostingdecouplesPCIandVMEbuscycle.Thisincreasesthe
performanceto~10MB/sforD32
Blocktransfers
D32=20..25MB/s(theoretical:40MB/s)
D64=40..50MB/s(theoretical:80MB/s)

21
 Buserrors
InVMEbuserrorscanoccurundertwoconditions
Aslavehasbeenaddressedbutisincapableofperformingtherequestedtransfer.In
thiscasetheBERRsignalisissuedbytheslaveandreachesthemasterwithinafew
s.
Themasterhasissuedanaddressthatnoslaverecognizes.Suchcyclesget
terminatedbythebusmonitor(arbiter)byassertingBERRafteraprogrammable
delay(typicalvaluesare16or256s)
ThereisnostandardwayforthedeliveryofaBERRfromtheVMEbus
interfacetotheCPUofthemaster
OnPowerPCsBERRistypicallyconverteddirectlytoanonmaskableinterruptand
thenconvertedtotheSIGBUSsignalbytheoperatingsystem
CertainIntelbasedSBCignorebuserrors
OtherIntelbasedSBCsconvertittoaregularPCIinterrupt.Thisinterruptis
typicallyhandledbytheVMEbusdriverandconvertedtotheSIGBUSsignal

22
 Interrupts
VMEbusprovides7interruptlevels(=buslines)toprioritizeinterrupts
Eachinterruptercanuseanylevel
Theremustonlybeoneinterrupthandlerforeachlevel
Theinterrupthandleruses(underH/Wcontrol)aspecialtypeofsinglecycle(IACK
cycle)toobtainan8bitvectorfromtheinterrupter.Thisvector(setbyjumpersor
S/W)mustbeunique(withinthecrate)andidentifiesthesourceoftheinterrupt
Therearetwotypesofinterrupters:
ROAK(preferred)
TheIACKcycleclearstheinterrupt
RORA
Theinterruptisclearedbyanadditionalregisteraccess(singlereadorwritecycle)
TypicallyaninterruptgetshandledbytheH/Winafews(oncetheVMEbusisfree).
Therecanbeadditional(possiblylarge)S/Woverheadsdependingontheoperating
systemusedandthestateoftheCPU
Iftwointerruptersareactiveatthesametimeandonthesameleveltheonecloserto
slot1willbeservicedfirst(IACKdaisychain)

SlotN SlotN+1

IACKIN

IACKOUT

23
 VME64x
VME64xisasetofextensionstotheVMEbusstandardmadein1997
Mostfeaturesareoptionalandfallintooneoffourcategories:
Mechanics
5rowP1/J1andP2/J2connectors
J0/P0connector
Alignmentpin
EMCgaskets
Injector/extractorhandles
Dischargestrips
Cardkeys
Soldersidecovers
Plugandplay
Geographicaladdressing(accessamodulebyitsslotnumber)
CR/CSRspace:Standardisedregistersfortheautomaticconfigurationofamodule(base
address(es),interruptvector(s),etc.)
Power
3.3Vand48V
Additional5V
2eVMEProtocol:Ararelyusedwayofspeedingupblocktransfers(theoretical
bandwidth:160MB/s)

24
 CR/CSRspaceaccessandgeographical
addressing
ClassicVMEbusslavesuseonboardjumpersorswitchesforthe
initializationofthebaseaddressandtheinterruptvectors
TheVME64(x)standardproposesaS/Wbasedmechanism(plugandplay)
Thebasicprinciplesare:
Eachslavehasaspecialwindowof512kBconsistingofaConfigurationROM
(CR)andaControlandStatusRegister(CSR)section
AccesstothiswindowisinA24modewithAM=0x2fonly
Theaddressofthatwindow(BAR)iseithersetbyjumpers(VME64)orderived
fromtheslotnumber(geographicaladdressing,VME64x)withtheformula:
address=slot#*0x80000
TheCR/CSRspacecontainsmany(mostlyoptional)featurestospecifyand
controlthefunctionsofaslaveboard
Slaveboardsareidentifiedbyamanufacturer+boardIDstoredintheCR.
BoardIDshavetobeunique(http://cern.ch/boardid)
ThemostimportantCSRspaceregistersaretheeightADERregisters.Theyare
usedtodefinethebaseaddress(es)ofthemainfunction(s)oftheslave.

25
 2eSST
2eSST=2edgeSourceSynchronousTransfer
Arecent(1999)additiontotheVME64xstandard
ItdefinesasynchronousprotocolforVMEbusblocktransfers
Threetransferspeedsaredefined:160,267and320MB/s(8bytes@
20,33.3and40MHz)
Allowsfordatabroadcastandmulticast
Worksonlyreliablyonhighqualitybackplanes(incidentwave
switching)andspecial(TexasInstruments)driverchips
Thereexist(sofar)onlyahandfullofVMEbusmodulesbuilttothis
standard

26
 TheVMEbussingleboardcomputer
Usuallythisistheonlymasterandinterrupthandlerinthecrate
Itoftenalsoprovidesthearbiterfunctionality(andshouldthereforebe
installedinslot1,despitewhatissaidaboutcooling)
ItbehaveslikeanormalPC
Operatingsystem:Linux,(LynxOS,Windows)
Developmenttools:gcc,g++,gdb
Environment:Shell,Xterm,vi,emacs
Accessedvia:RS232,Ethernet,(VGA)
ItinterfacestoVMEbusviaaPCIdevice
TypicallyTundraUniverse
DependingonthemodelandtheS/WusedithastobeconfiguredintheBIOSorat
startupbyspecialprograms
ThePowerPCCPUdoesinstructionreordering.Usetheeieioassembler
instructiontoenforcetheproperorderofexecution
SomeSBCscanbeequippedwithmezzanines(PMC,IP)butthisisanother
story

27
 TheVP110SBC
ThisSBC(manufacturedbyConcurrentTechnologies)isthedefaultSBCfor
ATLASRODcrates
RecentlyALICEhasdecidedtouseitformostVMEbussystems
YoucanrentthisprocessorfromtheEl.Pool

CPU PentiumIII@800MHz
RAM 512MB
VMEbusinterface TundraUniverse
PMCsites Two.32or64bit,33or66MHz,(canbeconfiguredfor5Vor3.3Vsignaling)

Massstorage ConnectionofIDEharddisksandfloppydrivespossibleviaP2adapterboard
(requires5rowVME64xbackplane)Aharddiskcanalsobeinstalledon
board.ThistakesoneofthePMCslots
Networkinterface Twochannels,10/100Mbit/s,RJ45onfrontpanel(basedonthe82559ER
interfacechip)
Mechanics VME64xcompliant:5rowP1andP2,P0(optional),frontpanelwith
alignmentpinandinjector/extractorhandles(alternativesolutionfor3row
VMEbackplanesexists),soldersidecover
Terminalconnection RS232viafrontpanelRJ45connector,noVGA/mouse/keyboard(canbe
addedbyadditionalPMCmodule)
28
 OtherVMEbusmastersandinterfaces
PCtoVMEbusinterfaces
Availablefromseveralmanufacturers
AsettypicallyconsistsofaPCIcard,aVMEbuscardandacable(copperor
opticalfiber)
Example:NationalInstrumentsMXI2
AllowstocontrolseveralVMEbuscratesfromonePC
LibrarysupportforLabViewandstandardC
VMEbusrepeaters
Allowsamasterincrate1toaccessaslaveincrate2
AsetconsistsoftwoVMEbuscardsandacable
Thereisusuallyaperformancepenalty
VMEbustoCAMACinterface
AllowsamasterinaVMEbuscratetocontrolaCAMACcrate
AsetconsistsofaVMEbusslave,aCAMACcratecontrollerandacable
YoucangetsuchinterfacesformtheCERNEl.Pool

29
 Subsystembuses
TheP0,P2andP3connectorshaveanumberofuserdefinedpins.Theycanbe
usedtoimplementspecializedcommunicationchannelsindependentlyfromthe
VMEbusprotocol.Examples:

VSB(VMESubsystemBus)
Obsolete.
Providesa32bitbusforupto8adjacentcards.
WasusedtointerfacetoFASTBUS
VXS
Arecentadditiontothestandard.
ItallowseachVMEbuscardtoconnecttoaswitchfabric.
InitiallyitusestheEthernetprotocolbutothertechnologies(e.g.Infiniband)
arebeingdiscussed
CustomP3
In9UcratestheP3istotallyuserdefined.
Specialbackplanesarepossibletoo

30
 Systemintegration
Findtherightcrateforyourmodules
J0/Jauxincompatibility
VME64x(alignmentpin,geographicaladdressing)
FindoutifyourcratestillhasBG/IACKjumpers
Rule:Eachslotmustbeequippedwith1cardor5jumpers
Attention:JumpersmaybeoneithersideoftheJ1connectordependingonbackplanetype
Cardhandlingandinsertion
VMEbuscardscanbesensitivetoelectrostaticdischarge.Takeprecautions
Neveraddorremoveacardifthecrateisswitchedon
Dependingonthetypeofmoduletheinsertionforceisbetween20and50kg.Checktwice
thatthecardreallyhasbeeninsertedproperly!!
DonottrustLEDsonthefrontpanel.Oncertain(VME64x)cardsthepowerpinsarelonger
thantheprotocolpins.
Cooling
AvoidinstallingCPUsintheleftmostorrightmostslot(therearespecialarbitermodules)
Leaveoneortwoslotsemptybetweencards,ifpossible
Closethefrontofthecratewithblindpanels
Checkthefanspeed
Useprograms(e.g.thermo)toreadthetemperatureoftheSBC
Addresslayout
Checkthattheaddresswindowsoftheslavemodulesdonotoverlap
Trytomapsimilarslaves(e.g.A32,A24)toconsecutiveaddresses
31
 VMEbusS/W
(Linux)Drivers
In(almost)allcasesaccesstotheVMEbusisviaadevicedriver
ThedriverallowstousetheVMEbusinmultiprocessingenvironments
Insomedriverstheread()/write()functionsareusedtoaccesstheVMEbus
Mosttransactionsareimplementedbyioctl()functions
Interruptsarehandledbythedriverandsignalledtotheuserapplicatione.g.
bymeansofsignalsorsemaphores
ThedriverstypicallyprovideDMArequestlists.Ablocktransfermay
thereforenottakeplaceimmediatelybutbedelayedbyotherDMArequests
Thedevicenodes(/dev/XYZ)createdbydifferentdriversarenotstandardized
Libraries
Thedriverisnotuseddirectlybytheapplicationbutviaauserlibrary
ThereisnostandardAPI(despiteVISION)forsuchlibraries
TheAPIofthelibrarydevelopedforATLAScanbefoundat:
https://edms.cern.ch/document/325729/4

32
 Debuggingtools
(Linux)S/W
Lookatthe/procfileofthedriver
IntheATLASpackageyoufindspecialapplications
scanvme:ScanVMEbusformodules
vme_rcc_test:Usethefunctionsofthelibraryinteractively.Thisprogramisalsoa
goodprogrammingexample
cctscope:DecodeanddumptheconfigurationoftheUniversechip(andsomeother
VMEbusrelatedresources)inhumanreadableform

cctscopeexampleoutputoffunction2/2:
=======================================================================================
LSIVMEaddressrangePCIaddressrangeENWPVDWVASAMTypePCIspace
0000000001000000090000000a0000000YesNoD32A32UDSCPCIMEM
10000000001000000a0000000a1000000YesNoD32A24UDSCPCIMEM
20000000000010000a1000000a1010000YesNoD32A16UDSCPCIMEM
30000000001000000a2000000a3000000YesNoD32CR/CSRUDSCPCIMEM
400000000ffffffff00000000ffffffffNoNoD32A32UDSCPCIMEM
500000000000000000000000000000000NoNoD32A32UDSCPCIMEM
600000000000000000000000000000000NoNoD32A32UDSCPCIMEM
700000000000000000000000000000000NoNoD32A32UDSCPCIMEM
=======================================================================================

33
 Debuggingtools(2)
H/W
VMEtroVBT325busanalyzer
Storesupto16000VMEbuscycles
Powerfultriggerandsequencer
Supportsprotocolanalysis
TooperateityouneedaVT100(Falco)terminaloraPCwitha
terminalprogram(e.g.HypeTerm,minicom,kermit)
CanberentedattheEl.Pool
CESVMDIS8004
Lowcostbusmonitor.Displaysthemostrecentcycle
Canlatchthefirstcyclewithabuserrororaninterrupt
Hasabuiltinarbiter(usefulifSBCrunshotinslot1)
CanberentedattheEl.Pool

34
 Thevme_rccpackage
Thevme_rccpackagecontainsthedefaultVMEbusdriverand
libraryforATLASReadOutCrateControllers(RCC)
Documentation:https://edms.cern.ch/document/325729/4
Incanofcoursealsobeusedforotherprojects
SupportedH/W:ConcurrentTechnologiesVP110aswellas
VPPMC,VPCP1,etc.PortingtootherUniversebasedSBCs
possible.
Thesourcecodeofboththedriverandthelibraryhasbeenfully
developedatCERNandtestedonLinuxkernelsupto2.4.18
WhynotusinganexistingVMEbusdriverinsteadof
developinganewone?
WewantedtohaveasufficientlygenericAPIthatcouldeasilybe
implementedforanyVMEbusinterfaceonaSBC
MostdriversfortheUniversechiplacksupportforsomefeatures
Externalcodeisnotnecessarilyoptimizedforourtypeofapplications
35
 TheUniversechip
TheAPIusedinthevme_rccpackageisgenericenoughtofit
(almost)anytypeofVMEbusinterface.Sofarthereexistsonlyan
implementationfortheUniversechip
TheTundraUniversechipisusedonmostcurrentSBCsbuthasa
numberoflimitations:
Thereareonly8mapdecodersformasterandslavepagesrespectively
Insystemswithmorethan8slavesonehastouseonedecoderforseveral
slaves.I.e.slaveshavetobegrouped>slavebaseaddress
ItisnotpossibletoexecuteblocktransferswithaconstantVMEbus
address
IfyouaredesigningVMEbusslaves:DonotimplementsingleaddressFIFOs
forthereadoutofinternalmemory
DatacanbelostifablocktransferisterminatedwithaBERR
SomeBERRs(postedwrite)aredifficulttocatch.TheVP110hasextra
logictocopewiththat
ThereisnoH/Wbyteswapping(requiredonlittleendianCPUs).The
VP110hasextralogicforthatpurpose
36
 vme_rcc:Overview
Thepackageprovides:
AVMEbusdriver
TobedynamicallyinstalledintheLinuxkernel
Youneedthecmem_rccandio_rccdriversaswell
IfyouarenotusingthestandardCERNkernelsyoumayhaveto
compilethedriversyourself
Alibrary
Thereareabout60functions.Somedetailswillfollow
Utilityprograms
vmeconfig
cctscope
DumptheregisteroftheUniversechipandsomeotherresourcesof
theVP110inhumanreadableform
scanvme
ScantheVMEbusforslavecardsatunknownaddresses
37
 Systeminitialization
BeforethefirstVMEbuscyclecanbemadeonehastoprogram
(atleast)oneofthe8mapdecodeswithappropriateparameters
ThisbasicallymeansmappingarangeofPCIaddresses(MEM
space)toanequallylargewindowofVMEbusaddresses
Inmanydriversthiscanbedonedynamicallyatruntimeviaa
functioncall(buttheVMEbusandPCIbaseaddressesstillhave
tobeprovidedbytheuser)
vme_rccisdifferent.Thelibrarycannotmodifythesetupofthe
mapdecoders.Thisisthejobofvmeconfig
Usingastaticsetuphastheadvantagethatonecannotrunoutofmap
decodersinthemiddleofanapplication
Thispolicyenforcessomedisciplineandisthereforenotlikedby
everybody.Weare,however,convincedthatithelpstoreduceproblems

38
 Addressspaces
CPUaddresses VMEaddresses
0x0 I/Ospace 0x0
UniverseASIC
Examples: memory
0x1fffffff 0x0fffffff
0x80000000

0x8fffffff A32(4GB)
PCI
0xa0000000
0xa0ffffff

0xb0000000 A24(16MB)
0xb000ffff

4GB
A16(64kB)

39
 vmeconfig
Ifcalledintheformvmeconfigavmetabitloadsausersetupintothe
Universechip.Thistypicallyhappensautomaticallyatboottime
Calledasvmeconfigivmetabitallowsyoutoedittheconfigurationfile
(vmetab)
Theconfigurationfileisabinary.Itcannotbemodifiedwithatexteditor
The(mostimportant)userparametersare:
Masterandslavemapping
Interrupts
Byteswapping
Arbitrationandbusrequestmodes
Inordertorunvmeconfigyouneedanumberofdynamiclibrariesfromthe
ATLASDFrelease.Usethecommandldd vmeconfigtocheck
Havealookattheonlinehelpinvmeconfig

40
 vmeconfig(2)
Example:Howtosetupamastermapping
Useoption4toprogramamapdecoder.vmeconfigwillaskyoutoenteranumberof
parameters:
Enter number of map decoder <0..7> [0] :
Enable map decoder <0=no 1=yes> [1] :
Select VMEbus base address [0x00000000] :
Select PCI base address [0x90000000] :
Select the window size (bytes) [0x00010000] :
Select Write posting <0=no 1=yes> [0] :
Select address space
<0=A16, 1=A24, 2=A32, 5=CR/CSR, 6=USER1, 7=USER2> [0] :
Select cycle type <0=User, 1=Supervisor> [0] :
Select cycle type <0=Data, 1=Program> [0] :

DefiningthePCIbaseaddressistheartisticpartbecauseyouhavetoguessit.Foran
educatedguessusetheseguidelines:
DRAMaddressesgrowfrom0x0upwards
PCIdevicesaremappedbythekernelfrom0xffffffffdownwards
Addressesintherange0x500000000xbfffffffshouldbesafe
/sbin/lspci -vmaybehelpful
WhenyouaredoneuploadtheconfigurationintotheUniverseandsaveyourchanges.Itis
recommendedtocheckthenewsetupwithcctscope(function2/2)forerrorslikeaddress
overlaps 41
 Thelibraryofthevme_rccpackage
Therearefourmajorgroupsoffunctions:
Singlecycles
Blocktransfers
Interrupts
Servicefunctions(includingbuserrors)
Presentingallfunctionsherewouldtaketoolong.The
programvme_rcc_test.cppshowshowthedifferent
functionsaretobeused
Allfunctionsreturnerrorcodesintheformatdefinedin
thercc_errorpackage
ThereexistsaC++wrapperinaseparatepackage
(RCDVme)

42
 Asimpleprogramdoingsinglecycles
Letshavealookataverysimpleprogramexecutingasingleread
cycle.ForthispurposeIassumethatthereisaVMEbusD32/A32
slaveataddress0x02000000withatotalsizeof4Kbandareadable
registeratoffset0x80
#include "rcc_error/rcc_error.h"
#include "vme_rcc.h" DeclareVMEbus
pointersvolatileto
int main(void)
{ avoidproblems
VME_MasterMap_t master_map; withcode
volatile u_int *lptr, ldata;
u_int ret, vbase; optimization
int handle;

ret = VME_Open();
Nevercallafunction
if (ret != VME_SUCCESS) withoutcheckingfor
{
VME_ErrorPrint(ret);
errors!!!
exit(-1);
}

43
 Asimpleprogram(2)
master_map.vmebus_address = 0x02000000;
master_map.window_size = 0x1000;
master_map.address_modifier = VME_A32;
master_map.options = 0;
ret = VME_MasterMap(&master_map, &handle); Createamastermapping.
if (ret != VME_SUCCESS) Remember:Your
{
VME_ErrorPrint(ret);
vmetabmustsupport
exit(-1); theseparameters
}

ret = VME_MasterMapVirtualAddress(handle, &vbase);

if (ret != VME_SUCCESS)
Getthevirtualaddressforfastaccess.
{
VME_ErrorPrint(ret); Alternativelyyoucouldusethesafe(but
exit(-1); slow)functionsoftheAPI
}
Castthegenericpointertoa
lptr = (u_int *)(vbase + 0x80); 32bitdatatypeandaddthe
ldata = *lptr;
registeroffset

ExecutetheVMEbuscycle
44
 Asimpleprogram(3)
ret = VME_MasterUnmap(handle);
if (ret != VME_SUCCESS)
{
VME_ErrorPrint(ret);

}
exit(-1);
Alwayscleanupwhen
ret = VME_Close(); youredone!
if (ret != VME_SUCCESS)
{
VME_ErrorPrint(ret);
exit(-1);
}
}

45
 Interrupts
AVMEbusinterrupt(identifiedbyitsunique8bitvector)canbeconvertedbythe
librarytoeitherasignalorasemaphore
Itispossibletolinkseveralinterrupts(ofthesametype)toonesignalorsemaphore
ItisnotpossibletoservicebothRORAandROAKinterruptsonthesameinterrupt
level
IfyouareusingRORAinterruptsyouhavetoreenabletherespectiveinterruptlevel
aftereachinterrupt
Remember:beforeyoucanuseaninterruptlevelyouhavetoenableitwithvmeconfig

irq_list.list_of_items[i].vector = 0x77;
irq_list.list_of_items[i].level = 5;
irq_list.list_of_items[i].type = VME_INT_ROAK;
signum = 42;

ret = VME_InterruptLink(&irq_list, &int_handle);

ret = VME_InterruptWait(int_handle, timeout, &ir_info);
ret = VME_InterruptRegisterSignal(int_handle, signum);
ret = VME_InterruptUnlink(int_handle);

46
 Blocktransfers
Blocktransferscanonlybemadetophysicallycontiguousmemorybuffers(>
cmem_rcc).Usingmemoryallocatedbymalloc()wouldtechnicallybepossiblebut
requiredadditionalcodeinthedrivertolockandchainthepagesandtherewould
alsobeaperformancepenalty
Supportedmodesare:A24D32,A32D32andA32D64andsinglecycleDMA
TheVMEbusandPCIaddresseshavetobe8bytealignedwithrespecttoeachother
(alsoforD32)
ThelibrarysupportschainedDMA
Blocktransfersareindependentofthemastermapdecoders
ThedrivercanmanagemultipleDMArequestsfromseveralprocesses.Itis
thereforepossiblethatatransferdoesnotstartimmediately

blist.list_of_items[0].vmebus_address = 0x10000000;
blist.list_of_items[0].system_iobus_address = 0x24000000; //PCI MEM space ->CMEM_RCC
blist.list_of_items[0].size_requested = 0x1000;
blist.list_of_items[0].control_word = VME_DMA_D32W; //Implies A32
time_out = 100;
ret = VME_BlockTransfer(&blist, time_out);
if (ret == VME_DMAERR) {
printf("Status: %d\n", blist.list_of_items[0].status_word);
printf("Bytes remaining: %d\n", blist.list_of_items[0].size_remaining);
}
47
 Theoddstuff
vme_rccofferssupportfor
CR/CSRspaceread/write(onlyD8)
UserdefinedAMcodes
SYSFAILinterrupts
Interruptgeneration
Supervisor/programAMcodes
ConstantaddressDMAinsinglecyclemode
Fullbuserrordetection
vme_rccdoesnotsupport
ReadModifyWritecycles
Addressonlycycles
ACFAILinterrupt
AnumberofotherexoticfeaturesoftheUniversechipforwhichnobody
hasrequestedsupportsofar

48
 Servicepackages
cmem_rcc
Driverandlibraryfortheallocationofcontiguousmemory(e.g.
forDMA)eitherviatheget_free_pages()kernelfunctionorthe
BigPhysAreapatch
Usedbysomeofthetestprogramsinthevme_rccpackage
io_rcc
DriverandlibraryfortheaccesstoPCIandPCI/Oregistersfrom
usercode
Usedbysomeofthetestprogramsinthevme_rccpackage
rcc_error
Asimplelibraryforerrorreporting

49
 Linksforfurtherinformation
VMEbusstandard
http://atlas.web.cern.ch/Atlas/GROUPS/FRONTEND/VMEbus/index.html
AtlasS/W
http://atdaqsw.cern.ch/cgibin/viewcvs.cgi/DAQ/DataFlow/vme_rcc/
https://edms.cern.ch/document/325729/4
https://edms.cern.ch/document/349680/2
https://edms.cern.ch/document/336290/2
https://edms.cern.ch/file/325729/4/wrapper.pdf
AlternativeVMEbusdriversfortheUniversechip
http://www.vmelinux.org/
VMEbusmarketoverview
http://www.vita.com
VP110SingleBoardComputer
http://www.gocct.com/sheets/vp11001x.htm
CERNElectronicsPool
http://ess.web.cern.ch/ESS/electronicsPool.htm

50
TheEnd

51
 UseofpointerstogenerateVMEbuscycles
unsignedintui_data,*ui_ptr,virtual_address;
unsignedshortus_dat,*us_ptr;
unsignedcharuc_data,*uc_ptr;

Main()
{
virtual_address=Map_VME_module(physical_address,AMcode,);//Hypotheticalfunction

ui_ptr=(unsignedint*)virtual_address;
us_ptr=(unsignedshort*)virtual_address;
uc_ptr=(unsignedchar*)virtual_address;

ui_data=*ui_ptr;//D32read
*ui_ptr=ui_data;//D32write
us_data=*us_ptr;//D16read
*us_ptr=us_data;//D16write
uc_data=*uc_ptr;//D8read
*uc_ptr=uc_data;//D8write

ui_data=ui_ptr[0];//equivalentto*ui_ptr;
ui_data=ui_ptr[4];//ReadD32atoffset0x10(4*4bytes)
uc_data=uc_ptr[4];//ReadD8atoffset0x4(4*1byte)
}

52
 Signalhandling
#include<signal.h>
//Prototypes
voidSigBusHandler(intsignum);

main
{
structsigactionsa2;

sigemptyset(&sa2.sa_mask);
sa2.sa_flags=0;
sa2.sa_handler=SigBusHandler;
stat=sigaction(SIGBUS,&sa2,NULL);
if(stat<0)
{
printf("CannotinstallSIGBUShandler(error=%d)\n",stat);
exit(1);
}
}

voidSigBusHandler(intsignum)
{
printf(Buserrorreceived\n);
}

53
 Managingdrivers
Ismydriverloaded?
/sbin/lsmod
Inwhatstateismydriver?
more/proc/<name>(valueofnamedependsonthedriverused.
ATLAS:vme_rcc,cmem_rccorio_rcc)
Isthedrivercurrentlyused
/sbin/lsmod
ChecktheUsedbynumberinthethirdcolumn.
Howtorestartadriver
Usedbyhastobe0
su(thenenterrootpassword)
cd/etc/rc.d/init.d
<name>restart(e.g.vme_rcc restart)

54
(Common)VMEbusAMcodes
AMcode Description
0x08 A32,user,64bit(MBLT)blocktransfer
0x09 A32,user,data,singlecycle
0x0A A32,user,program,singlecycle
0x0B A32,user,32bit(BLT)blocktransfer
0x0C A32,supervisor,64bit(MBLT)blocktransfer
0x0D A32,supervisor,data,singlecycle
0x0E A32,supervisor,program,singlecycle
0x0F A32,supervisor,32bit(BLT)blocktransfer
0x29 A16,user,data,singlecycle
0x2C A16,supervisor,data,singlecycle
0x2F CR/CSRsinglecycle(geographicaladdressing)
0x38 A24,user,64bit(MBLT)blocktransfer
0x39 A24,user,data,singlecycle
0x3A A24,user,program,singlecycle
0x3B A24,user,32bit(BLT)blocktransfer
0x3C A24,supervisor,64bit(MBLT)blocktransfer
0x3D A24,supervisor,data,singlecycle
0x3E A24,supervisor,program,singlecycle
0x3F A24,supervisor,32bit(BLT)blocktransfer
55
 Otherinformation
FortheATLASVMEbuslibrarythereisaC++wrapper
https://edms.cern.ch/file/325729/4/wrapper.pdf

56
 Glossary
ACFAIL:AlineontheVMEbusbackplanedrivenbythepowersupply.Ifassertedthe+5Vpowerwillbeavailable
foratleastanother4msandthendropbelow4.875V
BBSY:Theprotocollinethatindicatesifthebusisbeingused(BusBusy)
BERR:Theprotocollinethatsignalsabuserror
BG0..3:Protocollinesusedbythearbitertograntthebustoamaster
BR0..3:Protocollinesusedbymasterstorequestthebus
CR/CSR:ConfigurationROM/ControlandStatusRegisters,afeatureofVME64(x)slavecardsfortheplugandplay
configurationoftheonboardfunctions
Daisychain:SomeofthesignallinesofaVMEbusbackplanearenotbussedbutconnectonlytwoadjacentslots.In
ordertopassasignalfromslotNtoslotN+mtheVMEbusmodulesinbetweentheseslotshavetopassthesignal
fromtheinputsidetotheoutputside.Incaseofanemptyslottheconnectionhastobemademechanically
(jumper)orautomatic(specialbackplane)
DMA:DirectMemoryAccess(blocktransfers)
EMC:ElectroMagneticCompatibility:
IACK:Protocollineusedfortheinterrupthandshake
J0,J1,J2,J3:Thefemalejacks(connectors)ontheVMEbusbackplane
Jaux:AspecialconnectorsittingbetweenJ1andJ2onsomebackplanesusedatCERN.Requiredforcertainfrontend
modulesbutincompatiblewiththeP0connectorofVME64x
P0,P1,P2,P3:ThemaleplugsonVMEbuscardsconnectingtothebackplane
ROAK:ReleaseOnAKnowledge,AtypeofVMEbusinterrupterthatclearstheinterruptinresponsetotheIACK
cycle
RORA:ReleaseOnRegisterAccess,AtypeofVMEbusinterrupterthatrequiresaspecialinterventionfromthe
mastertoclearaninterrupt
SBC:SingleBoardComputer
SYSFAIL:AVMEbuslinethatindicatesaproblemwithonecard.SYSFAILcanbemonitorede.g.byanSBCwhere
itwouldbeconvertedtoaninterrupt
WritePosting:Awayofspeedingupsinglewritecycles.ThePCIcyclegetsacknowledgedbeforetheVMEbuscycle
completes.Thisdecouplingofthebussesincreasesthespeedbutcancomplicatethedetectionofbuserrors
57
TheTundraUniverseIIASIC

BridgesVMEbusto
PCI
Usedonmostofthe
commerciallyavailable
VMEbusprocessors

58