You are on page 1of 53

Tile ACR training

June 2011
Intended for regular training on Thursdays,
after Tile Operations Weekly Meeting
1 Friday, July 29, 2011
Material we will cover

How to start your shift

Arrival time and Tile desk
Interaction with previous shifter and shift leader
Applications to open and sign on

Tile busy
Tile trips and status
Data quality: digital errors


How to write useful e-log entries

When to call the experts

Other information
DAQ = Data AcQuisition
DQM = Data Quality Monitoring
DCS = Detector Control System
2 Friday, July 29, 2011

What to check before taking shifts (I)
To access the Atlas Control Room (ACR) you need :
Safety courses 1, 2, 3, 4A
ATLAS Control room ATL_CR
Check your access detail sheet on EDH
It should look like :
More informations on access :
For newcomers : check
your access rights at least
2 weeks before your shift!
Request your access via EDH. More information in this twiki
For newcomers: check
your access rights at
least 2 weeks before
your shift!
3 Friday, July 29, 2011
Exercise Self-training

Minimum of 2 shadow shifts, where you ask to be

shown how things are done.
Take shadow during the morning shift: ~9am
What to do when Tile is busy
What to do after a module trips

This presentation is a subset of what you should read

4 Friday, July 29, 2011
Start of shift
You sit here
He controls ATLAS
Shift Leader:
Ask him/her about
the plan for today
He knows about
L1Calo trigger
Arrive 10 min before your
shift starts, we HAVE to talk
to the previous shifter
5 Friday, July 29, 2011
Start of shift

Ask the previous Tile Shifter for a brieng and the Shift Leader for the plan of
the day

Log-in to RunCom.

Open (if not already open) the following:

GeneralATLAS Logbook. (Login as yourself.)
TileCalDCSTile DCS Alarms
TileCalL1Calo Map
6 Friday, July 29, 2011
7 Friday, July 29, 2011
FSM panel
Clickable / Browsable!
8 Friday, July 29, 2011
DCS Alarm Panel

Voltage trips

Temperature alarms

Yellow = warning.

Often some value ickers in/out of legal range.

Browse the source of the warning in FSM. Copy-paste it to the e-log.

(Screenshot can be taken using snapshot linux tool)

Red = Alarm!
9 Friday, July 29, 2011
L1Calo Map
Tile Modules
Tile Modules/
without signal
A side C side
10 Friday, July 29, 2011
L1 Calo Map
C side A side
11 Friday, July 29, 2011

Its an environment, appropriately congured, in which data are

taken. It has a name, and it uses some parts of ATLAS.

Partitions can exist in parallel, as long as they dont try to use the
same part of the detector.
TILE partition: We use it for calibrations. You control it!
ATLAS partition: Run Manager uses this for collisions,
cosmics, or any other combined runs. You only watch.
Examples: ATLAS partition, TILE partition.
12 Friday, July 29, 2011
Monitor/Start Partition
DAQPanel: Set partition info & laun
1. Congure, from whiteboard.
2. Read Info.
3. Launch the panel you need.
13 Friday, July 29, 2011
How to monitor ATLAS partition?

Congure DAQPanel according to parameters in the whiteboard.

Click Monitor Partition.

14 Friday, July 29, 2011
The TDAQ software (IGUI)
15 Friday, July 29, 2011
Look out for...

Load Panels TileIguiPanel to see how Tile is doing.

Is Tile well-congured? That will depend, but typically for physics

we use Reconstruction, and not Raw Data, especially if the L1
event rate is over ~30-40 kHz.
for stable beams
16 Friday, July 29, 2011
Look out for...

17 Friday, July 29, 2011
Look out for...


Is Tile BUSY? Serious incident. See:
These would be red.
18 Friday, July 29, 2011
Tile is busy
19 Friday, July 29, 2011
Look out for...

Is Tile BUSY? Serious incident. See:
Some ROD(s) would be red,
similar to this.
Please re-load this
panel if all crates are in
RED and no busy is
shown in the BUSY
20 Friday, July 29, 2011
Ritmo Panel
21 Friday, July 29, 2011
Re-congure a module
22 Friday, July 29, 2011
Data Quality Monitor
Online Histogram Presenter

Digital errors
Occupancy plots
Noisy channels
Masked channels
Known problems (in the whiteboard)
23 Friday, July 29, 2011
24 Friday, July 29, 2011
New histogram to monitor digital errors
Last bin is always
current LB
Time when module
was recovered
Fraction of
ANY type of
digital errors
over LB
One histogram per module and run

25 Friday, July 29, 2011
Back - up
5 lumiblocks ago
75 lumiblocks ago
Known problem see
Small percentage,
just report this in the
26 Friday, July 29, 2011
27 Friday, July 29, 2011
28 Friday, July 29, 2011
29 Friday, July 29, 2011
30 Friday, July 29, 2011
31 Friday, July 29, 2011
Detector Control System

High Voltage
Low Voltage Power Supply (LVPS or fLVPS, where f
is for nger )
Power trip!
Alarm Panel
32 Friday, July 29, 2011
FSM panel
Clickable / Browsable!
33 Friday, July 29, 2011
Alarm screen
34 Friday, July 29, 2011
35 Friday, July 29, 2011
36 Friday, July 29, 2011
1rlpped Lv examples
A message ln Alarm panel !"#$%&'"#(()* wlll appear!
uCS Lakes auLomauc acuons and recovers Lv of Lhe Lrlpped modules
- lf module falls Lo sLarL +&,-). consecuuvely, auLomauc acuon ls .'/(()*&
AuLomauc recovery ls only for Lhe Lv.

37 Friday, July 29, 2011

Tile Calibration: TileCal TileCalibration

Pedestal, CISmono, CISscan, laser-,
laser LG, laser HG.

L1Calo: TileCal Level1 L1 Calibration
Manuall Step-by-step instructions:
Check schedule in Whiteboard Check schedule in Whiteboard Check schedule in Whiteboard Check schedule in Whiteboard Check schedule in Whiteboard
L1Calo Tile Calibration TileCalibration L1Calo
38 Friday, July 29, 2011
To take L1Calo calibration
39 Friday, July 29, 2011
To Take Tile Calibration

TileCal TileCalibration
Monitoring is not enable so,
in case histograms need to
be check you should take
manually calibration
40 Friday, July 29, 2011
We use TILE partition for calibrations
Conguration from Whiteboard.
Her we dont just monitor; we START the partition.
Step-by-step in the twiki. Please skim it.
41 Friday, July 29, 2011
To log or not to log?
When to write an e-log:
End of each run.
Include detector and data quality information.
For any significant problem requiring the intervention of an expert.
Power cycle is a significant problem, please report this
At the end of your shift, make a summary.
Rough chronology of what was done, new problems, general comments.
The goal is to have one e-log report per run for routine operations, one
shift summary, and additional posts only in case of serious problems.
E-log TEMPLATES on the whiteboard
42 Friday, July 29, 2011
Hot issues

100% Tile Busy begin of a run:

Could be due to missing ECR at the start of run.
Tile will be busy after ~16000 events at Level1.
Faster solution of problem:

Stop / Un-congured (Tile) / Start run

Try at least 2 times if doesnt work call Tile DAQ

Medium term solution is to update the software to
avoid this
43 Friday, July 29, 2011
HOT issues

Module Trips:
Look into FSM and ALARM panel to identify if a module trips and auto-recovers.
Check in FSM if module is ON again.
Check MRS for auto-reconguration is launched (with 3-5 minutes after module is
Check L1Calo for auto-reconguration succeeds.
Check RITMO for increasing number of discard events, if that is the case:

re-congure again:

DAQ Panel Tile Stopless Recovery

if still discarding events:

Check OHP histograms: Browse TileGather DMUErrors

FracDigErrorsXXX . If more than 20% in Last lumiblock (number 0 in plot):

Call DCS on-call to ask for a power-cycle

Please, write all this information in your RunSummary e-log for all the trips in the run.
44 Friday, July 29, 2011

If MRS messages from TileLaserRCD complaining about absence of laser TT

Lutin detecter that although the number of laser pulses emitted is changing, the
number of Laser Trigger Types received by LASTROD has not changed for a long

Check in DAQPanel TriP (TriggerPresenter) the L1_CALREQ2 PreScale

if PS = -1 , it is trigger problems, there is nothing to do (except
if PS=1 then:
Laser should be disable (tell Run Control Shifter, see next slide for
emergency stop of laser)
SHAFT conguration should be checked (send an e-mail to laser
HOT issues: LASER
45 Friday, July 29, 2011
!"##$"%$!" &'()*+,-./&01(2-342. 5
How to disable the laser in a run ?
On the Tile FSM, if you log in, there is a
button "Laser stop" (or similar)
Push the button:
Laser will not be fired anymore
A stopless removal will be triggered to
remove the associated ROBlN
46 Friday, July 29, 2011
When things go wrong

Be prepared!
47 Friday, July 29, 2011
More information
48 Friday, July 29, 2011
This application is
ABSENT during
is OK it should
be like this
49 Friday, July 29, 2011
Masking a super-drawer
50 Friday, July 29, 2011
ECR Mismatch
51 Friday, July 29, 2011
52 Friday, July 29, 2011
Basic electroncs
Basics of Tile Electronics
In order to properly understand the various problems that can occur in TileCal, it is important to understand the basics of the TileCal front end
electronics. In case of a trip of a module or barrel, the shifter must be able to identify the nature of the trip and take appropriate action. The
nature of the trip is given by the error messages in the DCS alarm panel and the colored status display on the DCS.
Each TileCal drawer has a High Voltage (830V) input that provides the voltage bias for all pmts in that drawer.
In addition, low voltage is supplied to each drawer via a two-tier system:
1. A 200 V Bulk Power Supply channel serves four tile drawers. These are the represented by the innermost ring of each barrel in the DCS
2. The fLVPS (nger Low Voltage Power Supply) transforms the 200V into 8 different output voltages necessary for the functioning of the
TileCal Front-End electronics inside Super-Drawer(S-D). The fLVPS provides two sets of voltages: a low voltage for the readout
electronics (digitizers and motherboard) on on side of S-D and the low voltage for the High Voltage distributor system on the other side of
S-D. These two sets - "MB_side of fLVPS" and "HV_side of fLVPS" has to be distinguish / mentioned in the case of fLVPS trip.
Given this, drawer trips can be classied as follows:
Trip of the HV input channel to the TileCal drawer. One drawer becomes nonoperational and must be restarted by the DCS expert.
Trip of the fLVPS. One TileCal module becomes nonoperational and must be restarted by the DCS expert and in then the case of "MB_side
of fLVPS" recongured for data taking by the DAQ expert.
A drawer can still be operated in so called "emergency mode" if the "HV_side" of the fLVPS is tripped.
Trip of the 200V power supply. In this case, four drawers are made inoperable and each one must be treated as in a fLVPS trip. The DCS
expert must restart the 200 V Bulk Power Supply channel and fLVPS in all 4 drawers and the DAQ expert must recongure all 4 drawers for data taking.
53 Friday, July 29, 2011