Linux device driver for Radial Neutron Camera

in view of ITER long pulses with variable data throughput

B. Santos1, N. Cruz1, P.F. Carvalho1, A. Fernandes1, J. Sousa1, B. Gonçalves1, M. Riva2, F. Pollastrone2, C. Centioli2, D. Marocco2,
†,3
B. Esposito , C.M.B. Correia , J.M.R. Cardoso3 and R.C. Pereira1
2

1 Instituto de Plasmas e Fusão Nuclear, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal
2 ENEA C. R. Frascati, Dipartimento FSN, via E. Fermi 45, 00044 Frascati (Roma), Italy.
3 LIBPhys-UC, Department of Physics, University of Coimbra, P-3004 516 Coimbra, Portugal

Introduction Device Driver Implementation

The ITER Radial Neutron Camera (RNC) Data Acquisition (DAQ) prototype is based on the The Device Driver implements a kernel thread to
PCIe protocol as the interface to be used between the I/O unit and the host PC, enabling for the retrieve the data from the kernel space to the user
scalability of the final RNC DAQ system and allowing a sustainable 2 MHz peak event to cope space (accessible to high-level applications).
with the long plasma discharges, up to half an hour.
The thread runs in an isolated core and polls the
The prototype architecture comprises two digitizer modules with two channels in each, installed in status DMA. Based on its value, retrieves the last
the host PC. The maximum produced data throughput is up to 0.5 GB/s per channel in event available data DMAs.
mode (1 GB/s per digitizer) and up to 1.6 GB/s in continuous mode (for test purposes). The thread transfers to the user space all written
The host PC, a high performance computer, receives the acquired data through the Direct DMAs from last status change. Typically, the number
Memory Access (DMA) channels. The Linux Device Driver provides the interface between the of DMAs to transfer is one (all transitions are
hardware and the host applications. triggered) but if some status change is missing, the
thread automatically transfers all missing data
Previous preliminary tests showed that a traditional implementation of the Linux Device Driver blocks up to the length of the kernel circular buffer.
based on interrupt handling is not suitable for demanding data throughput. The Linux kernel
Based on the prototype specification, the device
miss several interrupts and receives Transaction Layer Packets (TLP) errors, reflecting a poor
driver provides individual device nodes (and
stability of the system.
memory buffers), to retrieve the data from the pair
The directly usage of the polling mechanisms to retrieve the data is not suitable as the DMA#n, ADC #n.
variable event rate over the same discharge does not allow to define an optimum fixed time to
retrieve data and identify missing data packages.

This contribution presents the architecture, implementation and test of a different device driver
approach using the polling mechanism which improves the performance and reliability.

System Architecture Each built data DMA block have multiple channel packets which can be from different ADCs.
The kernel thread separates the data from different ADCs into independent memory buffers,
The Host Computer runs Scientific Linux 7 as using the channel packet tag, which identifies the source ADC.
Operating System with kernel 3.10-rt
The DMA #n buffers stores the
Intel® Core TM i7-5930K@3.50 GHz
non-separated data that is
256 GB SSD and 64 GB of RAM available for test purposes (only)
The FPGA sent event data through two data paths [1]: during the prototype phase.
DMA#0 - Event base raw data path The high-level applications,
reads the data from the driver
DMA#1 - Processed data path
memory buffers, based on the
Each data DMA has 8 KB. In case of the raw data path, write and read pointers.
the driver transfers data in blocks of 32 KB (4 DMAs). Error counters were implemented
The DMA#2 is a special DMA containing 32 bits with to provide real-time information
the FPGA status information about data transmission. about data packages losses.

Some validation bits and also a counter flag are added to improve the reliability of the DMA
status information Results & Conclusions
The COUNT_DMA#n and DMA#n CYCLE on the status information gives the number of sent The tests show a stable solution during 60 minutes acquisitions with data acquisition rates up
DMAs for each data path and the last memory address written by the FPGA in the kernel space. to 1.5 GB/s.
The DMA#0 has 16 memory addresses and DMA#1 has 4, and each memory address has 8 KB of Using the interrupt approach
memory space. (implemented and tested in a previous
phase of the prototype), there are
missing packets above 512 MB/s for
single DMA #0 acquisitions, and above
64 MB/s for acquisitions from two DMAs
at same time.
The packets recovered is a relevant issue to guarantee the data integrity, despite small impact.
The usage of data transmission information can be a valuable contribution to solve problems when
a traditional implementation of a Linux device driver based on interrupt handling or polling
mechanisms cannot be used.
The presented architecture is scalable and adjustable. Also, is data-agnostic because only
the status information is considered to trigger the data to transfer.
The presented solution implements an internal data transmission recovery algorithm, enabling
the device driver to automatically check and recover missing data blocks in a transparent way
for the host applications. Using a traditional approach, in which a block is transferred in each status
changed, if some transition was missed the data would be lost.

Main References:
[1] A. Fernandes et al., FPGA code for the data acquisition and real-time processing prototype of the ITER Radial Neutron Camera, 21st IEEE Real Time Conference, 2018

[2] N. Cruz et al., The design and performance of the real time software architecture for the ITER Radial Neutron Camera, 21st IEEE Real Time Conference, 2018

[3] B. Santos et al., Real-time data compression for data acquisition systems applied to the ITER Radial Neutron Camera, 21st IEEE Real Time Conference, 2018

[4] R.C. Pereira et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer , Fusion Eng. Des. 123 (2017) 901-905

† Deceased 30 March 2018. Corresponding author: bsantos@ipfn.tecnico.ulisboa.pt

The work leading to this publication has been funded partially by Fusion for Energy under the Contract
F4E-FPA-327. IST activities also received financial support from “Fundação para a Ciência e
Tecnologia” through project UID/FIS/50010/2013. This publication reflects the views only of the author,
and Fusion for Energy cannot be held responsible for any use which may be made of the information
contained therein.
P3.060, September 19th 2018 | Related Contribution: P3.068 30th Symposium on Fusion Technology, 16-21 September 2018, Giardini Naxos, Sicily, Italy