Control and monitoring functions for all CMS detector subsystems are coordinated by the Detector Control System (DCS) Group in order to design and implement a single control system to operate the detector. Using guidelines provided by DCS, each of the subdetector groups are to define and design the data acquisition and control function needs for their sub detector. Data from these subdetectors is available to the global control system which provides console hardware and software, as well as display, archiving and other higher level services. This section describes the control and monitoring of the HCAL subdetector.
Parameters to be measured by the control system exist in three locations, the counting room, the side galleries of the cavern, and inside the detector itself. Physically, the electronics in the counting room and attached to the forward calorimeter are housed in VMEbus crates. Inside the detector, electronics resides in the readout boxes distributed throughout the HCAL detector. Access to data in the readout boxes will be via a network connection from dedicated DCS crates in the counting room to controller cards in the readout boxes. All VMEbus crates include a DCS processor with a network connection to the higher system levels. The DCS architecture is shown in Fig. 13.1.
The DCS hardware architecture consists of three principal layers. At the top are operator workstations and general purpose computers which provide access to all of the DCS services. The frontend or bottom layer consists of large number of distributed frontend processors (VMEbased). The middle layer is a dedicated local network interfacing the previous layers. This network is segmented for availability purposes, and is connected to the general CERN network by a filtered access. The general CERN connection provides for world wide access by authorised persons. The frontend layer connects to detector systems and equipment through direct I/O or through field buses shows the architecture of the DCS network.13.2.1.2
The software architecture is a cooperating network of dedicated control applications, DCS general services and gateways to other CMS and CERN systems. The dedicated control applications are the distributed controls for CMS detectors which are composed of real time process control components and supervision components.13.2.1.3
DCS services are needed for all operational phases of the experiment, from construction and commissioning to physics data taking runs, calibration tests and monitoring during shutdown periods. According to the operational phase, one or more operators will concurrently operate the CMS detector. These operations will be done from control rooms, equipment rooms, or remotely under the filtering of the DCS access control.
DCS provides general services; Archiver, Logger, Alarms, Access controls, Central Repository and a manager for the overall DCS activities. A toolkit is provided to build the dedicated control applications at the individual system level. The kit consists of drivers for the commanded I/O hardware, a control kernel, generic applications and a set libraries for specific applications.
The CMS collaboration has decided to implement a common general services software approach using an existing controls package. Commonality among all four LHC experiments is presently under discussion as well. In CMS and ATLAS, the respective DCS groups are evaluating EPICS as a possible general software system and have demonstration projects underway. EPICS is a highly developed software controls system that is widely used in the high energy and nuclear physics community and is supported by many laboratories.
To have a specific configuration and software environment, the remainder of this chapter is based on the use of EPICS in its current VME implementation. In this context, the starting point for the individual detector subgroups is the EPICS InputOutputController (IOC), a VMEbus based frontend crate containing a singleboard computer running EPICS compatible software. At the present time, IOCs use the VxWorks realtime operating system running on Motorola processor boards. The data request protocol, Channel Access (CA) allows upper level processors to read and set parameters in the IOC. At startup, each IOC is downloaded with the programs and database entries needed to acquire and process locally, the parameters associated with that particular IOC.
Because EPICS is used at many laboratories, software drivers are available for most commercial modules. The DCS group has indicated that they will recommend modules to use for standard analog and digital control system signals. This standardisation goal is intended to minimise the software effort required to implement subdetector control and monitoring systems.
An important feature of the total EPICS/IOC software package is an extensive system for alarms monitoring and reporting of analog and digital alarm conditions. Included in the alarms task is the provision for twotier analog alarms, with independent settings for high and low values, plus highhigh and lowlow values to distinguish severe from offnormal alarm conditions. Using this capability, less severe outoftolerance conditions can be reported, allowing preventative maintenance to be performed before real hardware failures occur. Alarm system values are part of the data that is downloaded to each IOC at initialisation time.
In EPICS, the database is a key element that allows the system to be adapted to different applications. Database records that are downloaded to the IOCs are processed at a rate dictated by that record. Processing the record may be as simple as reading an analog value, but other operations such as state machines and PID control loops are also implemented by configuring parameters of predefined database entries rather than writing special programs.
The majority of the remote data for HCAL consists of readings
and settings associated with the HF phototube high voltage supplies
and the data channels in the HB and HE sections
of the detector. There are 11,232 data channels distributed over the readout boxes for the barrel and endcap calorimeters. For each of the channels in a given readout box, there are eight bytes of test data, two bytes of timing trim setting data, and a byte of miscellaneous control and mode settings. Also for each readout box, there are about ten voltage and temperature readings to digitise.
To accommodate the above requirements for HB and HE, a small controller card will be added to the card cage used to support the photodetector electronics. This controller card is interfaced to the individual channel ASICs via CAN bus. Support for CAN bus will be included in the channel ASIC. Communication between the readout boxes and the EPICS nodes in the counting room will be by way of fibre optic Arcnet, a small network that can easily provide the necessary data communication requirements. A processor such as the MC68376 can provide the local processing, the CAN bus node interface, the AD converter and operate the Arcnet protocol chip. Inside a readout box, the CAN bus connection could be duplicated or segmented to minimise the effects of any potential single point failure.
From the EPICS VMEbus crates in the counting room, the readout boxes will be starconnected using a commercially available Arcnet copper to fibre optic hub. For convenience, the network cable is to be bundled with the other cables connected from the counting room to the individual readout boxes.
Arcnet is a small local area network that is well matched to the requirements for accessing data in the HCAL readout boxes. Protocol interface chips and network hardware for this LAN are readily available. Arcnet adapters for the counting room nodes are available as VMEbus boards and as IndustryPack modules. Because the readout boxes are inaccessible, the controller cards there should be as simple as possible. One of the important reasons for selecting this network is that very little software is required to support the communications. The protocol supplies secure, acknowledged, collisionfree transmission of data frames between nodes with little software overhead.
To conserve backplane space and minimise pin connections, some sort of serial connection is needed to communicate with individual channel ASICs. Instead of developing a nonstandard serial connection, the established commercial standard CAN bus will be used. Support for the CAN bus interface can be included in the design of the channel ASIC. During normal operation of the detector, no control system access to the individual channels is needed. At startup time, the timing delay setting and the mode setting bits are downloaded. For testing purposes, several bytes of test data can be downloaded to each channel. The size of the CAN bus frame is a good match to the needs of the individual channels. A standard CAN bus interface transceiver is used to connect with the backplane of the readout box.
The Motorola MC68376 microcontroller will be used for the controller board. This device includes a CAN bus module and a 1channel 10bit AD converter to read the voltages and temperatures within the readout box. These readings will be returned to the EPICS crates in the counting room once a second.
Control and communication functions for distributed data from detector systems that use standard electronics crates are implemented by providing an EPICS IOC node in each crate.
In the case of readout channels where the CAN protocol is used for communications with the channel ASICs, VME 9U crates are used. The P3 user defined area of the backplane is used for CAN bus lines to each module. Connections back to the higher levels of the DCS are through its standard LAN interfaces.
High voltage supplies for the HCAL photodetectors are commercially available units that reside in the counting room in the case of HB and HE, and in local relay racks for HF. VMEbusbased controllers allow voltage values and maximum current values to be set, read, and alarmed as normal EPICS devices. Control functions for the high voltage channels will include ON, STANDBY, OFF, and the RESET of overcurrent trips conditions.13.4.1.1
Hybrid photodetectors (HPD) are used in the barrel and endcap sections of HCAL. There are 36 readout boxes for the barrel section, 36 boxes for the endcap sections, and 72 boxes for the outer calorimeter compartments. The HPDs require an accelerating voltage in the range of 1016 kV plus a diode bias voltage of 150 volts. There are 144 high voltage supplies, one per readout box, and about 500 bias supplies, one per HPD. These voltages will be cabled separately to each of the boxes. Each supply includes a fast overcurrent trip. The controllers for the high voltage supplies reside in an EPICS IOC crate. Operating parameters are downloaded as normal EPICS settings.13.4.1.2
The forward detector uses photomultiplier tubes that require high voltage in the 12 kV range. Controllers for the high voltage supplies reside in EPICS IOC crates that are located in racks near the detector. Each supply includes a fast overcurrent trip. The total number of PMT high voltage channels is 3744. Operating parameters are downloaded as normal EPICS settings.
Voltages are distributed to each readout box separately (chapter 12) where local regulators are used for each channel. Low voltage bulk power supplies, +5V and +10V, for the front end electronics in the readout boxes are located on the side galleries of the cavern. Voltage and current readings for these supplies, both at the source and at the load, along with measurements of the lead temperatures will be returned to IOC crates in the counting house. The forward detector uses standard crate and rack based electronics, and the DCS infrastructure for rack systems can be applied.
Electronics in the counting room will be implemented in the 9U
VME64x format. The design, manufacture, installation, and cooling
of the racks and crates is the responsibility of the
CMS infrastructure group. Monitoring and control functions are the province of the DCS group.
The space required to house electronics in the counting room depends on the allowed power dissipation per card. For the case of 16 channels per module and 18 usable slots per crate, the 14,000 readout channels in HCAL will require about 50 crates located in 16 racks. Monitoring operating parameters for each crate is necessary as well as twotiered alarming on overtemperature, overvoltage or overcurrent conditions.
Within each of the boxes in the detector, the ambient temperature of the air and the cooling plates will be monitored for alarm conditions. In this case, no additional control is needed, because the reason for an alarm condition can usually not be corrected by the HCAL group. Temperature readings would be returned to the IOC crates in the counting room using the Arcnet connection to the individual boxes. Temperature monitoring and alarming for the forward calorimeter rack systems are included in the standard rack protection package.
A source calibration system is integrated into the HCAL sub detector. A radioactive source attached to a wire is guided by a small tube through portions of the detector while the DC response of the scintillator, light pipe, and photodetector system is recorded using the normal fast data collection system. Control is needed to select between several tubes and to drive the wire out and back during this calibration process. Twenty four such calibration systems are needed to cover all parts of the HCAL system. The source motion can be controlled either by the processor in the IOC crate, or by a companion processor board in the IOC crate. To interpret the calibration data, the source position must be correlated with the response data from the photodetectors. Even without a special timing system, measurements in separate EPICS IOCs can be correlated to a few milliseconds.
For the calibration process, the frontend electronics will be downloaded with parameters such that 16 consecutive readings are summed by the FIR filter of the second level trigger electronics. Then several hundred such sums will be averaged and the results correlated with the position of the source. Both the frontend electronics crate and the source controller crates will contain an EPICS CPU card. The reading of source calibration data will be controlled by the EPICS processor in the frontend crate which is able to cause the Level_1_Accept trigger needed to control the second level filter. This EPICS node will request and receive timestamped values of source position from the EPICS CPU in the source controller crate. It is desirable to have the calibration system independent of the Data Acquisition System and so the processing of the calibration data is done by the EPICS system in the frontend electronics crate. It is not necessary for the trigger and DAQ systems to be operating during source calibration operations, only the system clock is needed. Details of the source controllers, the mechanism for moving the source and the readout of source position are described in chapter 10.
Part of the Luminosity monitor instrumentation includes several Roman Pots which contain devices that measure small angle scattered particles near the beam. In addition to the frontend electronics used to process the detector data, the Roman Pots need slow controls support for motor drivers, position readouts, high voltage control and readout, and downloading operating parameters and test patterns. Frontend data processing electronics will reside in two VMEbus crates, and to provide the slow controls support, an EPICS IOC processor will also reside in each of these crates. These processors are also used to read and monitor the parameters for the Roman Pots and provide an access filtered path for authorised personnel (in the accelerator control room) to read and control the position of moveable devices near the beam.
The upper levels of DCS will provide archived information for HCAL. Two types of archives are needed; constants used for calibration and configuration of the detector, and values of monitored parameters. Values include voltage and current readings, temperature measurements, and results of calibration runs.
Test capability will be needed for the frontend DAQ and trigger electronics located in the counting room. It is expected that boundary scan testing will be incorporated into the design of the digital electronics. All electronics in the counting room will reside in VMEbus crates, so test data and readback will be possible over the DCS network. Test events can be written to and read back from the DAQ buffers using EPICS software in a processor located in each crate.
Operational testing and troubleshooting should be possible to
accomplish from a remote workstation, located anywhere. Realtime,
or pseudo realtime, correlated displays of accelerator data,
detector data, and DCS parameter values are essential for diagnostics
and troubleshooting. Gateways to the data acquisition processor
farm and to the accelerator Controls System should offer this
functionality.