Pierre Auger Project

Progress Report

Summary (photo album)

The May/June FD shifts were generally smooth.  Although the FD – lidar interference has been reduced, there are still occasional T3 bursts from crossfire when shoot-the-shower is used.

Modifications to improve the FD shutter reliability have been completed.  The optical filters and some of the mirrors have been cleaned.  The Loma Amarilla building is progressing well with the contractor pressing to complete the roof before the snow flies.

There are now 1152 tanks in the field with 930 in operation.  Considerable effort is being focused on a small percentage of tanks with performance problems in the field.  New monitoring and failure tracking tools are making this work more effective. The battery shipment anticipated for many months has arrived.  Batteries that successfully pass acceptance tests will be installed in the large number of tanks now without power or electronics.

Land access in the North-East part of the site continues to slow construction.  Progress, however, is being made in negotiations with one of the owners.

The DPA-offline continues to mature with new features being added.  Planning is also underway to initiate a more formal change control process for the addition of modules.

Two draft proposals for Observatory enhancements have been received and are under preliminary review.

WBS 1.1 Fluorescence Detector (Jonny Kleinfeller – Karlesruhe)

The main activity at FD during the last two months was completion of the shutter exchange,
modification of the seal at the aperture box of bay 6 at Los Leones, cleaning of the filters in all buildings and cleaning of the Karlsruhe mirrors at Los Leones and Los Morados.

The alignment of the mirrors at Los Leones and Los Morados had been checked.
Minor realignment was necessary in both buildings.

The misalignment at Los Morados was systematically in one direction; this could be related to the air-conditioning system, which was out of order for most of the last month.
The alignment needs to be cross checked once the bloody air- conditioning is working again reliably.

The shift went, apart from the weather, reasonably well, without too much frustration for the shifters, we had two experts on shift this time.

The Lidar group worked hard to reduce interference of the Lidar with the data acquisition.

WBS 1.1.3.3 Fluorescence Detector Atmospheric Monitoring (Stefan Westerhoff – Columbia)

Lidar Systems (reported by Segev BenZvi)

Since the March/April shift, three lidars have been running continuously with high-repetition lasers, and the raman inelastic lidar has been taking data at Los Leones.  The majority of the shift time during this period has been devoted to debugging two effectively "new" sites (Los Morados and Los Leones) and mitigating lidar interference with FD operations.

The current lidar effort is focused on three tasks:

1) Automating detection of laser triggering and transferring the lidar stations to a "safe" operating mode;

2) Simplifying operating procedures so that non-experts can safely run the

equipment.

3) Fine-tuning cuts and vetoes to reduce lidar "crossfire" into other FDs during shoot-the-shower.

For obvious reasons, much attention has been devoted to the first of these tasks. During the debugging and testing period, laser-related triggers increased significantly at all three sites.  Some of these were discussed and coordinated with the FD operators, but several were accidentally induced because of miscommunication between the lidar and FD personnel.

These incidents highlighted a need for better communication among FD and lidar operators, and during this time the cooperation among the runners has markedly improved.  Thanks to new monitoring software put in place during the last shift, the FD runners are now aware of lidar-induced trigger bursts; when such events occur, the lidar immediately stops running.  We are now writing software to automate this behavior.

Regarding the second task, we are now upgrading the lidar software to allow nonexperts to operate the lidar stations.  Current runners are required to run and track several separate programs at each lidar site. These will be consolidated into a simple interface that minimizes user

control over the system.  This will not only simplify running, but also ensure that novice runners cannot improperly initialize equipment (such as the high repetition laser).

Finally, our attention has also been devoted to optimizing shoot the shower.  To ensure that all high energy showers are being shot, the lidar has cast a wide net; unfortunately, this means that too many low-energy showers are being shot.  Outside the lightning season, shoot the shower is

a major source of trigger bursts in the FD.

To lower the noise level, we are trying to dramatically reduce the number of shooting requests, and restrict the allowed shooting directions. However, it has become apparent that vetoes based in the lidar alone will probably not be sufficient to prevent all bursts.  Some kind of filtering

will also need to be done at the FD level to remove laser bursts from the data.  In the FD, software capable of such filtering is largely written already.

Raman Lidar (reported by Aurelio Grillo, Carla Macolino, Vincenzo Rizi,

Marco Iarlori, Jorge Rodriguez)

Thanks to the help of many people, both from the lidar group and outside it (we couldn't go to Malargue ourselves for several reasons) the situation of the Raman lidar has greatly improved. The improvements concerned mainly the optics (a different collimator lens has been used, with better focusing. Also a 1/3 pass neutral filter has been inserted in front of the pmt of N2 channel to lessen saturation, and a similar one is on the way for the elastic channel) and electronics (a pre-amplifier has been put again in the acq channel).

Operation is now stable and sensible and VAOD(4.5 km asl) values can be easily obtained which look reasonable (at least after visual scan of the atmosphere by people in Malargue, mainly Jorge Rodriguez and Roberto Mussa). While we are awaiting a detailed comparison with the CLF data when present, we are planning to have CLF data brought in L' Aquila by the Lecce shifters and to perform a quick analysis for them.

In the following table we report the VAOD values for a few days.

19/05/06 VAOD=0.21 +or- 0.050

20/05/06      0.16      0.053

22/05/06      0.078     0.067

23/05/06      0.062     0.043

24/05/06      0.20      0.058

The analysis follows he lines presented in November in Malargue, and goes through: a) correction of the signal for the molecular transmission as for the monthly average atmosphere b) fitting of the resulting logarithm of the aerosol transmission via a broken line, which

allows to dispose of a (unknown) constant related to acq efficiency and laser power, by imposing VAOD(LL)=0. For the moment the reported errors are the (propagated) errors in the fits.

We will report more in detail on the analysis with a GAP note as soon as we collect more statistics and we compare with other approaches. Meanwhile, we are planning to have (preliminary!!) values of VAOD (4.5) reported in the daily lidar-shift run log.

The attached plots regard the data of May 23. The data could be analyzed for the aerosol content, although the raw data for the N2 channel clearly show a cloud at about 7 km height.

More structures for this night are present in the elastic channel.

FRAM (reported by Michael Prouza)

After upgrade of FRAM mount internal software in March, we finally took advantage of the new functions and started to routinely center the selected catalogue stars in the field of view of the photometer. It means that now we have both photometric data from stars and from HAM lamp

observations.

Still, there is some work to optimize the star selection from the catalogue and to optimize especially HAM centering process. Further, during next months we have to process the available data from both terrestrial and stellar sources, compare them and finally - and most

importantly - establish an automatic processing pipeline for the data. However, it seems likely that we will have some results for the wavelength dependence of the extinction for both types of sources quite soon.

Concerning the secondary program of FRAM, we succeeded in having of second positive detection of an optical counterpart (OT) of a GRB on April 18th (GRB 060418). This time we observed the optical counterpart for several hundreds of seconds and were also able to detect the relative rebrightening about 300 seconds after the gamma emission. The OT was much

fainter than the one detected in January (GRB 060117), actually near the magnitude limit of our instrument (~ 15 mag in R filter). The image of the observed counterpart is in the photo album.  This particular case of OT was well monitored by a number of other telescopes; therefore an independent publication of our results is unlikely.  However probable, our results will be included in the common study of that particular GRB (work coordinated by the people of Liverpool Telescope) and we will inform the collaboration as soon as a first draft of the paper will be ready. Because of this effort of other institutions, the host galaxy was identified just days after the GRB appearance, at z = 1.5.  Our paper about previous GRB OT observation, already approved by the collaboration, was also accepted for publication in Astronomy & Astrophysics Letters at the end of May.

The overall quality of data taking of FRAM was negatively influenced by two major electric dropouts, followed by an incorrect recovery of the system. For the first time (end-April), our UPS was damaged and later some problems of electronic board driving the roof appeared. After the second dropout (mid-May), again problems with the same electronic board appeared

and one of the CCD cameras was not responding. Both problems were solved (first by Czech shifters Petr Travnicek and Jan Ridky, second by Primo Vitale), however these resulted also together in about three weeks of dead time without any measurements. During June some measures will be taken to try to establish better surge protection of the FRAM house.

Finally, the HAM light source was out of operation for a large part of April, and after being set up again by Primo Vitale at the beginning of May, HAM is about 3x brighter than it used to be before. Therefore, we will have to have two separate calibrations of HAM - before May 06 and after May 06.

WBS 1.2 Fluorescence Detector Electronics digital electronics and readout systems (Matthias Kleifges – FZK-IPE)

Hardware improvements

We have found the reason for the sometimes missing seconds in the data and the inaccessibility of the SLT board through the DAQ (see previous bi-monthly report). The problem will be fixed with the next firmware release to be downloaded at the end of the April/May shift. The last shift April/May and the current shift May/June have not complaint any of these errors; everything is working well.

Software progress

H.J. Mathes is working on improvements of the DAQ software, which he checks out with the test system in IPE, i.e. electronics waiting for the installation in Loma Amarilla. The following improvements or new features are still under test, but will be installed in Malargüe with the next software release (if found to be working):

The inter-camera trigger signal is now also supported by the DAQ readout scheme. This feature will not improve the trigger efficiency for showers falling in between two telescopes. However, the data quality is improved as small light tracks on adjacent cameras – which would not trigger by its own – are also readout and those pixels put into the data.

Light sources in the field (CLF and any kind of lasers) can now activate the ExtTrg signal of the GPS clock as described in GAP 2005-032. Events tagged by the ExtTrg signal will be handled as artificial light events and are stored in the appropriate data sink.

Fixes of bugs reported in several applications.

In order to ease the handling, some manual action actions of the shifters are automatically done by the DAQ. The recording of background data (command bgrecord) can now optionally started automatically with the beginning of shower data recording. Similarly, the script for the analysis of calibration data is automatically triggered after the recording of calibration data is stopped.

Since the transition to the FDEventLib v3r0p1 there were problems reported from groups working with the 64-bit SunFire architecture. This mainly affects programs for data merging of FD and SD data. We clearly need help from groups working with this architecture in order to find a solution for this problem.

Performance Monitoring

The groups at Wuppertal and Karlsruhe have continued their activity to implement data bases for the performance monitoring of the FD operation. The program for the analysis of calibration data was refined and a perl script to fill the data base with values was developed. The definitions of the database tables are finished and described in our Wiki pages. Recent work in Wuppertal has been in a number of different areas:  a.) defining the format and structure of the (PHP) web pages, b.) testing of the planned replication setup and, c.) implementation of the first pieces for the "alarms" system. The current priority is to produce a standard PHP environment for the development of the web pages, so that more developers can get involved in the project. Therefore the FD monitoring group is currently preparing a meeting in Wuppertal (June 12-16, 2006) for the coding.

R&D for Auger North and the Enhancement project

IPE has continued its design work for Auger North and the enhancement project. We have finished the schematic diagram and started the PCB layout for the digital backplane and for a first FLT test board of the new design.

WBS 1.2 Fluorescence Detector Electronics (Daniel Camin – Milano)

The groups meet frequently to consider eventual changes on:
 
1) Analog Boards.
2) Head Electronics.
3) A debugging Test System for the HE, recently completed.
4) The CAEN HV-LV system is a modular and flexible commercially available
system. Changes should be done at the manufacturer site (CAEN).
 
1) The agreement on the use of a faster system, 20 MHz instead of 10 MHz, require use of faster monolithic chips, and eventually a new layout and the introduction of current limiting resistors and decoupling capacitors. The layout would probably need  to be optimized to allow faster response and to reduce the noise. The item 1) would probably require some modifications, to enhance the rejection of noise. The variable monolitic resistors shall be substituted by the new generation of integrated potentiometers with higher resistor values. The new generation of differential input and output will help to reach a high dynamic range.
 
2) The Head Electronics proved to be a reliable part, that presented extremely low failure rates. The production was done in Intratec, Berlin. Use of an elaborated test system allowed to identify units that were outside the specifications. No major changes are foreseen for those parts.
 
3) A debugging system to screen-out failures has been designed. The system is under control of LabView running on a portable PC.
 
4) The CAEN HV-LV Power Supply behaves well although failures appeared in some modules that were later repaired.
 
The above mentioned issues have been discussed within the groups. We expect to come with further and fresh news before the new bi-montly report.

WBS 2.1 SD (Ingo Allekote – Instituto Balseiro)

The numbers corresponding to this period are:

- Tanks received and inspected in Malargue: 60
- Detectors assembled: 36
- Detectors positioned: 24
- Detectors filled with water: 27
- E-kits installed: 25

As of May 31st, 2006:

- Total SD in the field: 1152
- Total SD with water: 1123
- Total SD installed: 953
- Total SD working: 930

The long awaited shipment of batteries finally arrived, and in the beginning of June the first batch of 180 batteries was received in Malargue.

The detector Estelle was located on the riverbed zone at the junction of the Atuel and Salado rivers. On February 7th, when the rivers carried a huge amount of water, the tank collapsed
and its battery box was flooded. The tank has been emptied and maintenance work is underway to reposition it on a platform made with wire mesh and stones. The tank might be brought back
to the AB for repair. The "pala" (frontloader) was also used to reinforce the dams protecting the tank Ofid, which is also located in a risky area in the Atuel riverbed.

To assess the impact of emptying and refilling a detector (of relevance for the twin tank proposal), bacteriological water tests were performed on all tanks that were re-filled.
No anomalous bacteriological activity was detected in these tanks.

WBS 2.2 Surface Detector Electronics (Tiina Suomijarvi – IPN Orsay)

The total number of PMTs received in Malargue is 4932 (96.4% of the total order). The number of tested PMTs is 3793 and potted PMTs 3607.  In March, the potting was stopped due to some concerns in the potting procedure.  The potting equipment was carefully cleaned which increased the quality of the potting. However, further controls and studies are currently undertaken.

There are currently about 50 PMTs back from the field to be tested in the PMT house. A technician from the IPN-Orsay (Thi Nguyen Trung) will come to Malargue (end of June early July) to help tracking the failures. Only the PMTs with a clear PMT failure (not base of connector failures) will be shipped back to Photonis.

A batch of 12 PMTs with strange temperature or instability behavior was shipped back to France for further testing in laboratories (Torino, Grenoble, Ohio). We are currently waiting for the test results of these PMTs.

When PMTs are removed in the field, the RTV sometimes looks opaque or have some small bubbles. This problem will be followed and the PMT gluing procedure at the AB will be controlled.

Total 42 Ekits with Cyclone boards have been deployed to the field.  A batch of 198 FE boards just arrived to Malargue.  The next batch of 216 TPCBs (A+B) from CWRU will be shipped next week and is expected to arrive to Mendoza June 5. In addition to the Cyclone Ekits, there are 22 Ekits (with the FE3.3 boards) stored in the SDE-Fabrica for the field maintenance purposes. The Cyclone boards seem to work fine in the field. However, a more detailed study on their performance should be done before starting massive deployment.

A database for Ekit failures has been developed (Fernando Contreras). The database is currently under PMS (SDECO DATABASE). It will allow better tracking of the failures than BUGZILLA.  A similar database exists for the PMT failure tracking (developed by Walter Fulgione).

Figure: SDECO DATABASE for Ekit failure tracking.

The 500 mA UB fuses are being replaced to 750 mA fuses. There are currently in the field 238 Ekits with 750 mA fuses, the rest (about 700) are with 500 mA fuses. It seems that the increase of the fuse size decreased the number of  field failures. For the moment, with 750 mA fuses, no unexplained failures (fuse blown without any severe problems) have been observed.

Two students, Anne and David, from the Applied Physics master program of the University of Paris South – Orsay are helping the SDE-Fabrica staff since April first for their 3 month training program (under supervision of Beatriz Garcia).

There are  957 Ekits in the field. Currently there are only 27 available positions for the Ekits due to lack of batteries. A batch of 180 batteries should arrive in a couple of days, which can then allow Ekit deployment to continue.

Maintenance program has been running with a steady rate of about 3 field trips per week. The strategy is still to repair black tanks and PMT problems in selected areas.

The strategy of SDE deployment for the next 2-3 weeks is the following:

· While the PMT potting procedure is being checked and the potting quality is being increased, no PMT potting is done for the moment. There are currently PMTs available for about 80 tanks total (including tanks that are in the AB area and already have PMTs).

· There will be 165 tank positions available soon (Italian landowner). If the water delivery rate is about 40 tanks per month, more PMTs are needed in the middle of August (if Italian land is available early July). This would leave about 2 months to check the potting procedure.

· The PMT assembly in the AB is about 10% working time of the total SD assembly. In case PMTs are not delivered, the assembly work can continue and PMTs can be installed later without any major difficulties.

· If batteries arrive there will be about 100 tanks available for Ekits. Ekits with Cyclone boards will be deployed to these tanks when green light is given from the studies of  the current 42 Cyclone FE boards, which are already in the field.

· The regular maintenance operation will be maintained in parallel with the Ekit deployment.

WBS 5.0 DPA/Offline- (Bruce Dawson, Markus Roth and Tom Paul)

Migration of the Offline software from the CVS repository to an SVN repository has been completed.  SVN is a more modern replacement of CVS, and will afford the offline team much more flexibility in reorganizing the code, and in particular the module distribution scheme.

Collaborators can access the repository with the usual Auger username and password; there is no need to request a computer account.  See the wiki pages (https://www.auger.unam.mx/AugerWiki/SvnUsageNotes) for details.

A buildbot farm has been set up, comprising 5 slavebots at the current time.  The buildbot system performs automatic checkout, compiling and testing of the code each time the repository is altered, and informs developers in case of problems.  This system allows the Offline team to

quickly catch new bugs, and to confirm the software works with different platforms, different compilers, different external libraries, and so forth.  The goal is to guarantee ongoing support for each system configuration for which we have a corresponding slavebot in the farm.

The status of the buildfarm can be viewed at http://hepgis.physics.neu.edu:8010.

The framework configuration machinery has been enhanced to perform md5 fingerprinting on all configuration files.  This functionality is meant to provide a quick way to detect whether or not 'standard' configurations are used to generate results.  In much the same way that analysis coordinators can recommend specific physics modules, they can also propose specific configurations, with the fingerprinting mechanism providing a way to confirm that these configurations are properly employed in production.

Some new database features have been added, including support for listing multiple database servers.  The intention is to provide a way to specify backup servers in case a primary server cannot be contacted, and to  deal with certain MySQL version compatibility issues. A first

"smart" Mie model has also been prepared, which reads information from the databases if available, and uses a parameterization otherwise.

New tutorials have been included in the distribution, including an example of how to call the CDAS Ec, Es and Er algorithms from within an Offline run.

A new central trigger simulator has been prepared, which uses the actual code deployed by CDAS (Xb).

The FD simulation has been revised; in particular the telescope and light at the diaphragm simulations, and the original "OG" simulation modules have been complemented by "KG" modules from the Karlsruhe group. A new laser simulator has been written by the Napoli group and is now included in the distribution.

WBS 9.0 Observatory Operations – (Julio Rodriquez Martino - INFN)

Apart from some minor hardware problems, the shift spanning over April and May went smoothly from the FD point of view. Most problems were due to LIDAR interference, both in the regular scan mode and during the "shoot-the-shower". These problems are still being investigated.

The weather was generally good, with a few stormy nights.

The shift slots are being covered without major problems, both for the FD and the LIDAR. We hope this tendency remains stable during the rest of the year.