Pierre Auger
Project
Progress Report
June & July
2007
Summary (photo album)
Both deployment and data taking at the Observatory have been disrupted over the past few weeks by unusually heavy snow and cold weather. Current surface array deployment operations on the “Italian” lands in the northeast part of the array have been suspended until conditions improve. Prior to the bad weather, however, good progress had been made installing and bringing surface detector stations on line. As of the time of this report (24 August), there were 1480 detector stations deployed with 1355 taking data.
Once the Italian land is populated we will need to fill the holes in the array. Of these the largest is near Coihueco. No progress has been made recently in persuading the current occupier of this land to allow access.
It appears that progress at last is being made in understanding the mystery of the unstable (“raining”) PMTs. Although the performance of these tubes does not seriously impact the quality of the data, understanding its source and a possible path to mitigation will be most welcome.
Good progress has been made recently on the data base of calibration constants for the fluorescence detectors. Such constants will significantly improve the quality of the data.
Preliminary work will soon start on both the HEAT and AMIGA enhancements. A contract is pending for the site preparation for the three HEAT mirror enclosures near Coihueco. The first five infill tanks have been deployed for the AMIGA infill. The AMIGA detectors will be used for R&D studies.
A number of important physics papers are in preparation.
WBS 1.1 Fluorescence
Detector (Jan Ridky – FZU)
We are implementing new scheme for shifts, we were able to enlist enough shifters to run FD during austral winter.
WBS 1.1.3.3 Fluorescence Detector Atmospheric Monitoring
(Stefan Westerhoff –
Atmospheric Analysis
(CLF) reported by Laura Valore
This is an update on
the status of the reconstruction of atmospheric parameters from CLF
data performed at
Since the Malargue
meeting, we reanalyzed all available CLF data from LL and CO with
the new FD
calibration constants to extract the aerosol content and to reconstruct
CLF energy on data
covered by the present aerosol database.
With the new
calibration constants, the decreasing trend with time of the reconstructed
to nominal CLF
energy ratio is much less evident than in the past, however some
structures are still
there, in particular:
1- Data from April
2005 to March 2006 (the longest uniform period) still show some
energy decrease with
time (FD calibration "C" needed?).
2- The same data
(which correspond to a single CLF epoch) show a larger energy deficit
than other periods
(both from LL and CO, which have different changes in the FD calibration).
This jump may be due
to a jump in the CLF calibration constant for that epoch (within
the quoted CLF
systematic energy errors).
3- Data from April
to August 2006 (very low laser energy E<4 mJ) have structures that
are still not
understood.
4- There is an
overall energy deficit of about 13% in all data since November 2004 (is
this correlated with
the shift of the Auger spectrum with respect to other experiments?)
In order to obtain a
consistent set of data to extract the aerosol concentration, we
dropped the CLF low
energy period (April to August 2006) and corrected the remaining
data to account for
the energy loss and recalibrated each data period to recover the
missing energy with
respect to simulations!!
With these changes
the analysis is not Monte Carlo-normalized any more, but depends less
on calibration
issues. It is also no longer completely
independent from the current CLF
analysis by Mike
Roberts), which as currently used for the atmospheric database. For our
method, we estimate
a conservative 10% systematic on the vertical aerosol optical depths
with this analysis.
The correlation
between our results and the optical depth in the database is now very
good for all periods
(see Fig. 1). In addition, 457 hours
from LL and 404 from CO not
in the database are
successfully characterized by our analysis.
WBS 1.2 Fluorescence Detector Electronics
digital electronics and readout systems
(Matthias Kleifges – FZK-IPE)
Tackle of recent problems (by H.J. Mathes and
Kai Daumiller)
At the end of the June FD shift Los
Leones ran out of disk space due to problems with the replication of the FD
monitoring database (DB). Consequently, the FD databases were left in a highly
corrupted state and July FD shift started with a delay caused by repairing the
DB. Some data - especially the BG data - could not be recovered. To ease the
recovery in the future, a number of changes to the DB filling code have been
done or will be done in near future. In addition, a better supervision of the
DB replication is needed and no-longer used log-files have to be purged
automatically. The problem will be solved in future also by a factor 3.5 larger
raid disks in the EyePCs which have been procured already.
The FD inter-camera trigger was
enabled in Los Leones during the June and July shift. At the same time a
drastically increased rate of crashes in Los Leones for the Eventbuilder (Evb)
was observed. The analysis of previously recorded log-files revealed a slight
increase since the beginning of 2007 when the T3 was modified. The failure rate
of the Evb seems to be correlated with an increase of the Evbs throughput
capability and load. At the moment the most likely reason for the crash is a
memory leak eventually correlated with some faulty interaction with the MySQL
database. Further investigations and tests are going on.
Slow Control activities
Kai Daumiller is extending the slow control
system for two respects:
1. Software to address a watchdog card
in the SC PC, which would disconnect the Profibus and in turn brings all
connected components in a save state. The watchdog would become active in case
the monitoring functions of the system are blocked for any reason.
2. Implementation of a serial protocol
to access the Liebert UPS for the shutters, which would allow reading UPS
parameter and a defined switch-off of UPS load.
The extensions will become effective
with his next stay scheduled for October 2007.
R&D for Auger North and the HEAT proposal
Since July we have a first board of
the new FLT board available. Unfortunately, the PCB layout misses some bus
lines to the backplane thus testing the data transmission to the SLT is possible
only with a revised board. The new revision will heal also a fault preventing
the proper FPGA programming.
WBS 2.1 SD (Ingo Allekote – Instituto Balseiro)
Tank
fabrication at Rotoplastyc (
There are 148 tanks waiting for shipment to Malargue and no remaining resin.
With 1540 tanks already in Malargue, this will bring the total to 1688. Funding
for the shipment has been proceeding through government channels at a slow pace
but is now expected to
become available by mid-August and delivery of tanks to Malargue may resume by
the end of August.
R&D for foam-insulated tanks for Auger North at Rotoplastyc has
demonstrated that the foaming polyethylene resin produced in
being prepared.
In the period June - July 2007 the following tasks have been accomplished:
- Detectors assembled: 84
- Detectors positioned in the field: 66
- Detectors filled with water: 57
- E-kits installed: 34
As of July 31st, 2007:
- Number of tanks in AB yard: 28
- Assembled detectors in AB yard: 33
- Total SD in the field: 1482
- Total SD with water: 1418
- Total SD with e-kit: 1344
Deployment and water delivery is continuing in the land owned by the Italians.
Some stops in water delivery were caused by heavy snowfall. During the last
week of July the water plant couldn’t deliver water with the minimum quality,
but this problem could be solved by Pedro Barraza, who washed the EDi with a
saline solution.
Roads were cleared in the
and batteries.
At the end of July, 5 infill tanks were positioned in the field of
Pietrelli-Ibarra, South of El Sosneado. These will be filled during August to
conform the unitary cell for the infill. Two additional tanks were installed in
the same area, with a separation of 200 meters, to be used as
a mini-array (not integrated to CDAS) for the muon detector prototype.
WBS 2.2 Surface Detector Electronics
(Tiina Suomijarvi – IPN Orsay)
Deployment status:
There are currently about 1370 Ekits in the field. The installation of the Ekits is currently done by the SD staff, only in case of problems the SDE technicians are called. The SDE technicians are essentially performing maintenance.
Ekit component
inventory:
|
Current
production: 1560 Ekits |
UB Box |
UB box back panel |
UB box front panel |
TPCB box |
TPCB box back panel |
|
Available
Elements |
118 |
72 |
76 |
96 |
71 |
|
to
complete 1600 (40 Ekits more) |
118 |
72 |
76 |
96 |
71 |
|
Necessary
for 1680 Ekits (80 spare Ekits) |
-2 |
-48 |
-44 |
-24 |
-49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Red: necessary amount |
|
|
|
|
|
|
Green: leftover amount |
|
|
|
|
|
TPCB box front panel |
FE |
UB |
GPS |
LED |
TPCB |
Radio |
|
53 |
118 |
243 |
77 |
184 |
20 |
70 |
|
53 |
118 |
243 |
77 |
184 |
20 |
70 |
|
-67 |
-2 |
123 |
-43 |
64 |
-100 |
-50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
192
TPBC pairs were sent to
About
50 TPCB's that have failed the tests are still in CWRU. Some of them could
probably be fixed.
100
each of both spare GPS receivers and spare antennas have been sent
to
Marc Kaducak two weeks ago to be put in the next shipment from Fermilab.
There
are still about 50 more GPS spares in CWRU.
The total number of UB and TPCB boxes that was ordered is 1750 each.
1524 were shipped in
UMN-01-12-MAY-2003 and 110 were shipped in UMN-03-25-JUN-2003 to Malargue. 141
were shipped to PSU. That's a total of 1775.
PMT inventory:
PMTs received in Malargue (Photonis data
base):
PMTs
ordered for the base-line SD: 5040
Additional
PMTs ordered: 240
Total
order: 5280
Replacement
PMTs: 227
(of which 9 with Chinese glass)
Total number received: 5507
PMTs received in Malargue (Malargue
inventory):
PMTs
in the field: 3x1497 =4491
PMTs
in the AB yard: 3x38 =114
PMTs
in the AB building: 38
Test
tanks (AB+ SDECo): 12
Stored
in the SDECo: 80
(returned from the field)
Stored
in the SDECo: 91
(failed in the tests)
Stored
in the SDECo: 9
(failed vis. inspection)
In
the potting area: 18
In
the dark room: 24
Ready
to be sent to
In
the
Shipped
back to Photonis: 240
Available
for deployment: 15
Waiting
to be tested: 60
Without
data for testing: 71
To be
re-potted: 14
Without
bases: 144
Total: 5445
The difference in Photonis and in Malargue
inventory is about 60 PMTs. It should be clarified what is the reason for this
difference.
Total
number of PMTs that can be deployed, without the 144+12 PMTs which need bases,
is
4822.
This is enough for the array of 1600 tanks.
Torino
has shipped 160 bases, which are currently in
LED cables:
There
is a total number of 1632 LED cables (pairs). To have enough for the infill 70
more cables should be ordered.
PMT maintenance:
There are currently 80 PMTs in Malargue that were returned from the field. Further diagnostic should be done in Malargue and it should be studied if some of the PMT bases, in case the failure is on the base, could be repaired in Malargue.
A procedure to remove potting has been developed in Malargue. To remove potting from one PMT takes about 30 min.
The
WBS 3.0 Comms
(Jorge Abraham – UTN)
AntennaKits mounted: 1418 AntennaKits made: 1570 We have all components for 1600 Antennakits and near 50 Spares. About the re-orientation plan and the UV protection programme, We are working altogether with the Observatory tasks groups. It is possible to end the task for November. I make a control periodic of the SD Signal quality. About bad links, We will change some AntennaKits for High-Gain Antenna Kit to solve the problems.
WBS 4.0 Online Monitoring (Cyril Lachaud
– APC)
Three important steps have been achieved:
The computers dedicated to this task
have been bought, and are currently arrived in
Kai has written a nice code purging the MySQL bin logs in Malargüe (this will avoid the full disk that already happened in the past)
Ruben and Richard have made some important progress on the implementation of Nagios in Malargüe to monitor the whole Observatory equipment!
WBS 5.0 DPA/Offline- (Bruce Dawson, Markus
Roth and Tom Paul)
Work on the required changes for new LS trigger and the new Ct features are almost finished and will be merged onto the main development within couple of weeks. We are working on revising the Simulation Calibrator to mimic the muon histograms generated by stations in real life, in order to be consistent with newer calibration versions. The cleaned-up Geant4 simulation has been cross-checked against earlier versions and shows agreement. Some new features have been added to switch on floating point error detection for help in debugging. We have also added detection and logging of the SVN revision number used in a run, which should make it considerably easier to reproduce to bugzilla reports in the future. The offline is working successfully on some OSX platforms, but not others, and we are working to resolve this in the configuration machinery. A 2-week offline developer meeting is being organized for next month in Karlsruhe, at which time we hope to resolve a number of problems, and produce a new release. As usual, the participants list and agenda outline are posted on the wiki.
WBS 9.0 Observatory Operations –
(Julio Rodriquez Martino - INFN)
June and July had
generally bad weather. Snow storms, wind and clouds made measurements difficult
and this shows clearly in the data quality. There are few high energy hybrid
events in the
data set.
Some problems with data taking in LL due to the implementation of the
intercamera trigger. The problem was debugged and hopefully solved by H-J.
Mathes
Minor hardware problems were seen, related to power outages. LA stopped
operation during part of this period due to lack of gas for the generator. From
now on it is foreseen to have a permanent supply that will allow us to power
the building even in between shifts.
Disk space in the EyePCs was a problem, due to a problem synchronizing the
monitoring data that did not allow erasing old data from the computers. Data
taking was stopped at LL. This problem should be avoided in the future; maybe
by a larger disk space in the PCs.
In any case, data taking should be given priority over the collection of
monitoring data. Without data available, it does not make much sense to try
monitoring it.
WBS 9.0 Observatory Operations cont. – (Ricardo Sato – Southern Observatory)
The daily data taking
looks almost stable during the two months (2007/Jun and 2007/Jul). However, we
had some few problems. Bellow, a small list of some notes:
Disconnection with Loma Amarilla:
It happens mostly due to a lost connection between Coihueco and Loma Amarilla
after few days of cloudy or snow. It happen because the power supply to the
communication is made by Solar Panel and batteries. The batteries look bad.
There is a plan to replace those batteries and also to put one equipment to
charge the batteries directly with generator.
We had some few days which were very cold (in one night the temperature in
Malargue was about -19 degrees C).
In the detector beside the
Some very difficult access due to the weather condition.
There was basically three snow periods (middle of June, beginning of July and
one in the beginning of August). Because of this, the access to the field
become very difficult and this problem may extend for a quite long time in some
regions of array.
Radio Detection R&D – (Ad Van de Berg – KVI)
The OSU/Leeds
system has all stations running and continues to take data. A service trip was
made in July to recover from a power loss and to address an ongoing battery
problem. Work was done during this period to explore the possibilities of the
self-trigger. A bug was corrected in
the hardware level one trigger and is now functioning properly. The gain for
the
triggers searching for the optimal configuration.
The data-taking using the system operated by the Dutch group has met serious
problems concerning the (mains) power. Because of many power interrupts on the
mains of the BLS and a poorly functioning UPS system, not many events have been
recorded in June and July. A few trips to the BLS have been made to recover
from these power failures and work is in progress to improve the power
situation. At the same time, small changes to the system have been made (e.g.
increasing the received band width).
HEAT enhancement -Hans Klages (FZK)
Report on
progress for HEAT (Hans Klages, FZK-IK)
Preparations for the installation of three additional FD telescopes near
Coihueco (HEAT) have advanced properly.
Following advice from the HEAT Review Committee the tilting systems and
concrete foundations have been modified slightly.
Revised quotations for the HEAT site preparation work have arrived from several
vendors. We are ready to go.
Nearly all of the material and equipment needed for the new
"buildings" and their infrastructure has arrived in
One of the 3 steel frames has been erected inside an experimental hall and the
installation of shutters, aperture systems, etc. is being tested.
The first 40" container with two of the ground plates was already sent to
Malargue and was successfully unloaded by Ricardo’s team.
All additional support structures for the necessarily enhanced mechanical
stability have been defined and are under construction.
A system of measuring devices for mechanical changes, vibrations etc. during
tilting the enclosures has been designed and purchased by the
The mirror production is going on. The design work for the new HEAT readout
electronik is continued according to schedule.