Pierre
Auger Project
Progress
Report
Summary (photo album)
Three of the six telescopes at Los Morados recorded their first showers at the end of the March dark period and were in operation for the April dark period. The remaining three telescopes at Los Morados are being commissioned. All six need drum calibration. Los Morados DAQ is running on a new code version that is still being debugged. The Los Leones and Coihueco DAQ systems will be updated once the new code is commissioned.
There are 793 surface detector stations in the field. Of these 764 have
water and 697 electronics kits. In spite
of considerable effort the rate of detector assembly and water delivery remains
well below that needed to finish the array in a timely way. We are investigating the possibilities for
obtaining more equipment and people.
The last of the communications equipment has been installed and commissioned at Loma Amarilla. The tanks now being deployed in the north of the array can now be brought on line. A number of antennas on tanks already in operation are being reoriented to Loma Amarilla.
Considerable instability remains in the data collection for a verity of reasons. The sources of instability are mostly understood and solutions are being found.
We are also plagued by a decreasing data rate from the Auger
campus to
A number of events are being planned in association with the World Year of Physics are being planned. Among these is the Auger Celebration to be held 9, 10 November. The Auger Celebration marks approach to completion of the observatory and the presentation of the first physics results.
WBS 1.1 Fluorescence Detector ( Jonny Kleinfeller –
Karlesruhe)
The status of the FD sites has not changed much since the
last collaboration meeting.
We have three telescopes in Los Leones and in Coihueco equipped with corrector
ring lenses, the lenses for Los Morados (promised for March 2005) have not yet
arrived, but have been dispatched to Mendoza (C. Escobar).
At Los Morados are three telescopes in operation.
The new software has been installed on the eyepc. This software is identical to
the software installed at Los Leones and Coihueco as far as operations is
concerned, but adapted to the new LINUX operating system. Unfortunately most of
the bugs have been transferred as well.
The a-calibration fibres have been installed.
Los Morados participates in the current shift with three telescopes
(calibration and data taking).
Currently the remaining three cameras at Los Morados are being installed
(mercedes, positioning, alignment).
I expect those telescopes to take data at least in one or two nights of the
April dark period.
I have modified the fail safe-curtains in all buildings to increase
reliability, seems to work.
The cloud camera for Los Morados is scheduled to be installed at the end
of June.
WBS 1.2 FD digital electronics and readout
systems (Matthias Kleifges – IPE)
Installation work on site:
We have
installed the FE sub-racks and the MirrorPCs for telescopes # 4, # 5 and # 6 in
the Los Morados building. These 3 telescopes took data in test mode (no T3 to
CDAS) for the first time during the March shift at 17/18 March. Electronic
gains and the HV values will be adjusted as part of the next drum calibration.
Although the telescopes were not well adjusted, laser events from CLF as well
as cosmic showers could clearly be identified.
As the
adjustment of the cameras in bays 1 to 3 was not finished till the end of our
stay, we only connected the MirrorPCs (MPC) and the FE sub-racks to the AC
power from the SCS. Tests with the connected camera were not possible, but the
MPCs boot procedure and the access to the FE crate was proofed to be working.
Other activities include following work:
·
A
time calibration was performed for all telescopes in LL and Co (LM was
unavailable at that time. The measurement uses the LCU to produce a LED pulse
(semi-sinusoidal shape) and distribute it through fibre A to all telescopes in
parallel. Taking into account cable delays and the light transit time in fiber
A we derived stable calibration constants for both buildings (Co and LL) of
about 315...320 ns for bays 2-5 and about 285...290 ns for bays 1 and 6 (due to
longer cables from the GPS clock in these bays).
·
IPE
has produced the electronic for a flashing calibrated LED light source which is
fixed on a captive balloon. H. Klages and his crew performed first promising
tests and record data with one telescope in LL. The analysis showed pulses in
the virtual channel, but the normal channel saturated due to a factor 30 too
high intensity. More measurements are planned with a modified setup.
·
Most
of our time was devoted to find a bug in the DAQ readout system. For still
unknown reasons the access to the front-end through the FireWire interface is
sometimes inhibited, especially while performing calibration B and C (less
frequent also calibration A). In this case, processes on the MPC stop and
access to the hardware is no longer possible. The error diagnostic is very
difficult as the problem occurs only intermittent on site and cann’t be
reproduced in
Information on the web claim an internal timing problem of the FireWire chip we
have in use. In order to exclude the timing problem we have changed the
firmware for the FireWire interface. The new version 4.7 may have reduced the
frequency of occurrence of the problem, but didn’t let it disappear. We hope to
get more information from the experience of the April shift.
·
Adjacent
telescopes in LM and Co are now interconnected by a twisted pair cable.
Although the inter-telescope trigger is not yet supported by the DAQ, tests are
now possible due to this cable.
DAQ and related software: (by H.J. Mathes)
Since the
April shift the DAQ is identical for all FD buildings; the same DAQ code is
running under Suse 9.1 in LM and under Suse 7.3 in LL and CO. The difference to
the old DAQ versions is:
·
FDEventLib
v2r6p1 implements the latest data format.
·
The
DAQ controls the inputs of the GPS clock. Depending on the DAQ mode
(calibration or shower measurement), the relevant inputs on the GPS clocks are
enabled. The veto signal of the LIDAR is only enabled during shower
measurements.
·
In
order to avoid large T3 rates at CDAS several filters are implemented in the
data path. 1.) LIDAR events are recognised from their GPS sub-second time and
rejected. This however is only possible if the LIDAR is triggered by the FD GPS
clock. We recommend applying a LIDAR hardware veto which makes this filter
redundant.
2.) We apply a fixed software dead time after each T3 trigger send to CDAS.
Events recorded during this software dead time are saved to disk without
sending a T3 to CDAS. This filter is not yet enabled, as a similar rejection is
implemented in CDAS software.
WBS 1.2 Fluorescence Detector Electronics (Daniel Camin –
Milano)
FDE-Analogue
1) The development of the debugging system for the Head Electronics is
progressing. The final version of the software (developed in Labview) is now
released. This version comprises an optimized graphical user interface and some
corrections of minor bugs. The design and fabrication of a printed circuit
board for the PMT signal emulation circuit and the HE test fixture is underway.
The system is able to perform 250 test ( two hours in continuous run) when the
laptop is operated only in battery mode.
2) We have developed a procedure to determine the absolute pointing of all
FD telescopes by looking to the signals left by stars, superimposed to the
baseline over many months. The difference of the actual pointing compared to
the specified values is, at most, 15 arc minutes in elevation and 13 arc
minutes in azimuth. A modification in the elevation of telescope #3 in Cohiueco
was "seen" by the programme as a difference of 0.13 deg in elevation.
This program allows checking the long-term stability of the telescope's
pointing.
3) The people of the Naples INFN Electronics Laboratory are strongly motivated
to contribute to the construction of parts for the future upgrades. The Naples
Electronics group counts with about 15 people with large experience in the
design of boards that include Programmable Logic Devices. They are willing to
participate in the construction and test of analogue boards as well. Their Lab
is well equipped. We are in close contact with them and will interact
more closely as soon as the upgrades start to be defined.
WBS 2.1
SD (Ingo Allekote – Instituto Balseiro)
In this period the SD crew at Malargue assembled 62 detectors,
deployed 79 tanks, filled 73 with pure water and brought 87 new detectors into
operation. As a result, as of March 31, 2005, a total of 765 SD were deployed
in the field, of which 731 contained water and 686 had electronics installed.
Due to difficult terrain conditions in the area north of the
with the water transport trucks and front loader, the water deployment did not
proceed as fast as expected.
Three tanks in the interior of the array, with extremely difficult access
during the last two years, could finally be filled with water: Lisa,
Maria-Maria and Castelao.
Progress has been made in the manufacture of the fourth water transport tank,
Tk-12-DELTA. It
has been finished on April 5, 2005, and was delivered to Malargue on April 8th.
All accessories are already available, so it is expected to enter in operation
very soon.
On March 29, two days in advance of schedule, liner production was completed at
the Liner Production Facility at UTN Mendoza. A total of 1501 liners were
produced there, the quantity required to complete the array plus spares is now
available. The liner facility will be dismantled and returned to the
Brackets are being produced at Equipos y Proyectos in
Rotopol completed the order of 200 Battery Boxes and pipes, which were all
delivered to Malargüe.
Fermilab has submitted an order for resin for 309 tanks, to be divided
between
A battery testing and monitoring system is being implemented at the
Observatory. It includes two levels of testing: one at the assembly building,
under reception of the batteries from
the manufacturer, and another one which is done with data collected from
batteries installed in the field. The manufacturer is supposed to deliver
batteries in Malargue at 100% charge. In the first level, a fraction of the
batteries is tested in the
The second level test is carried out with data acquired by the CDAS monitoring
package. The data from each battery in the field is observed every day from 0AM
to 4AM. During this
period the batteries should be smoothly discharging, providing electric power
to the detector without sun light. Preliminary results of the off-line
monitoring are shown at
http://www.cbpf.br/~lsd/Bat/Bat_1.htm.
In the near future, more information will be added to this web site, and an
alarm system shared with Bugzilla and BigBrother will be set up, so that
failures are detected and correcting
actions are timely planned.
WBS 2.2 Surface Detector Electronics (Tiina
Suomijarvi – IPN Orsay)
The deployment of the Ekits goes smoothly.
Special care should be taken with Black Widows who have invaded tanks. There are
currently about 3-4 per tank.
The deployment was stop for a while due to the
delay of the FE board shipment. The shipment arrived a few days ago and the SDE
deployment will resume shortly.
Walter Fulgione will be on site in May and a
major campaign of PMT maintenance will take place.
About 60 % of the PMTs have now been received in Malargue. The rate
will continue 144 per month.
The new master order has been sent to Photonis
for the rest of the production.
Figure 1. PMT statistics by Federico Suares.
If the cable failures are ignored (they can be repaired on site), the
PMT failure rate on the field is about 3%. The majority of problems are early on bathtub curve (infant mortality). Two
PMTs failed after >1.5yrs in the field (they are to be recovered and
tested). Some problems have not been fixed in a timely fashion, a couple of the
problems have been “fixed” multiple times. A PMT batch with temperature
problems to be sent back to
The Ekit failures on the field
are large, about 5% for the UBs. The distribution of the failures among
different components is shown below.
Figure 2. Ekit failures.
Most
of the UB failures are due to blown fuses. As a corrective action, the fuse size
has been increased and the failures are being carefully monitored. The “not
detected” failures are mostly failures to obtain a radio connection. This is
probably due to some installation problems and has been fixed with new
procedures.
The figure below shows the
trigger uniformity.
Figure 3. The trigger
uniformity.
The tanks that have been
recently installed have higher ToT rate than the older ones. A stabilization
time of a few months is required before the trigger rate falls into the normal
rate of about one Hz.
Figure 4. Stabilization of the
ToT trigger rate.
The muon spectra are carefully
monitored. The following figures show the results of the monitoring.
Figure 5. the RMS over mean
values for PMTs for different time periods.
Figure 6. The RMS over mean
values for the areas of the muon signals.
An anomaly is observed for the
period of January to February. The table below shows the results of the
monitoring.
The high dispersion of the VEM
area values during the period of January-Frbruary still needs to be understood.
The good news from the UCLA long
term monitoring are:
•
~90% of tanks
have smooth behavior:
–
<10% loss in
Area/Peak ratio at the beginning.
–
Characteristic
time is ~ ¼ year.
–
~1% of seasonal
variation, strictly correlated to annual temperature change.
•
These tanks will survive
for 20 years!
–
Initial quality
loss happens within ¼ year.
•
Water quality or
•
Tyvek
reflectivity
–
Then it goes into
extremely stable condition.
Concerning the fabrication of SDE components there are
also some good news:
•
KVI (
•
The Cyclone boards have been successfully tested in
Malargue and a critical design review
will take place during the analysis meeting in Orsay (June 6-10).
WBS 3.0 Comms (Paul Clark –
We passed a
number of major milestones in the Comms task recently
19th
March - Completion of the last Communications tower
March 19th
saw the installation of the last piece of Comms hardware (a 1.2m microwave
dish) on the last tower in the backbone system. All 5 towers are now fully
operational with 28 base-station sectors available for the surface detector
array (capable of supporting 1680 surface detectors) and a 2Mbps dedicated link
available for each FD eye building.
April 6th
– last radio leaves the test chamber – final hardware shipment imminent
Last week the
last radio left the test chamber at
Comms
procurement completed
All the required
components for the SD Comms system for the full array have been purchased and
the vast majority of equipment is actually in Malargue now. The final shipments
will arrive at the site over the next couple of months.
Surface
Detector Antenna Team
Jorge’s team
continue their good work for the Comms task in Malargue;
At the
moment they have several programmes of work underway
1. Antenna Kit Preparation – the total number of kits made is now 915 (with the hope of reaching 1000 in May)
2. Antenna Kit Deployment – (performed
by the water teams and managed by Jorge) – 740 installed in total
49 Kits were mounted during January
38 during February
39 during March
3. Antenna Heatshrink ultraviolet protection programme – this campaign aims to make sure that no heatshrink on the antenna system is directly exposed to the sun. This has required retro-fitting protective tape to a total of 275 kits – current progress, to date 116 kits have been protected.
4. Antenna Re-orientation programme – with all comms towers now complete and the array filling up, it is necessary to re-orient ~100 antennas to their final tower – these antennas were pointed to an interim tower during construction when the correct tower was unavailable. A plan is in place and future reports will provide details of progress made (how many done and to-do).
WBS 4.0 Central Data Acquisition System - (Antoine
Letessier-Selvon – LPNHE)
The following are the current CDAS problems:
1] NFS server (network file system service) / disk space crisis
2] NTP server (network time protocol service). Broken Antenna
3] Ik/Cm (Information kernel, Communication manager) saturation
4] Fd shifts (Trigger problems)
5] Network access to
And here are their present status
1] New RAID array hardware bought and installed. Compatibility problem with NFS
solved.
Data storage partition is now 551GBytes less than 40% used.
2] Problem forwarded to the on site maintenance (Ruben). No solution yet. Need
to buy a new antenna.
3] Hopefully solved. This ultimate part of the code that was not written by
CDAS team members has been entirely recoded by Patrick and Xavier.
4] Several actions taken on the Fd side. On CDAS side a new veto with better
characteristics was
implemented (see Xavier message as above). Also some of the comms bug we had
were solved.
5] Link is desperately poor (average 300kbits instead of 1000) but amount of
data sent to
WBS
5.0 DPA/Offline- (Bruce Dawson, Markus Roth and Tom Paul)
During the past two months, the
offline development team has focused on addressing the issues raised at the PRR
held last November, especially in the areas of code stability, release
frequency, ease of installation and database filling.
A new set of release procedures has been adopted, as described in detail at https://www.auger.unam.mx/AugerWiki/ReleasePolicy.
Briefly, releases will henceforth be categorized as either "beta" or
"production", with beta releases occurring at regular intervals
(generally bimonthly) and production releases being generated as needed.
Specific milestones
are attached to each beta release and tracked via the offline bugzilla
system. This helps the development team prioritize its workload, plan
specific release dates, and assess progress. The production releases are
meant to be prepared in close consultation with relevant analysis
coordinators, and will be split off onto separate CVS branches. Keeping them
separate from the main development trunk insulates production code from
day-to-day work of the development team and thus ensures stability of the code
used for actual physics analysis. A first implementation of these new
procedures has been carried through, with beta release 1_1 cut in March and a
first SD simulation production release prepared in February in close collaboration
with
To help ease installation of the code, a new script has been prepared which
builds all the required external packages, including Geant4, as well as the
offline code itself. The installation instructions have also been
substantially revised, and are now split into a short section for beginners and
a detailed section for advanced users who may wish to customize their
installation or build directly from the CVS repository. Scattered web
pages have been consolidated into one wiki-based Offline portal.
The offline databases have undergone some further revisions, and are being
populated with data. The molecular databases now contain
Development of modules has also progressed well. Particularly important, the
full hybrid simulation chain is now operational. The CDAS team has
provided modules which allow one to directly invoke the event selection and
calibration codes of the CDAS package. Support for CDAS v.4 has also been
prepared and will be available in the next beta release. The Offline Modules
area now contains 20 contributed module directories.
WBS
8.0 Education and Outreach (Greg Snow –
Construction begins for the new James Cronin
School
A groundbreaking ceremony for the new
James Cronin School of Communication, Art, and Design was held in Malargüe on
February 7, 2005. The 822 square meter facility is foreseen to be completed in
about 6 months and will replace the present Escuela 4-190 that has an
enrollment of approximately 200 secondary students. The new building is made
possible, in part, by a generous donation from the Grainger Foundation in the
World Year of Physics events in Malargüe
A number of events in Malargüe and
“The World Year of Physics, an Event
for Everyone” in the Convention Center on March 15. The city of
Preparations for upcoming events
During the March collaboration
meeting, planning and preparations began in earnest for two major events in
November. The first is the Auger Observatory Celebration, November 10-11, to
which many dignitaries, representatives from supporting agencies and
institutions, special guests, and the press will be invited. The two-day event
will include an official ceremony, a number of talks about the Observatory, and
tours of the site. The second event is an Auger-sponsored science fair for
WBS 9.0 Observatory Operations – (Julio Rodriquez Martino - INFN)
The FD trigger problems continued
during February due to a large number of thunderstorms seen in the vicinity of
the site. The situation was far more stable in March. As a temporary solution,
a trigger veto was implemented from the CDAS side, to avoid that the high rate
coming
from the FD during these particular data taking conditions crashes the system.
The FD T3 algorithm is being improved. I have already programmed a new version
that is being tested off-line and that will also be evaluated using
simulations, to study its impact on the detection efficiency.
A program to analyses the FD run data was written. It tests the quality of the
events using different parameters. The description of the program and the
analysis method will be part of a GAP note, also proposed as an ICRC poster.
Results from this program will soon be displayed in the FD web page, as a way
of monitoring the data quality.
Transfer of LIDAR and CLF files to