Pierre
Auger Project
Progress
Report
Summary (photo album)
The construction of the Observatory is humming. The installation of the telescopes at Los Morados is proceeding very smoothly helped by the newly completed power line. As of this writing (17 February) the mirrors are all installed and mounting of the cameras has begun. Commissioning will begin in April.
We now have a presence at the at the Loma Amarilla fluorescence site. The communications tower was completed and the antennas rigged in December. Unfortunately legal problems associated with the ownership of the land have delayed the start of construction of the Loma Amarilla building.
There are now 715 tanks in the field, 683 of which have water and 652 have electronics. Good weather has speeded deployment although the distances and travel times from the campus to the detector sites continue to increase.
Plans are being made for the grand inauguration of the Auger Observatory on 10-11 November, the week before the collaboration meeting. In the morning of the first day there will be some talks on the Auger science and on the construction of the Observatory for a general audience. In the afternoon the inauguration ceremony will take place followed by a dinner. On the second day we will provide our guests with guided tours.
Other news: The construction of the new building for the James Cronin school has begun. The school is funded by a grant secured by Jim from the Granger Foundation of Chicago. The new building will provide the first permanent home for the school.
WBS 1.1 Fluorescence Detector (Jonny
Kleinfeller – Karlesruhe)
Los
Morados
At last, the air conditioning is in operation, just in time to
the fine
tuning of the mirror alignment in Los Morados.
Filters, corrector ring frames and safety curtains are mounted. As of
today, Monday
February 14, 2005, the mirrors of telescopes 10, 11, and 12 are mounted, the other telescopes will have their
mirrors
mounted by Wednesday of this week.
LAN and the slow control system are completed, all sensors are
installed,
tested, and fully operational. Shutters and curtains are integrated
into the
slow control system, the interlock are in operation, except for the HV
/
Los Morados is ready for the installation of the cameras. The cameras
are
expected to arrive at Los Morados this week.
There is no DAQ installed on the EyePC of Los Morados. The EyePC at Los
Morados
is a new system running with a newer version of SUSE, which requires
major
changes of the existing DAQ software.
The DAQ is scheduled to be implemented in March. The EyePCs at Los
Leones and
Coihueco are going to be replaced by the new type after the DAQ has
installed
successfully at Los Morados. The new EyePCs are already mounted and
configured
for the LAN at these buildings, but currently disconnected.
HV/LV cabling for the cameras is 50% ready, completion in February.
Installation of electronics is scheduled for the end of February.
Coihueco
The safety curtains have been replaced by a new design, the major
change is
the drive mechanism, a cam belt, instead of a rope. This should
overcome the
frequent misalignment of the curtains.
The missing four corrector ring frames have been mounted.
Coihueco is completed to the point that only the corrector ring lenses
are
missing.
Los
The service cranes have been installed.
As in Coihueco, the safety curtains have been replaced by the new
design.
WBS 1.1 Fluorescence Detector Calibration (John
Matthews –
The status is that the initial installation of the new Morados APF is complete. Only the wide wavelength light source is installed at this time. The control computer is in the calibration room of the Morados FD building and runs the Morados APF via a radio link. All components
have been tested and work well. Routine operation of this source awaits completion of the Morados telescopes as the FD telescopes are needed to measure the light scattered from the APF light beam(s).
The status of the Coihueco APF is that 3 sources are installed:
one wide wavelength and 2 narrow (10nm) wavelength beams centered on 334nm and 390nm. At this time only the wide wavelength source is run. It fires a set number of light pulses, across the field of view of the Coihueco FD, every hour during FD data taking.
WBS 1.2 FD digital
electronics and readout systems (Matthias Kleifges – IPE)
Installation
work on site: (by N. Kunka and
The
installation of telescopes in the Los Morados building is progressing
within
the schedule presented at the November Malargüe meeting. Main
installation
items from the FD digital electronics group are:
·
The
building is now equipped with complete LAN infrastructure. This
includes the Mirror
LAN (the LAN connecting the Mirror PCs with the EyePC) and the Eye LAN
(connecting the EyePC with all other subsystems and the
telecommunication). WLAN access is
possible and all necessary services (like DNS, time server, etc.) are
configured. The UPS of the EyePC is in place and will provide monitor
information on times of power breakdowns (if they are long enough).
·
In
all 3 buildings (Los Leones, Coihueco, Los Morados) new 19” rack
mounted RAID
computers are built up. They will replace shortly the current EyePCs.
Due to
the raid disk the system reliability is very much improved. However,
they will
operate only with newer Linux releases (we use the German SuSE 9.1
version).
The Linux system has been modified for the automatic boot procedure of
the
MirrorPCs and its file system. The boot process and system functions
have been
tested with all MirrorPCs (in LL, Co, LM).
·
An
additional raid system was installed in
·
The
SCS system is installed in LM in all 6 bays. It includes among other
things the
shutter control, the fail-safe- curtain steering, relays for the power
control
of FE sub-rack and MirrorPC, light sensors, and the interface to the
High
Voltage system. Tests of the systems with the newest SCS software are
in
progress.
·
The
BIOS settings of the MirrorPCs are stored in CMOS memory which is
buffered by a
small battery cell. In the past we had frequently problems with
non-booting
MirrorPCs caused by lost BIOS settings. In close communication with the
PC
manufacturer we got an Auger-specific BIOS which holds our
configuration as the
default one. Thus even without the battery backup the PC starts the
boot
process with the needed settings.
DAQ
and related software: (by H.J. Mathes)
The search
for bugs and the implementation of new features in the DAQ is
continuing. First
the DAQ in the current version but for the new Linux System will be
installed
in LM, but soon a new DAQ release will cure following bugs:
·
Memory
leaks in the Eye Run Control GUI will no longer crash the DAQ in longer
runs. In
addition, the robustness against communication failures of this
software part
is increased.
·
The
scheme for registering of CORBA based programs at the (3rd party) CORBA
Naming
Service (NSD) is changed to avoid continuous memory consumption by the
NSD
leading to unpredictable system crashes after some weeks.
·
The
data flow inside the FD event builder will be changed. In future
“minimum bias”
events - i.e. a fixed percentage of events rejected by the TLT but
recorded for
investigations of the trigger performance – will bypass the T3
algorithm. The
same is true for events from external light sources (CLF, APF) as these
event
types are then already tagged by the Mirror DAQ. This results in a
relief of
the TLT and T3 data throughput.
·
A
new FDEventLib will improve the compatibility with 64-bit and PPC
platforms and
implement minor extensions and corrections towards proper C++ coding.
Other
activities in IPE:
·
The
program for hardware tests of the FE sub-rack is now available for
Linux.
Online help functions are integrated. The hardware access is either
from the
EyePC via the MirrorPCs possible or from a service laptop.
·
A
service laptop (dual-boot Windows and Linux) was installed with
necessary
program for use by Primo Vitale.
·
H.
Klages has proposed to check the FD calibration by means of a captive
balloon
equipped with a flashing calibrated LED light source. We have designed
the
flying LED flasher which will be operated using a commercial radio
control and
interfaced to a GPS receiver for determination of its position.
Components for
3 devices are procured. First tests are planned in March 05.
·
We
have started activities to measure cosmic ray showers with radio
detectors. The
design of a logarithmic, periodic dipole antenna was finished and
successfully
tested in
WBS 1.2 Fluorescence Detector Electronics
(Daniel Camin –
Milano)
1) Debugging system for the HE + PMT assembly the
development of the HE debugging system is now completed. The system is
fully
portable, the software (implemented using Lab view) is installed in a
laptop
computer equipped with a PCMCIADAQ Card (National Instruments 6062E,
500ksamples/sec). The power required by the Head Electronics under test
and the
analogue part (PMT signal emulator and the differential receiver) is
delivered
by the laptop battery, via the USB connector (5 Volts, 500 mA).The
different
voltage is required by the unit under test are obtained by means of a
DC-DC
converter and linear regulators. In the same computer there is the
data-base
server in which the parameter for each HE unit are stored. The
data-base
contains the parameters measured during the production phase at the
manufacturer
plant, Elbau/Intratec, in
2) Determining the pointing of the FD telescopes. We have calculated the absolute pointing of the FD telescopes from the analysis of sky background data acquired during regular data taking periods. The method we used (see GAP-2005-008) is based on the knowledge of bright stars coordinates that provide a reliable and stable coordinate system. It can be used to check the absolute pointing of the telescopes as well as its long-term stability during the whole life of the project. We have analyzed background data taken from January to October 2004 to determine the absolute pointing of the 12 telescopes installed both in Los Leones and Coihueco. Our method is based on the determination of the mean-time of the variance signal left by a star traversing a PMT's photocathode which is compared with the mean-time obtained by simulating the track of that star on the same pixel.
WBS 2.1 SD Site (Ingo Allekote – Instituto
Balseiro)
In the two-month
period ending January 31, the SD crew at Malargue assembled 56
detectors,
deployed 67, filled 87 with pure water and brought 55 new detectors
into
operation. As a result, as of January 31, 2005, 686 SD were out in the
field,
of which 659 contained water and 602 were in operation. Tanks have been
deployed in several new areas, including along Route 40 near Coihueco.
The fourth water transport tank, Tk-12-DELTA, is
still under
construction, as the provider has reported some delay in the
manufacturing
process.
The stainless steel tanks for water transport suffered a lot under the
vibrations due to the uneven roads to Morados and needed to be repaired
a
couple of times in San Rafael.
During January, a large road grader was borrowed from CNEA San Rafael
to
improve the roads to Morados and to open new tracks to the central part
of the
array North of the
Manufacture of 134 tanks in
All the resin that was in
Resin for 209 tanks was purchased by Fermilab and shipped to
There was a problem with some of the solar panel brackets when they
were
delivered to Malargüe: The holes for mounting the panels were located
properly
for an early version of the Isofoton solar panels. The assembly crew in
Malargüe was able to provide a temporary procedure to assemble the
brackets by
relocating some holes. The manufacturing problem was traced to using
the
original drawings for the brackets. When modified drawings were used
later,
they were never officially revised and installed in the CERN EDMS
database that
Auger uses for all official drawings. The revisions have been made to
the
drawings and the revised drawings are now in the EDMS database.
Production of
solar panel brackets is proceeding smoothly, managed by Tandar and
Bariloche.
A group of solar power regulators failed during this period in the
field. The
failure was due to a manufacturing fault and was confined to regulators
in a
narrow serial number range. Nearly all of the forty or so regulators
involved
have been recovered from the field and a sample of eight has been sent
to the
manufacturer for review and replacement.
A shipment of 1024 more solar panels arrived, provided by
remainder of the Array plus spares have been ordered by
Two developmental fiberglass tanks were delivered to Malargüe and
inspected.
There were a few manufacturing errors that can be corrected in any
future
procurements. These tanks were assembled into surface detectors and are
expected to be deployed soon
WBS 2.2. Surface Detector Electronics (Tiina Suomijarvi
– IPN Orsay)
Status of
the SDE-Fabrica
The FE
and TPCB boards are the most critical items due to their small
batch sizes (typically 100). Currently 100 TPCBs are in transit and the
assembly and testing of E-kits which was stopped in December due to the
lack of
these boards, should resume next week. The small batch sizes of these
boards
are due to the lack of funding.
The
radio test bench which was not operating properly is now working and
allows to diagnostic radio field failures. A few failures of the GPS
receivers
have been detected and a test bench for the GPS cards will be installed
in the
SDE-Fabrica.
There
have been some problems in the UB shipments. Some of the boxes are
damaged when arriving to Malargue. However, no mechanical damage on the
UBs was
observed in the last shipment.
Status of
the PMT-house activities
There are currently PMTs arriving in batches
of 36 corresponding to two
different orders: the order from the collaboration and the order from
In
addition to this order by
The
failure rate in the PMT tests is low. The most frequent failure,
non-linearity, has now disappeared due to an improvement in the design
and due
to a linearity test in Photonis prior to shipment.
SDE Field
Failures
The UB
fuse failures (17 up to October 2004) were analysed by the CdF
group and Mariela (see GAP Note GAP_2005_002). An important amount of
the fuse
failures is in relation with a regulator failure (7). For each of these
cases,
after the regulator has been changed, no other failure was detected.
The measurement of
the general consumption of electronic in the field, gives a load
generally
close to 310 mA for 28 Volts voltage supply (batteries full), then for
18
Volts, the current value should not exceed 480 mA, which is under the
fuse
limit.
Figure 1. Current load and battery
voltage on the tank Selknam (from GAP_2005_002).
It seems that there has been a bad batch of
regulators which has caused
part of the UB fuse failures. For the
moment, no action is taken to increase the fuse from 500 mV to 750 mV.
There are currently 17 E-kits back from the
field waiting for further
failure analysis in the SDE-Fabrica. The most of the E-kit failures are
due to
radios and UB fuses. A more complete report on the failures will be
given in
the March collaboration meeting.
Lightning
hit
There is a
clear hit of lightning on the PIQUILLIN. The cable cover of the antenna
was
burned, radio and Ekit needed to be replaced. Now PIQUILLIN is again
taking
data.
It seems
that lightning hits are rather rare, this is a first clear case
observed. Last
year a big storm stopped 3-4 tanks but a simple reset on the UBs was
enough to
get them running again.
PMT field
failures
Walter
Fulgione, Javier and Fabian had a two weeks debug session, 6-19
December 2004 on the field failures. 44 field failures (tanks with
failure)
were studied, 17 PMTs, 5 E-kits, 6 SMA connectors and 1 radio were
changed.
Some
of the PMT failures are due to instable behaviour. The figure below
shows a muon peak as a function of time for PMT2 in tank MAITEN. 13 PMTs removed from the field were sent back
to Photonis. Further tests for these PMTs will be performed in
The
PMT field failures are carefully monitored. Some of the failures are
due to connectors and can be repaired in the field. Some PMTs present
very large
variations as a function of temperature or are very unstable. For the
moment,
no particular cause has been found for the unstable temperature
behaviour.
Typically, these PMTs present no failures when they are re-tested in
the
PMT-House after their removal from the site. Furthermore, 2 such PMTs
were
tested in the temperature chamber of the IPNO without being able to
reproduce
the unstable behaviour. A more detailed report on the field failures
will be
given in the March collaboration meeting.
Figure 2. Unstable behaviour of muon peak (PMT2 MAITEN).
Solar
panel cables (SD)
The solar
panel cables currently deployed to the field have UV-resistance
problems (see
figure below). A corrective action is
currently planned by the SD group.
Cyclone
testing
Ten
front-end boards with the Cyclone chip have been fabricated in
Seven
Cyclone front end boards were shipped to Malargue and Zbigniew and
Xavier
performed tests with the JAMIE test tank. A simple data acquisition
program was
used. Preliminary data collected from the tank for couple of days look
very
promising
The normal
data acquisition requires the new version of the UB software. Laurent
Guglielmi
is currently working on it. The plan is to run data acquisition in a
few tanks
to check the full functionality and stability of the Cyclone design
before the
Critical Design Review.
LED assembly and tests
Ygor and
Konstantin from MEPhI were
in Malargue after the November collaboration meeting to set up a
portable LED
test facility, designed in MEPhI, for the LED flashers. This facility
allows to
study main parameters of the flashers by means of the reference PMT
(which is
the part of facility) and also to check how flasher drivers provide the
linearity measurements. The PMT output pulses are analyzed by means of
the
two-channel digital oscilloscope (100 MSPS). We use signals from two
different
dynodes (from 8-th and from 11-th) to provide 105 dynamic
range.
Flexible software interface allows us to study different
characteristics of
both drivers and flasher as a whole. About 100 flashers were studied
(after the
flashers were potted). The flashers were observed to have large
differences in
the output luminosities. Currently the
flashers are used for tests and the variations in the light output are
not
crucial. However, care should be taken when flashers are used for
linearity
measurements.
SDE
deployment and maintenance
There are
currently about 687 tanks in the field, 652 of them have water and 602
are with
electronics. The electronics deployment was stopped in December due to
the lack
of TPCBs. 100 TPCBs are currently in transit from US to Malargue and
the
deployment activities should start again next week.
The
deployment is proceeding smoothly with a rate of 10 E-kits per day. Parallel to the deployment, Javier and Fabian
also do maintenance. This week a fatal failure was discovered:
batteries were
stolen (see photo album).
WBS 3.0 Comms
(Paul Clark –
Progress Achieved in Communications Task during recent December Site Visit
All the surface detector wireless LAN antennas, cable and other equipment has been installed and the tower just requires installation of the microwave link to the Coihueco tower to be completed. This will happen before the next meeting.
Software progress –
Commissioning
The comms system has suffered from an intermittent bug that has caused system availability to drop a few % below 100%. This was due to base-station radios 'disappearing from the network. This bug was found and solved during the December trip, we have not had a base-station disappear since we solved the problem in mid-December. According to site staff, all outages
experienced since mid-December have been due to power failures.
Software progress -
performance monitoring
A brand new real-time comms system performance reporting system was written and installed during the last site visit. This system allows instant analysis of any radio link in the network by ARQ (packet error) rates, signal strength or network disconnections. This new facility allows the
operator to know that all is well and allows under-performing radio links to be quickly identified. Analysis may be done on individual links or whole sectors also enabling base-station performance to be analysed easily.
WBS 5.0 DPA/Offline-
(Bruce Dawson, Markus Roth & Tom Paul)
Unfortunately Stefano Argiro departed Auger at the end of December; his DPA leadership responsibilities have since been reapportioned, with Bruce Dawson, Markus Roth and Tom Paul coordinating ongoing efforts. Bruce and Markus will act as the principle liaisons between
the offline software team the analysis tasks for FD and SD, respectively. Their role will be to work together with analysis coordinators to ensure that the physics modules distributed with the
offline framework represent the best knowledge of the collaboration. Tom will coordinate ongoing development of the framework machinery itself.
During the last collaboration meeting, a production readiness review was held. The offline team is now working to address the concerns and suggestions of the committee. The reorganization of the DPA leadership into three categories was in part meant to help address the committee's observation that the offline team and the analysis groups need to work in close concert from here on.
Since the collaboration meeting, Version 1.0.0 of the framework has been released, which includes improvements in many of the physics modules as well as the core framework code.
A program to validate the SD simulation chain and
begin a
full production run has recently got underway, with
Significant progress has also been made getting the FD simulation chain to run reliably, and it will be crosschecked in detail against the old FDSim in the near future. On the SD side, Darko Veberic and Markus have made considerable progress refining the reconstruction algorithms so
they are in good shape. New interfaces to the CDAS calibration and event selection algorithms are also nearing completion.
The offline databases for FD calibrations, SD monitoring, and some atmospheric monitoring are being filled and tested in preparation for wide scale distribution.
The framework machinery has also been enhanced with new tools to make configuration simpler and to check configuration files for common errors.
Near the end of January, a second DPA school was held in Salt Lake City, hosted by the University of Utah. It was well attended and provided a useful forum not only to introduce the offline framework to new users, but to feed back ideas and concerns to framework developers.
WBS 9.0 Observatory Operations – (Julio Rodriguez
Martino – INFN)
The analysis of the FD data, from the point of
view of the
detector performance, was improved. The dead time corresponding to
moonlight,
bad weather or hardware/ software problems is now calculated using the
DAQ log
files and the information written to the e-log by the people in shift.
The
event rate is calculated for each run and several plots are generated,
both in
PostScript and ROOT format. The event rate and the total number of
events are
plotted, both for T2 and T3 triggers. This information is available
from the FD
web page: http://www.auger.org.ar/FD/
Several improvements are foreseen and a GAP note will soon be available
containing
a detailed explanation of the analysis method. It is also proposed that
this
work is presented in the next ICRC.
Starting from January 2005, the data taking period is defined as all
days when
the moon fraction is 60 % or less. This means in practice that the
period is 2
days longer than before.
It is still necessary to establish if those two extra days are worth
from the
point of view of the data quality.
We had some problems with the data taking. Some unstable behavior of
the DAQ
system, that will hopefully be solved soon, increased the dead time
during the
past months. More important during January were several thunderstorms
that
affected the normal operation. The lightning filter that used to be
present in
the T3 algorithm is not used anymore. It must be rewritten to
avoid the loss of important events due to its malfunction. Some
hardware
problems affected also the trigger rate. It is clear that these
external
factors should be handled by the trigger algorithm in a correct way,
since they
will sooner or later be present during the data taking period. The
responsible people have been contacted and the work is being organized
to solve
this problem as fast as possible.