Machine learning Master's thesis

My master's thesis: reconstructing extensive air showers detected at the HAWC observatory

My master's thesis in Artificial Intelligence (UNIR, 2024): a model based on a convolutional neural network that predicts the core position of the extensive air showers produced by cosmic rays at the HAWC observatory.

Author
Daniel Galán Triguero
Master's
Artificial Intelligence, UNIR
Presented
Supervisor
Ibrahim Torres Aguilar
Download the full thesis80 pages · 5.8 MB · in Spanish

Abstract

Cosmic rays, high-energy particles from space, hit the Earth's atmosphere and create extensive air showers, which spread out laterally and vertically and are detected by specialised observatories. HAWC, on the slope of the Sierra Negra volcano (Mexico), detects them with water Cherenkov detectors. Most of the observatory's algorithms are optimised for gamma rays, so they are not optimal for studying cosmic rays. This work proposes using artificial intelligence to build a model tailored specifically to cosmic rays, providing one of the variables needed to reconstruct the showers: the prediction of their core position.

Main results

−38%spread of the error in X across all events (σ 37.7 vs 61.1 m)
−75%at low energies, log E 1–2 (σ 17.6 vs 69.3 m)
≈ 0 mmean bias, against 94.6 m in X and 262.3 m in Y

01

Context

Cosmic rays and air showers

A primary cosmic ray is a subatomic particle that arrives from space at close to the speed of light. When it collides with a nucleus in the atmosphere it sets off a cascade of reactions: a shower of secondary particles that widens vertically and laterally until it reaches the ground. The point where the axis of that shower meets the ground is the shower's core.

~ 15–20 km · first interaction the shower grows and widens core primary cosmic ray secondary particles muons 4,100 m · the observatory's water tanks
Figure 1. An extensive air shower. When a primary cosmic ray collides with a particle in the atmosphere it sets off a cascade of reactions that produces many secondary particles. Illustration.

How the HAWC observatory sees them

HAWC (High Altitude Water Cherenkov) sits on the slope of the Sierra Negra volcano, in Puebla (Mexico). It does not see the particles directly: it sees the light they emit as they cross the water in its tanks. When a charged particle travels through a medium faster than light does in that medium, it emits a cone of light: this is the Cherenkov effect.

7.3 m diameter 4.5 m PMTs at ~4 m Cherenkov tank 180,000 l of water 4 PMT sensors facing up Cherenkov light cone
Figure 2. When a charged particle crosses the water faster than light travels in that medium, it emits a cone of light. The four PMTs at the bottom record it: that is a “hit”.
The observatory in numbers
Altitude4,100 m
Detectors300 tanks in a grid
Area~22,000 m²
Tank7.3 m × 4.5 m, steel
Water180,000 l, ultrapure
Sensors4 PMTs per tank, at ~4 m

The problem

Locating the core is the first step of the reconstruction: the energy and direction of the primary depend on it, and with them the measurement of the cosmic-ray spectrum. But HAWC is optimised for gamma rays, its main line of research. Cosmic-ray showers are different: sensors far from the core can record sub-showers, which does not happen with gammas.

The result is a systematic error. With the current algorithm, the core of cosmic-ray showers comes out shifted by 94.6 m in X and 262.3 m in Y on average. On top of that, many showers fall partly outside the area covered by the tanks, and their core still has to be estimated.

02

Method

Data

Training requires knowing where the core really landed, and real data does not tell you that. So the model learns from events simulated by the observatory's own software, stored in XCDF files. The positions of the tanks and their sensors come from HAWC's survey.xml file.

Before going into training, each event passes a filter: valid reconstruction, angle and core (status = 0), more than 75 hits and a ratio of hits to available channels above 0.22. The final set has 4 million events, with angles between 0 and 0.60 rad and between 0 and 1,200 hits.

From an event to an image

The tanks are laid out in a grid and each one holds 4 PMT sensors, so the information of each event can be represented much like an image. Each tank takes up a 2 × 2 submatrix within a 40 × 30 matrix, and the different values of each PMT are stored in separate layers, just as RGB images store each colour in a layer.

The water tanks of the HAWC observatory, steel cylinders with canvas roofs laid out in a compact grid on the slope of the volcano, with mountains in the background. A HAWC event seen from above: each tank is a circle with its sensors; those that recorded light are coloured by the time of the hit, clustered around the core.
Figure 3. Left: HAWC's 300 tanks, laid out in a grid. Right: an event seen from above (E = 41.5 TeV, θ = 31.5°); each circle is a tank with its PMTs, and the colour shows the time of each hit. That grid layout is what carries over to the matrix. Source: HAWC Observatory.
1 · Tank, 4 PMTs 2 · 2 × 2 block 3 · 40 × 30 × 4 matrix layers: energy · angle · PMT X · Y Only cells with a hit carry values
Figure 4. The event becomes an “image”. Each tank takes up a 2 × 2 block in a 40 × 30 grid that respects its position on the ground, and each sensor variable goes into a layer, like the R, G and B channels of a photo.

The four layers are the energy, the zenith angle of the shower (the same for the whole event) and the X and Y position of each sensor relative to the centre of the observatory. Only cells with a hit carry values. Building the set costs O(n × m), with n events and m hits per event: the position of each hit in the matrix is computed in O(1).

Architecture

The goal is to predict spatial values, the coordinates of a position on a plane, and the input represents values on that same plane. That is why a convolutional neural network was chosen: it combines Conv2D layers with MaxPooling, followed by a dense layer that predicts the coordinates by linear regression.

LayerConfigurationPurpose
Conv2D128 filters 3 × 3 · Leaky ReLUDetects spatial patterns
MaxPooling2D2 × 2Reduces dimensionality while keeping what matters most
Conv2D256 filters 3 × 3 · Leaky ReLUDetects spatial patterns
MaxPooling2D2 × 2—
Conv2D256 filters 3 × 3 · Leaky ReLU—
Flatten + Dense256 neurons · Leaky ReLUTurns the maps into a vector and processes it
Dense2 · linearPredicts the (x, y) coordinates by linear regression

Unlike ReLU, Leaky ReLU does not turn negative values into zero: it scales them by a small factor. That avoids vanishing gradients, which matters when the input data contains relevant negative values.

Training

The model was trained for 30 epochs, tracking the loss and the mean absolute error (MAE). On the training set both go down steadily. On validation they stall early and oscillate.

Loss and mean absolute error curves over 30 epochs with 4 million events: training goes down, validation stalls and oscillates. Loss and mean absolute error curves with 1,000 homogeneous events over 15 epochs: training and validation move together.
Figure 5. Left: training with 4 million events (30 epochs). Right: a test with 1,000 similar events (15 epochs), where training and validation move together. Figures from the thesis.

To test that hypothesis I repeated the training with 1,000 similar events: angles between 0.45 and 0.50 rad and between 600 and 800 hits. With homogeneous data, the validation metrics behave much more like the training ones, and the training ones are significantly better. The thesis puts the stalling down to the high variability of the data: one event can be very different from another because of its position in the observatory, the energy of each hit or its inclination.

03

Results

Does it get the distance to the centre right?

To compare the prediction with the true value at a glance, the distance from the core to the centre of the observatory is used, r = √(x² + y²). If the model were perfect, every event would fall on the diagonal y = x.

2D histogram of true r against predicted r for all events: most sit on the diagonal, densest near the origin.
2D histogram for log E between 1 and 2. 2D histogram for log E between 2 and 3. 2D histogram for log E between 3 and 4. 2D histogram for log E between 4 and 5: the spread around the diagonal is larger.
Figure 6. 2D histograms of true r against predicted r: all events (left) and by energy level, from log E 1 to 4 (right). The higher the energy, the larger the spread. Figures from the thesis.

Most events sit on the diagonal, especially near the centre. The spread grows with distance and with energy: the further away the core lands, the less information reaches the tanks.

Precision: the spread of the error

For each event the true coordinate is subtracted from the predicted one and a Gaussian is fitted to those differences. Its width, σ, measures the precision: the lower, the better.

HAWC Convolutional network X coordinate 020406080100 1.52.02.53.03.54.04.5 Sigma log(E) log E 1–2 · HAWC: σ 69.34 log E 1–2 · Convolutional network: σ 17.60 log E 2–3 · HAWC: σ 48.45 log E 2–3 · Convolutional network: σ 19.75 log E 3–4 · HAWC: σ 52.56 log E 3–4 · Convolutional network: σ 28.50 log E 4–5 · HAWC: σ 101.31 log E 4–5 · Convolutional network: σ 72.47 Y coordinate 020406080100 1.52.02.53.03.54.04.5 Sigma log(E) log E 1–2 · HAWC: σ 67.95 log E 1–2 · Convolutional network: σ 17.92 log E 2–3 · HAWC: σ 50.93 log E 2–3 · Convolutional network: σ 18.70 log E 3–4 · HAWC: σ 53.71 log E 3–4 · Convolutional network: σ 29.03 log E 4–5 · HAWC: σ 102.41 log E 4–5 · Convolutional network: σ 75.46
Figure 7. Relationship between the event energy and the sigma value, for X and Y, as in figure 36 of the thesis. The lower, the more precise. Data from table 6.
log Eσ X HAWCσ X networkImprovementσ Y HAWCσ Y networkImprovement
All61.137.7−38% 62.538.5−38%
1–269.317.6−75% 68.017.9−74%
2–348.519.8−59% 50.918.7−63%
3–452.628.5−46% 53.729.0−46%
4–5101.372.5−28% 102.475.5−26%

σ in metres. Table 6 of the thesis; the improvement is (1 − σ network / σ HAWC).

Bias: where the error is centred

The mean of those differences tells you whether the model always errs in the same direction. This is where the two methods differ most clearly.

Current HAWC (mean ± σ) Convolutional network (mean ± σ) X coordinate -100 0 100 200 1.01.52.02.53.03.54.0 log(E) 1–2 · HAWC: mean 93.3 m, σ 69.3 m 1–2 · CNN: mean -1.9 m, σ 17.6 m 2–3 · HAWC: mean 96.9 m, σ 48.5 m 2–3 · CNN: mean -0.6 m, σ 19.8 m 3–4 · HAWC: mean 93.4 m, σ 52.6 m 3–4 · CNN: mean -2.3 m, σ 28.5 m 4–5 · HAWC: mean 92.6 m, σ 101.3 m 4–5 · CNN: mean -12.4 m, σ 72.5 m Y coordinate -100 0 100 200 300 1.01.52.02.53.03.54.0 log(E) 1–2 · HAWC: mean 265.6 m, σ 68.0 m 1–2 · CNN: mean -2.2 m, σ 17.9 m 2–3 · HAWC: mean 263.3 m, σ 50.9 m 2–3 · CNN: mean -0.5 m, σ 18.7 m 3–4 · HAWC: mean 261.4 m, σ 53.7 m 3–4 · CNN: mean 0.3 m, σ 29.0 m 4–5 · HAWC: mean 261.8 m, σ 102.4 m 4–5 · CNN: mean -0.7 m, σ 75.5 m
Figure 8. Relationship between energy and the mean of the differences (point), with its sigma (bar), for X and Y, as in figures 37 and 38 of the thesis. Data from tables 6 and 7.

HAWC's algorithm shifts the core by about 93 m in X and about 262 m in Y at every energy level. The network stays centred on zero: between −3.4 and 0.3 m in every case except X at the highest energies (−12.4 m).

04

Conclusions

  • Goal. Improve the prediction of the core coordinates of the extensive air showers detected at the HAWC observatory.
  • Methodology. A convolutional neural network that, without having images of the events, arranges their data in matrices that mirror the geographical layout of the sensors, with each detector's parameters in layers, as in colour images.
  • Results. The model predicts the coordinates more precisely than HAWC's current model, with a significant reduction in error, especially for events with a log energy below 4. HAWC's model shows larger errors at every energy level.
  • Conclusion. The convolutional neural network outperforms HAWC's model in precision, validated with histograms and visual comparisons.

Future work

Two lines: training with more resources and more events, and training several models specialised in similar events (by arrival angle, energy and number of hits), choosing the most suitable one for each case at prediction time.

Read the full thesis

The 80 pages cover the background on high-energy astronomy and machine learning, the requirements and design of the tool, the full evaluation and the per-energy histograms in the appendices.

Download the PDF (5.8 MB, in Spanish) ↓
Technical details and how to cite it

Stack: Python 3, TensorFlow/Keras, scikit-learn, XCDF, Matplotlib.

How to cite: Galán Triguero, D. (2024). Reconstrucción de cascadas atmosféricas extendidas detectadas en observatorio HAWC [Master's thesis, Máster Universitario en Inteligencia Artificial]. Universidad Internacional de La Rioja.

If you'd like to talk about it, write to me.