The 35th annual conference on Astronomical Data Analysis Software & Systems 2025 in Görlitz, Germany

Modern radio telescopes such as LOFAR2.0 generate enormous data volumes that are too large to be inspected manually. These data contain a wealth of transient and variable phenomena, but their scale requires automated detection methods. One of the objectives at ASTRON and LOFAR ERIC is to automatically search all upcoming LOFAR2.0 observations for possible radio transients. To accomplish this we are building on methods proven to be effective in the search for transients pioneered in the LOFAR Transients Key Project over the previous decade. To meet the performance and scalability requirements of modern telescopes the pipeline architecture is redesigned, still under a permissive open source license.

As observations are made a lightcurve catalogue is continuously updated, aiding in the search for variable sources in the time range of seconds to years. The pipeline is meant to serve as a filter, allowing researchers to query the resulting database to identify potential transients based on properties like time, space, flux density and variability metrics. This allows researchers to focus their efforts on interpretation and discovery, rather than on processing.

Key challenges include LOFAR’s distributed data-processing model, where data is stored and processed across multiple archive locations and data may become available for processing in a different order than they were observed. Our framework is designed to accommodate this asynchronous environment, ensuring that detections are incorporated into the source database as soon as they are processed. Performance of the transient detection has been significantly improved by using Dask for parallel task scheduling and by moving computational logic from the database to the application.

By combining large-scale automated filtering with flexible researcher-driven querying, this approach transforms LOFAR2.0’s high data volume into an opportunity for rapid and scalable discovery of the dynamic radio sky.