In science, the lifecycle of software products is typically managed with limited resources while facing unlimited demand. Scientific software requirements are necessarily often dominated by internal project specifications and deadlines, but these internal priorities, while beneficial for the community as a whole, do not always align with the individual needs of our ultimate customers: general...
The Astro Data Lab science platform recently marked eight years of operational service, a significant milestone in the fast-evolving domains of big data research, software development, and computational infrastructure. Initially designed to host and analyze data from the Dark Energy Survey, Data Lab has expanded its scope far beyond these (modest) first goals. Now integral to the success of...
LOFAR is a high throughput data facility facing several non-trivial technical
challenges in data processing and storage. Since start of science operations, LOFAR has accumulated over 50 petabytes of data in its science data archive. Following a major upgrade of the instrument, it is expected that over the course of the next five years of operations the archive will grow to well over 100...
Astronomy’s data lifecycle is no longer defined solely by storage and processing technologies, but equally by the social and organizational structures that enable long-term usability, interoperability, and trust. Within ESA’s ESAC Science Data Centre (ESDC), we face these challenges at scale through missions such as Gaia and Euclid, and in the development of the Euclid Data Space (EDS)...
