The 35th annual conference on Astronomical Data Analysis Software & Systems 2025 in Görlitz, Germany

This contribution reflects on 25 years of experience developing and maintaining automatic data reduction pipelines and related software at the European Southern Observatory (ESO). Over this period, the technology landscape has evolved from early MIDAS-based tools to systems built on the Common Pipeline Library (CPL) and the High-level Data Reduction Library (HDRL), and more recently toward Python-based frameworks such as PyCPL and PyHDRL.

Originally designed and implemented as quick-look tools to support Very Large Telescope (VLT) operations, the pipelines later evolved to address the broader needs of the user community by generating science-ready data products. This evolution naturally involved a larger group of stakeholders in verification, validation, and further development.

Despite changes in technology and an improved review process using multiple milestones and the involvement of multiple stakeholders, many core challenges and design principles have remained consistent throughout the projects' life cycles—from establishing close collaboration with consortia delivering initial versions, to continuous cooperation with all stakeholders through design, implementation, commissioning, and long-term operations. Key lessons concern the importance of full commitment and valuing complementary contributions from all participants, identifying and prioritizing areas for improvement, and adopting modular, reusable, and sustainable software architectures with well-defined interfaces between scientific and engineering teams.

This poster summarizes these lessons and how they have shaped the evolution of ESO's pipeline ecosystem, offering guidance for future projects in astronomical data reduction and analysis.