The 35th annual conference on Astronomical Data Analysis Software & Systems 2025 in Görlitz, Germany

Since 2004, the Keck Observatory Archive (KOA) has operated as a NASA-funded collaboration between the NASA Exoplanet Science Institute( NExScI) and the W.M. Keck Observatory. It ingests, curates and serves all data acquired by the twin 10m Keck telescopes on Mauna Kea, Hawaii. In the past three years, KOA has begun a modernization program that will replace the architecture and systems used since the archive opened with a new modern Python-based infrastructure. This infrastructure will position KOA to respond to the rapid growth of new and complex data sets that will be acquired by new instruments now in development, and enable follow-up to identify the deluge of alerts of transient sources expected by new survey telescopes such as the Rubin Observatory. Since 2022, KOA has ingested new data in near-real time, generally. within one minute of creation, and makes them immediately accessible to observers through a dedicated web interface. The archive is now deploying a new scalable Python-based, VO-compliant query infrastructure built with the Plotly-Dash framework and R-tree indices to improve performance (x20).

The project described here exploits the new query infrastructure to develop a dashboard that will return live metrics on the performance and growth of the archive. These metrics assess the current health of the archive, and guide planning future hardware and software upgrades. This single dashboard will enable, for example, monitoring of real-time ingestion, as well as studying the long-term growth of the archive. Current methods of gathering metrics, in place since the archive opened, are inadequate to support the archive as it scales going forward. These methods suffer from high latency, are not optimized for on-demand metrics, are scattered among various tools and cumbersome to use.