The 35th annual conference on Astronomical Data Analysis Software & Systems 2025 in Görlitz, Germany

As astronomy data sets become larger, efficient data ingestion systems are required to ensure science ready data products are promptly available to the community. Within Data Central’s Ingestion system, bottlenecks were identified with the use of py-spy and diagnostic queries against the ingestion database. Rectifying inefficient database usage resulted in an 8 times speed up of some ingestion components. This presentation covers the identification and rectifying of these bottlenecks, some useful features and tricks to be aware of when using py-spy and the importance of proper logging and metrics of long running systems.