Successful seminar - Big Data Processing in Astronomy and Earth Observation

Earth Observation

We look back at a very successful seminar "Big Data Processing in Astronomy and Earth Observation", that took place 23rd of January 2024 at SURF in Utrecht. Over 60 participants from both scientific domains gathered to exchange knowledge and discuss interdisciplinary challenges.

323

0 Praat mee

The seminar, organised by SURF and the Netherlands Space Office, brought together the Astronomy and Earth Observation communities to explore shared challenges and synergies related to Big Data.

Experts from the two scientific domains presented impressive projects, dealing with big data from satellite missions and ground-based observatories. The presentations can be downloaded through the title links in the programme below. Find the detailed programme with abstracts here.

Programme

  Session chair Haili Hu (SURF)
10:00 Opening Maurice Bouwhuis (SURF)
10:15

Measuring Air Quality and Greenhouse Gas Emission from Space using TROPOMI: Past, Presence and Future

 Jochen Landgraf (SRON)
10:35

Gaia: the two billion star surveyor

 Anthony Brown (UL) 
10:55

Serving satellite observations of the Earth system

 Ad Stoffelen (KNMI)
11:15 Coffee break  
11:45

Real-time data processing for the BlackGEM Array of telescopes

 Steven Bloemen (RU)
12:05

(Atmospheric) Clouds of Data: Ruisdael Observatory and Beyond

 Jordi Vila-Guerau de Arellano (WUR)
12:25 Data Intensive Radio Astronomy with LOFAR and SKA John Swinbank (ASTRON)
12:45 Lunch  
  Session chair Irene Bonati (SURF)
13:45 Gravitational wave data challenges for LISA Gijs Nelemans (RU)
14:05

Remote Sensing - Deployable Analysis Environment: power your workflow with HPC

 Meiert Grootes (eScience center)
14:25 The National Research IT Infrastructure Natalie Danezi (SURF)
14:45 Break-out sessions Moderator: Daniela Huppenkothen (SRON)
  Session chair Jolien Diekema (NSO)
16:15 Global Water Watch Jaap Langemeijer (Deltares)
16:35 The Euclid NL Data Center Edwin Valentijn (RUG)
16:55 Closing & Drinks  

The question of the day was: "What data-related challenges must we solve in both astronomy and earth observation projects in order for the science to be successful?". The participants' input formed the topics of the break-out sessions, where the goal was to identify shared challenges.

    poster
    Break-out topics Identified shared challenges
    Interoperability
    • How to maintain high quality (meta)data?
    • Hard to spot/navigate examples of successful implementation
    • Difficult to keep up with speed of new tools/standards
    • More data complexity (volume, variety, etc.) requires more complex infrastructure which can be overwhelming and hard to solve
    Long-term data and software storage and access
    • Funding on long-term scale
    • Responsible entity
    • What data and software to preserve
    • Mindset
    Efficient processing
    • Long-term versus short-term - upgrading old software is hard
    • Diversity of processing means - optimization takes a long time
    • Stubbornness! Being used to your silo. Not wanting to be a computer scientist.
    • Building interdisciplinary collaborations
    • Complex data requires specialist knowledge
    • Define shared performance metrics: cost, energy, runtime. Include them in proposed evaluations.
    • Consider other environmental impacts, e.g. water use - total costs
    AI in data processing
    • Training data
    • Low computational speed can slow pipeline
    • Testing
    • Licensing code generation by AI tools
    Deriving knowledge from data
    • How to approach the knowledge gap (technical and content) between consumer and producer of the data?

    The day was concluded with drinks, where possible follow-up meetings and collaborative initiatives were discussed. We hope that this day inspires many more conversations that cross domain boundaries!

    Organisation Committee

    • Haili Hu (SURF)
    • Raymond Oonk (SURF)
    • Irene Bonati (SURF)
    • Jolien Diekema (NSO)
    • Raymond Sluiter (NSO)
    • Daniela Huppenkothen (SRON)

    SURF Research Partnership Lead

    I primarily work with the Dutch research communities Earth Observation and…

    Dit artikel heeft 0 reacties

    Gerelateerde artikelen

    1. Draft ENES Infrastructure Strategy 2024-2033

      Earth Observation

    2. Save the Date - Seminar Big Data Processing in Astronomy and Earth Observation: Synergies & Shared Challenges

      Earth Observation

    3. Successful seminar "Digital Twins for Earth Sciences"

      Earth Observation