Engineering Papers⌕ Search
Home/Archive/NASA NTRS

NASA NTRS · 20240013104

Big Data Challenges at CCMC

Tina Tsui·Poly Manessis·Matthew Lesko·Sarabjit Bakshi·Damian Barrous-Dume·Karen Scheiber·Edgar D Russell·Philip Poole·Masha Kuznetsova·Leila Mays
View original source ↗Export metadata ↓

Abstract

Like other research centers, the Community Coordinated Modeling Center (CCMC, https://ccmc.gsfc.nasa.gov) at NASA Goddard Space Flight Center (GSFC) is also experiencing the big data challenges. CCMC hosts over 80 space weather models for Runs On Request (ROR), Continuous Runs and Instant Runs simulation services for the research community. In addition, CCMC has started to support simulation output onboarding in response to the Open Science initiative. Overall, we have accumulated over petabytes of simulation output data and are rapidly growing. In this presentation, we will discuss our data and storage challenges. We will present our attempts to address our challenges and any associated lessons learned. CCMC uses Apache Airflow to ensure data transfer is consistent. We will give a brief overview on how we leverage Apache Airflow to enhance our environment.

Explore related subjects

space weather

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Tina Tsui, Poly Manessis, Matthew Lesko, Sarabjit Bakshi, Damian Barrous-Dume, Karen Scheiber, Edgar D Russell, Philip Poole, Masha Kuznetsova, Leila Mays. Big Data Challenges at CCMC. https://ntrs.nasa.gov/citations/20240013104

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

NASA NTRS2023

SelenITA: A Dual Point Lunar Mission to Characterize the Near Surface Dust and Electromagnetic Plasma Environment

SelenITA is a dual CubeSat mission that will provide the first multi-point dust, plasma, and magnetic field measurements in lunar orbit. This mission will advance the understanding of the electromagnetic space environment at the Moon in support of the Artemis program, exploration, and the geosciences. Here we present the science rationale motivating the mission. The candidate mission science objectives include: (1) constrain the origins of crustal magnetic fields; (2) determine the nature of plasma interactions with crustal magnetic fields; (3) characterize plasma waves and turbulence at the Moon; (4) characterize the lunar surface electric potential in varying plasma environments; (5) constrain the composition, thermal state, and structure of the lunar upper mantle and crust; (6) characterize the ionizing radiation in lunar orbit; and lastly, (7) determine the density of the dust exosphere as a function of latitude, longitude, and altitude, including the lunar polar space environment. The measurement requirements include simultaneous two-point observations of the 3-component vector magnetic field, plasma distribution (flux, energy, density, temperature), and single-point observations of energetic particles (protons, electrons, gamma rays), and dust. These measurements are important because it helps us understand how future astronauts, robots, and space hardware will live and work on the lunar surface.

space weather↗
NASA NTRS2023

SelenITA: A Dual Point Lunar Mission to Characterize the Near Surface Dust and Electromagnetic Plasma Environment

SelenITA is a dual CubeSat mission that will provide the first multi-point dust, plasma, and magnetic field measurements in lunar orbit. This mission will advance the understanding of the electromagnetic space environment at the Moon in support of the Artemis program, exploration, and the geosciences. Here we present the science rationale motivating the mission. The candidate mission science objectives include: (1) constrain the origins of crustal magnetic fields; (2) determine the nature of plasma interactions with crustal magnetic fields; (3) characterize plasma waves and turbulence at the Moon; (4) characterize the lunar surface electric potential in varying plasma environments; (5) constrain the composition, thermal state, and structure of the lunar upper mantle and crust; (6) characterize the ionizing radiation in lunar orbit; and lastly, (7) determine the density of the dust exosphere as a function of latitude, longitude, and altitude, including the lunar polar space environment. The measurement requirements include simultaneous two-point observations of the 3-component vector magnetic field, plasma distribution (flux, energy, density, temperature), and single-point observations of energetic particles (protons, electrons, gamma rays), and dust. These measurements are important because it helps us understand how future astronauts, robots, and space hardware will live and work on the lunar surface.

space weather↗
NASA NTRS2023

Improved Space Weather Observations and Modeling for Aviation Radiation

In recent years there has been a growing interest from the aviation community for space weather radiation forecasts tailored to the needs of the aviation industry. In 2019 several space weather centers began issuing advisories for the International Civil Aviation Organization alerting users to enhancements in the radiation environment at aviation flight levels. Due to a lack of routine observations, radiation modeling is required to specify the dose rates experienced by flight crew and passengers. While mature models exist, support for key observational inputs and further modeling advancements are needed. Observational inputs required from the ground-based neutron monitor network must be financially supported for research studies and operations to ensure real-time data is available for forecast operations and actionable end user decision making. An improved understanding of the geomagnetic field is required to reduce dose rate uncertainties in regions close to the open/closed geomagnetic field boundary, important for flights such as those between the continental US and Europe which operate in this region. Airborne radiation measurements, which are crucial for model validation and improvement, are lacking, particularly during solar energetic particle events. New measurement campaigns must be carried out to ensure progress and in situ atmospheric radiation measurements made available for real-time situational awareness. Furthermore, solar energetic particle forecasting must be improved to move aviation radiation nowcasts to forecasts in order to meet customer requirements for longer lead times for planning and mitigation.

space weather↗