Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data exploration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

CVEVOLVE

CVEvolve is an agentic AI system for autonomous algorithm discovery for scientific data processing. It creates workflows where large language model agents freely set up and configure development environments and evaluation harnesses, develop and improve data processing algorithms with designed exploration-exploitation balancing mechanisms, log history and findings in a structured database, and run holdout testing to ensure algorithm generalizability. CVEvolve offers a zero-code interface and does not require users to provide structured data and evaluation scripts.

Cherukara, MatthewJoseph [Argonne National Laborat↗

Data-driven Stellar Models

We developed a data-driven model to map stellar parameters ( T eff , log g , and [Fe/H]) accurately and precisely to broadband stellar photometry. This model must, and does, simultaneously constrain the passband-specific dust reddening vector in the Milky Way, R. The model uses a neural network to learn the (de-reddened) absolute magnitude in one band and colors across many bands, given stellar parameters from spectroscopic surveys and parallax constraints from Gaia. To demonstrate the effectiveness of this approach, we train our model on a data set with spectroscopic parameters from LAMOST, APOGEE, and GALAH, Gaia parallaxes, and optical and near-infrared photometry from Gaia, Pan-STARRS 1, Two Micron All Sky Survey and Wide-field Infrared Survey Explorer. Testing the model on these data sets leads to an excellent fit and a precise—and by construction—accurate prediction of the color–magnitude diagrams in many bands. This flexible approach rigorously links spectroscopic and photometric surveys, and also results in an improved, T eff -dependent R. As such, it provides a simple and accurate method for predicting photometry in stellar evolutionary models. Our model will form a basis to infer stellar properties, distances, and dust extinction from photometric data, which should be of great use in 3D mapping of the Milky Way. Our trained model can be obtained at doi:10.5281/zenodo.3902382.

79 ASTRONOMY AND ASTROPHYSICS↗

From Atom to Engine: Understanding Fundamental Effects of Structure on Combustion Using Tandem Experiment and Computation

The ability to tie structural features to a fuel candidate’s ignition properties provides a path for rational design of advantaged fuels that facilitate higher-efficiency, lower-emitting combustion in engines. The time evolution of a fuel’s radical population directly correlates with global combustion parameters, and because the generated radicals depend on the initial structure and reaction pathways of the fuel molecule, it follows that the molecular structure of a fuel has a direct impact on this cascade, and therefore salient ignition-tied fuel properties. This work aims to understand the impact of structural features in the context of global combustion properties for direct application to emerging engine technologies. Specifically, we highlight the ability to identify potential fuel blendstocks for specific engine strategies by examining and elucidating the radical cascade tied to ignition. A representative subset of alcohols possessing a variety of structural features (e.g. branching, chain length) was selected for study using a robust combination of experimental and computational methods. The approach begins with identifying short-lived radicals and likely abstraction sites upon pyrolysis in a microreactor coupled with photoionization mass spectrometry for direct detection of product species. In tandem, electronic structure calculations were performed to calculate the relevant potential energy surfaces for fuel pyrolysis as well as the subsequent O2 addition-isomerization reactions, which govern ignition behavior. Furthermore, the theoretical energetic and ro-vibrational data are used to calculate rate constants for direct ignition modeling to explore how structural effects impact NTC behavior. The microreactor data combined with electronic structure theory calculations provide a well-informed picture of the reactions relevant to ignition for each fuel in this study, as well as the most likely radicals generated under engine conditions. This thorough fundamental picture of fuel decomposition and ignition chemistry is used to correlate the observed ignition behavior to molecular structural effects.

advanced biofuels↗

NSA Site Science: Use of ARM Observations from Northern Alaska to Evaluate and Improve Prediction Capabilities

The Arctic is warming at a rate nearly double that of the rest of the planet, leading to profound changes in atmospheric, oceanic, and ice processes. The U.S. Department of Energy's (DOE) Atmospheric Radiation Measurement (ARM) user facility has played a significant role in Arctic research, operating observatories in Alaska's North Slope for over 25 years. These observatories provide a rich, and wide-reaching dataset that offers insight into atmospheric processes in northern Alaska. This report details the results of a nine-year research project (2015–2024) supported by the DOE Atmospheric Systems Research (ASR) program, that leverages data from ARM’s deployment of observing facilities at Utqiaġvik (known as the North Slope of Alaska, or NSA, site) and Oliktok Point, Alaska. The project was conducted in two phases: - Phase 1 (2015–2019): Focused on understanding key atmospheric processes at Oliktok Point, including cloud formation, high-latitude precipitation, aerosol-cloud interactions, and cloud properties. - Phase 2 (2019–2024): Extended the research to the broader North Slope region, using data from both Oliktok Point and Utqiaġvik. Topics explored included surface energy budgets, atmospheric stability, ice nucleation processes, and microphysics in Arctic clouds. The project resulted in numerous research products, including 50 peer-reviewed publications and dissertations, 169 presentations, and 10 data products. These products cover a variety of topics, including: - Cloud Macro- and Microphysical Properties: Arctic clouds play a crucial role in energy transfer, and accurate representation in models is critical. The study explored cloud transitions, ice crystal shapes, and dual-wavelength radar data to understand ice crystal habits and size distributions. - Aerosol Properties and Processes: The team examined aerosol sources in the Arctic, including industrial emissions and natural sources. Observations showed significant spatial gradients in aerosol concentrations due to human activities and wildfire smoke. The influence of aerosols on cloud formation and the surface energy budget was also assessed. - Aerosol-Cloud Interactions: Research revealed that aerosols might suppress cloud ice production, affecting cloud radiative forcing and precipitation. The impact of local industrial emissions on cloud properties was also investigated. - Contextualizing the North Slope of Alaska in the context of the broader Arctic: To understand broader trends, the project evaluated large-scale circulation patterns and the influence of weather systems on the Arctic. Studies indicated that large-scale processes play a significant role in temperature patterns and the timing of snowmelt. - Advancing ARM Observational and Modeling Capabilities: The project developed new radar data products and advanced measurement techniques, including clutter mitigation and drizzle detection. Uncrewed aerial systems (UAS) and tethered balloon systems (TBS) were deployed to gather detailed atmospheric data. Additionally, the project supported 10 early career scientists, providing training and mentorship to undergraduate interns, graduate students, postdoctoral researchers, and early career researchers. These efforts contributed to the advancement of ARM research capabilities and fostered a new generation of scientists skilled in Arctic atmospheric research. Ultimately, this ASR-supported project has provided valuable insights into Arctic atmospheric processes and their broader climate implications. Recommendations for future work include continuing support for long-term observing at Arctic locations to foster additional research, further exploration of aerosol-cloud interactions and the potential impacts of enhanced industrialization of the Arctic, and expanded use of uncrewed systems to gather data in this remote and harsh environment. Additionally, the data products developed by this work, and the data products developed through the ARM infrastructure, leave a treasure-trove of additional information that should be explored for many years to come to gain additional insight into physical processes in the Arctic atmosphere that drive the rapid changes occurring in at high latitudes and their global impact.

58 GEOSCIENCES↗

Utah FORGE: Fault Shear Reactivation Experimental Data for Fluid Injection-Rate Controls on Seismic Moment

Included are experimental data recorded from shear experiments that specifically explore the link between fluid-injection rate and seismic moment resulting from shear reactivation of laboratory faults. Raw mechanical data from three experiments are included alongside corresponding MATLAB scripts that import and plot the data, as well as use it to calculate shear and normal stress. Experiments are performed on 2.5-3 inch long granitoid cores from the Utah FORGE EGS demonstration site, containing a single inclined fracture with small-scale roughness added to the fracture surface. The raw data included here were recorded from an aluminum triaxial pressure vessel (TEMCO) configured with three independent servo-controlled pumps, with distilled water used as the working fluid. The pumps control confining pressure, upstream pore pressure, and axial pressure, with each pump connected to a LabView interface to record applied pressures, cumulative injected water volumes, and pump flow rates. The downstream outlet from the fracture is closed to allow pressurization, which is measured by an external pressure transducer. A linear variable differential transformer (LVDT) attached to the axial piston measures axial displacement, from which we calculate shear displacement along the fracture. Additionally, P-wave transducers are used to record acoustic signatures, where acoustic emission events and maximum amplitudes are compared against seismic moment and shear slip velocity. Fluid injection rates range between 0.05 mL/min, 0.25 mL/min, and 0.75 mL/min for each experiment. Along-fault pressure distributions are progressively less uniform as injection rates increase, representing a switch from steady-state to transient conditions. Triggered shear displacement is used as a proxy for seismic moment and is indexed against cumulative injection volume and rate. Each experiment is performed under constant shear stress conditions, and the sample is fully saturated with DI water. Axial and confining stresses are applied to 3 MPa through pressure-stepping in 500 kPa increments. The pore pressure is held constant at 200 kPa prior to initiating the experiment, and initial axial displacement is recorded. The axial stress is then increased to initiate shear mobilization during the loading phase (run-in) until a peak steady state is achieved. The initial shear stress is reduced to approximately 80% of the peak shear stress by decreasing the axial stress, then held constant for the duration of each experiment.

15 GEOTHERMAL ENERGY↗

Operational Focused Data Analytics for Optimizing Radiation Portal Monitor-Based Nuclear Smuggling Detection Systems at Global Ports of Entry

The National Nuclear Security Administration’s Office of Nuclear Smuggling Detection and Deterrence has deployed a fleet of radiation portal monitors (RPMs) across the world at global ports of entry including seaports, airports, and land border crossings. These RPMs are integrated into radiation detection systems (RDS) that also include fixed cameras, optical character recognition (OCR) systems, primary scanning systems (e.g., X-ray or gamma-ray), and secondary scanning systems (e.g., spectroscopic radiation portal monitors, portable radiation detection systems). The data from these sensing technologies is collected at the Central Alarm Station (CAS) where servers and computers reside to control and operate the system. Operators utilize the data collected by the CAS and declared cargo information to make decisions on how to respond to an alarm.This work explores the use of CAS-located data, looking at both the sensor data streams and operator inputs, to perform analysis which supports customs and border protection agencies to improve training capability and operational effectiveness. We focus on analyzing site level effectiveness and behavior by rolling up CAS-located data collected from individual occurrences. To-date, more than 15 sites (e.g., seaports, airports, border crossings) have been analyzed in this manner with the goal of understanding system operations to verify effectiveness and recommend potential improvements. This work first aims to provide background information on relevant CAS-located data sources and our current operational system analytics process including example results. After summarizing our current analytic techniques, we discuss how the future data analytics systems can provide key benefits to improving operational performance while minimizing the burden these detection systems place on operators.

Kuhn, Michael↗

Exploring Nonmotorized Travel in New York State Using 2017 National Household Travel Survey Data

This report presents systematic analysis of nonmotorized travel behavior within New York State (NYS) utilizing 2017 National Household Travel Survey data. As walking and biking assume increasingly prominent roles in advancing active transportation objectives, accessible mobility, and sustainable transportation systems, comprehensive understanding of these modal patterns becomes essential for evidence-based policy development and strategic planning initiatives. Recognizing the substantial geographic, demographic, and socioeconomic heterogeneity characterizing NYS—spanning from the concentrated urban fabric of New York City (NYC) to dispersed suburban and rural contexts—this analysis employs the structured "4Ws" analytical framework to examine participant demographics, spatial and temporal distributions, and motivational factors underlying active transportation choices relative to motorized alternatives. This methodological approach captures modal behavior patterns, user characteristics, trip purposes, and temporal variations across the state's diverse contexts. Additionally, the research examines behavioral differences among distinct user classifications, including walk-only and bike-only travelers, while conducting comparative analysis of nonmotorized travel patterns between ALICE (Asset Limited, Income Constrained, Employed) and non-ALICE household categories.

99 GENERAL AND MISCELLANEOUS↗

Lossy checkpoint compression in full waveform inversion: a case study with ZFPv0.5.5 and the overthrust model

This paper proposes a new method that combines checkpointing methods with error-controlled lossy compression for large-scale high-performance full-waveform inversion (FWI), an inverse problem commonly used in geophysical exploration. This combination can significantly reduce data movement, allowing a reduction in run time as well as peak memory. In the exascale computing era, frequent data transfer (e.g., memory bandwidth, PCIe bandwidth for GPUs, or network) is the performance bottleneck rather than the peak FLOPS of the processing unit. Like many other adjoint-based optimization problems, FWI is costly in terms of the number of floating-point operations, large memory footprint during backpropagation, and data transfer overheads. Past work for adjoint methods has developed checkpointing methods that reduce the peak memory requirements during backpropagation at the cost of additional floating-point computations. Combining this traditional checkpointing with error-controlled lossy compression, we explore the three-way tradeoff between memory, precision, and time to solution. We investigate how approximation errors introduced by lossy compression of the forward solution impact the objective function gradient and final inverted solution. Empirical results from these numerical experiments indicate that high lossy-compression rates (compression factors ranging up to 100) have a relatively minor impact on convergence rates and the quality of the final solution.

58 GEOSCIENCES↗

Deployment priority of public charging speeds for increasing battery electric vehicle usability

To inform charger deployment decisions, this paper aims to understand the potential utilization and deployment priority for public charging infrastructure. A data-driven Cumulative Public Recharging (CPR) model is developed to explore the travel patterns by using 2017 National Household Travel Survey data. Given the daily trip sequence, trip distance, and dwell times, the study examines the daily expected driving range and BEV feasibility under different charging speeds, battery capacity, and charging behaviors. The results suggest that more advanced public chargers increase the daily expected driving range. Home charging is sufficient for most daily short-distance trips while public chargers are still needed for medium- and long-distance trips. Extreme fast charging (xFC) may not be necessary for people with home charging but could be more useful for people without home charging and for urgent charging. In conclusion, xFC becomes even less important to drivers with longer BEV ranges, a finding that contradicts conventional thinking.

33 ADVANCED PROPULSION SYSTEMS↗

Electricity Rate Designs for Large Loads: Evolving Practices and Opportunities

Electricity demand from large load customers such as data centers is projected to grow significantly in the near term. While data centers play an important role in advancing technology innovation and economic growth in the United States, data center energy needs present challenges and opportunities for electricity supply and infrastructure. This technical brief serves as a foundation for the discussion of issues and sharing of perspectives among utilities, regulators, large load customers, and other stakeholders. As utilities and regulators explore rate structures to address growing data center electricity demand, several issues have emerged: -Fair allocation of electricity system costs to large-load customers without unfair shifting of costs to other customers -Appropriate mitigation of the financial risks associated with stranded assets from underutilized utility system investments -Mitigation of operational and resource adequacy risks if electricity demand exceeds supply -Appropriate risk-sharing in commercializing newer electricity technologies such as advanced geothermal, small modular reactors, and long duration energy storage -Accommodating the diverse needs of large-load customers, such as having the option to match electricity consumption with output from carbon-free resources or using onsite generation to provide system capacity The technical brief also identifies key design elements that aim to address these issues and uses leading examples from pending and approved rate structures, agreements, and special contracts to ground the elements in practice.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Periodic Gamma-Ray Modulation of the Blazar PG 1553+113 Confirmed by Fermi-LAT and Multiwavelength Observations

A 2.1 yr periodic oscillation of the gamma-ray flux from the blazar PG 1553+113 has previously been tentatively identified in ∼7 yr of data from the Fermi Large Area Telescope. After 15 yr of Fermi sky-survey observations, doubling the total time range, we report >7 cycle gamma-ray modulation with an estimated significance of 4σ against stochastic red noise. Independent determinations of oscillation period and phase in the earlier and the new data are in close agreement (chance probability <0.01). Pulse timing over the full light curve is also consistent with a coherent periodicity. Multiwavelength new data from Swift X-Ray Telescope, Burst Alert Telescope, and UVOT, and from KAIT, Catalina Sky Survey, All-Sky Automated Survey for Supernovae, and Owens Valley Radio Observatory ground-based observatories as well as archival Rossi X-Ray Timing Explorer satellite-All Sky Monitor data, published optical data of Tuorla, and optical historical Harvard plates data are included in our work. Optical and radio light curves show clear correlations with the gamma-ray modulation, possibly with a nonconstant time lag for the radio flux. We interpret the gamma-ray periodicity as possibly arising from a pulsational accretion flow in a sub-parsec binary supermassive black hole system of elevated mass ratio, with orbital modulation of the supplied material and energy in the jet. Other astrophysical scenarios introduced include instabilities, disk and jet precession, rotation or nutation, and perturbations by massive stars or intermediate-mass black holes in polar orbit.

79 ASTRONOMY AND ASTROPHYSICS↗

Refining HPCToolkit for application performance analysis at exascale

As part of the US Department of Energy’s Exascale Computing Project (ECP), Rice University has been refining its HPCToolkit performance tools to better support measurement and analysis of applications executing on exascale supercomputers. To efficiently collect performance measurements of GPU-accelerated applications, HPCToolkit employs novel non-blocking data structures to communicate performance measurements between tool threads and application threads. To attribute performance information in detail to source lines, loop nests, and inlined call chains, HPCToolkit performs parallel analysis of large CPU and GPU binaries involved in the execution of an exascale application to rapidly recover mappings between machine instructions and source code. To analyze terabytes of performance measurements gathered during executions at exascale, HPCToolkit employs distributed-memory parallelism, multithreading, sparse data structures, and out-of-core streaming analysis algorithms. To support interactive exploration of profiles up to terabytes in size, HPCToolkit’s hpcviewer graphical user interface uses out-of-core methods to visualize performance data. The result of these efforts is that HPCToolkit now supports collection, analysis, and presentation of profiles and traces of GPU-accelerated applications at exascale. These improvements have enabled HPCToolkit to efficiently measure, analyze and explore terabytes of performance data for executions using as many as 64K MPI ranks and 64K GPU tiles on ORNL’s Frontier supercomputer. HPCToolkit’s support for measurement and analysis of GPU-accelerated applications has been employed to study a collection of open-science applications developed as part of ECP. This paper reports on these experiences, which provided insight into opportunities for tuning applications, strengths and weaknesses of HPCToolkit itself, as well as unexpected behaviors in executions at exascale.

Adhianto, Laksono↗

Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing

In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.

Hou, Shiyue↗

Steptoe Valley NV Data Compilation: Understanding a Stratigraphic Hydrothermal Resource through Geophysical Imaging

Sandia National Laboratories partnered with a multi-disciplinary group of subject matter experts to evaluate a stratigraphic geothermal resource in Steptoe Valley, Nevada using both established and novel geophysical imaging techniques. Provided here are a compilation of newly acquired data over the area and select modeling efforts. This encompasses a 3D geological model (inclusive of full Leapfrog files, Leapfrog viewer files, and XYZ data for faults and stratigraphy) with embedded geophysical modeling, controlled-source electromagnetic (CSEM) and magnetotelluric (MT) data packages, aqueous spring geochemistry data, seismic reflection interpretations, and a gravity data package. The stratigraphic reservoir in Steptoe Valley was previously discovered during oil and gas exploration. Subsequent studies, such as the Nevada Play Fairway Analysis, added data which further highlighted potential resource targets in the basin. Geophysical surveys, complimented with refined geologic mapping and geochemical sampling, were deployed to further characterize the resource. The resulting 3D geologic interpretation, conceptual model refinements, and reservoir simulations suggest that a power-capable reservoir is economically accessible in the Paleozoic carbonates of the deep/central basin. Additional geophysical characterization and exploration drilling efforts are recommended to calibrate interpretation and determine where/how to potentially develop the Steptoe resource. The geophysical tools, interpretations, lessons learned, and publicly available data generated by this study establish an exploration methodology to inform decisions for successful development of stratigraphic reservoirs.

15 GEOTHERMAL ENERGY↗

Updated resources for exploring experimentally-determined PDB structures and Computed Structure Models at the RCSB Protein Data Bank

The Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB, RCSB.org), the US Worldwide Protein Data Bank (wwPDB, wwPDB.org) data center for the global PDB archive, provides access to the PDB data via its RCSB.org research-focused web portal. We report substantial additions to the tools and visualization features available at RCSB.org, which now delivers more than 227000 experimentally determined atomic-level three-dimensional (3D) biostructures stored in the global PDB archive alongside more than 1 million Computed Structure Models (CSMs) of proteins (including models for human, model organisms, select human pathogens, crop plants and organisms important for addressing climate change). In addition to providing support for 3D structure motif searches with user-provided coordinates, new features highlighted herein include query results organized by redundancy-reduced Groups and summary pages that facilitate exploration of groups of similar proteins. Newly released programmatic tools are also described, as are enhanced training opportunities.

Burley, Stephen K.↗

Validation of NEAMS Tools Using MSRE Data

This report documents the benchmarks developed and performed for the tools and codes developed under the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program utilizing the MSRE experiment data. There are three focus areas explored in this work: fuel evolution, CFD, and system transient analyses. Fuel depletion is important for MSR reactor physics and fuel cycle analysis. Fuel depletion capability is currently being developed in the Griffin reactor physics code. To support this, a benchmark problem is developed using the MSRE data and an existing depletion code with flowing fuel analysis functionality (ADDER) to validate this new capability in Griffin. The ADDER calculation is compared with the TRITON-MSR results as well as the MSRE experimental data. This benchmark can be directly used to verify and validate the Griffin fuel depletion capability in future years. For CFD simulation, the MSRE CAD model is obtained from the Copenhagen Atomics and a mesh is generated for the MSRE inlet plenum. The Nek5000 code will be employed to find the flow distribution in the MSRE inlet plenum based on the mesh generated. For system analysis, a simple benchmark problem for three types of MSR system transients is developed. The SAM and SPECTRA codes are used to solve the simple problem, and the results from the two codes agree well. The SAM code is also employed to obtain the frequency response of the MSRE system at the 5 and 8 MWt power levels. The results agree well with the experiment measurements. These verification and validation efforts gives SAM users more confidence in MSR system transient simulations using SAM. Lastly, the SAM-Griffin Multiphysics coupling work is initiated. A simple one-way coupling of evaluating the precursor drift effect on the MSRE k-eff is demonstrated. This preliminary coupling work serves as the basis for future more complex Multiphysics simulation and modelling.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Estimating building occupancy: a machine learning system for day, night, and episodic events

Building occupancy research increasingly emphasizes understanding the social and physical dynamics of how people occupy space. Opportunities in the open source domain including social media, Volunteered Geographic Information, crowdsourcing, and sensor data have proliferated, resulting in the exploration of building occupancy dynamics at varying spatiotemporal scales. At Oak Ridge National Laboratory, research into building occupancies through the development of a global learning framework that accommodates exploitation of open source authoritative sources, including governmental census and surveys, journal articles, real estate databases, and more, to report national and subnational building occupancies across the world continues through the Population Density Tables (PDT) project. This probabilistic learning system accommodates expert knowledge, experience, and open-source data to capture local, socioeconomic, and cultural information about human activity. It does so through a systematic process of data harmonization techniques in the development of observation models for over 50 building types to dynamically update baseline estimates and report probabilistic diurnal and episodic building occupancy estimates. This discussion will explore how PDT is implemented at scale and expanded based on the development of observation model classes and will explain how to interpret and spatially apply the reported probability occupancy estimates and uncertainty.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Integrating HPC, AI, and Workflows for Scientific Data Analysis: Report from Dagstuhl Seminar 23352

The Dagstuhl Seminar 23352, titled “Integrating HPC, AI, and Workflows for Scientific Data Analysis,” held from August 27 to September 1, 2023, was a significant event focusing on the synergy between High-Performance Computing (HPC), Artificial Intelligence (AI), and scientific workflow technologies. The seminar recognized that modern Big Data analysis in science rests on three pillars: workflow technologies for reproducibility and steering, AI and Machine Learning (ML) for versatile analysis, and HPC for handling large data sets. These elements, while crucial, have traditionally been researched separately, leading to gaps in their integration. The seminar aimed to bridge these gaps, acknowledging the challenges and opportunities at the intersection of these technologies. The event highlighted the complex interplay between HPC, workflows, and ML, noting how ML has increasingly been integrated into scientific workflows, thereby enhancing resource demands and bringing new requirements to HPC architectures, like support for GPUs and iterative computations. The seminar also addressed the challenges in adapting HPC for large-scale ML tasks, including in areas like deep learning, and the need for workflow systems to evolve to leverage ML in data analysis fully. Moreover, the seminar explored how ML could optimize scientific workflow systems and HPC operations, such as through improved scheduling and fault tolerance. A key focus was on identifying prestigious use cases of ML in HPC and understanding their unique, unmet requirements. The stochastic nature of ML and its impact on the reproducibility of data analysis on HPC systems was also a topic of discussion.

97 MATHEMATICS AND COMPUTING↗