Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

Review of the Current State of Knowledge on the Effects of Radiation on Concrete

A review of the current state of knowledge on the effects of radiation on concrete in nuclear applications is presented. Emphasis is placed on the effects of radiation damage as reflected by changes in engineering properties of concrete in the evaluation of the long-term operation (LTO) and for Plant Life or Aging Management of nuclear power plants (NPPs) in Japan, Spain, and the United States. National issues and concerns are described for Japan and the US followed by a discussion of the fundamental understanding of the effects radiation on concrete. Specifically, the effects of temperature, moisture content, and irradiation on ordinary Portland cement paste and the role of temperature and neutron energy spectra on radiation induced volumetric expansion (RIVE) of aggregate-forming minerals are described. This is followed by a discussion of the bounding conditions for extended operation, the significance of accelerated irradiation conditions, the role of temperature, creep, and how these issues are being incorporated into numerical and meso-scale models. From these insights on radiation damage, analyses of these effects on concrete structures are reviewed and the current status of work in Japan and the US are described. Also discussed is the recent formation of a new international scientific and technical organization, the International Committee on Irradiated Concrete (ICIC), to provide a forum for timely information exchanges among organizations pursuing the identification, quantification, and modeling of the effects of radiation on concrete in commercial nuclear applications. Lastly, the paper concludes with a discussion of research gaps including: 1) interpreting test-reactor data, 2) evaluating service-irradiated concrete for aging management and to inform radiation damage models with the Zorita NPP (Spain) serving as the first comprehensive test case, 3) irradiated-assisted alkali-silica reactions, and 4) RIVE under constrained conditions.

36 MATERIALS SCIENCE↗

The medical science DMZ: a network design pattern for data-intensive medical science

Abstract Objective We describe a detailed solution for maintaining high-capacity, data-intensive network flows (eg, 10, 40, 100 Gbps+) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet-filter firewalls, network intrusion-detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive datasets between research institutions over national research networks, and give 3 detailed descriptions of implemented Medical Science DMZs. Discussion The exponentially increasing amounts of “omics” data, high-quality imaging, and other rapidly growing clinical datasets have resulted in the rise of biomedical research “Big Data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large datasets. Maintaining data-intensive flows that comply with the Health Insurance Portability and Accountability Act (HIPAA) and other regulations presents a new challenge for biomedical research. We describe a strategy that marries performance and security by borrowing from and redefining the concept of a Science DMZ, a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

Peisert, Sean↗

Running Ensemble Workflows at Extreme Scale: Lessons Learned and Path Forward

The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.

Mehta, Kshitij↗

torch-einshard v1.0

torch-einshard is a Python library for describing local and distributed PyTorch tensor computations with compact, einsum-like notation. Its expressions name logical axes, specify how they are sharded across a PyTorch DeviceMesh, and represent partial reductions. The library automatically performs contractions, permutations, reshaping, splitting, gathering, reduction, reduce-scatter, and repartitioning while preserving autograd. Additional features include sharding-aware FFTs, tensor rolls, halo exchange, sliding windows, 1D–3D convolutions, uneven-shard handling, parameter initialization and gradient management, and cost-based execution planning. It is designed for scientific machine learning and large-model workloads, including tensor-, sequence-, and spatial-parallel MLPs, attention, convolutions, and spectral operations. Compared with manually combining torch.einsum and distributed collectives, torch-einshard expresses both the mathematical operation and data placement in one readable formula. This reduces boilerplate and synchronization errors, keeps forward and backward communication consistent, and allows the library to select optimized collective strategies without changing model code.

Morozov, Dmitriy [Lawrence Berkeley National Labor↗

Using and Distributing Spaceflight Data: The Johnson Space Center Life Sciences Data Archive

Life sciences data collected before, during and after spaceflight are valuable and often irreplaceable. The Johnson Space Center Life is hard to find, and much of the data (e.g. Sciences Data Archive has been designed to provide researchers, engineers, managers and educators interactive access to information about and data from human spaceflight experiments. The archive system consists of a Data Acquisition System, Database Management System, CD-ROM Mastering System and Catalog Information System (CIS). The catalog information system is the heart of the archive. The CIS provides detailed experiment descriptions (both written and as QuickTime movies), hardware descriptions, hardware images, documents, and data. An initial evaluation of the archive at a scientific meeting showed that 88% of those who evaluated the catalog want to use the system when completed. The majority of the evaluators found the archive flexible, satisfying and easy to use. We conclude that the data archive effectively provides key life sciences data to interested users.

Cardenas, J. A.↗

The Chandra X-Ray Observatory

The Chandra X-Ray Observatory is the X-ray component of NASA's Great observatories which also includes the recently decommissioned Compton Gamma Ray Observatory, the Hubble Space Telescope, and the soon to be launched Space Infra Red Telescope Facility. Chandra is a unique X-ray astronomy facility for high-resolution imaging and for high-resolution spectroscopy. Chandra's performance advantage over other X-ray observatories is analogous to that of the Hubble Space Telescope over ground-based observatories. Chandra is a NASA facility that provides scientific data to the international astronomical community in response to proposals for its use. Data becomes public one year after the observation. The Observatory is the product of the efforts of many commercial, academic, and government organizations in the United States and Europe. NASA's Marshall Space Flight Center (MSFC) manages the Project and provides Project Science; TRW Space and Electronics Group served as prime contractor responsible for providing the spacecraft, the telescope, and assembling and testing the observatory; the Smithsonian Astrophysical Observatory (SAO) provides technical support and is responsible for ground operations.

Weisskopf, M. C.↗

Use of Remote Sensing for Decision Support in Africa

Over the past 30 years, the scientific community has learned a great deal about the Earth as an integrated system. Much of this research has been enabled by the development of remote sensing technologies and their operation from space. Decision makers in many nations have begun to make use of remote sensing data for resource management, policy making, and sustainable development planning. This paper makes an attempt to provide a survey of the current state of the requirements and use of remote sensing for sustainable development in Africa. This activity has shown that there are not many climate data ready decision support tools already functioning in Africa. There are, however, endusers with known requirements who could benefit from remote sensing data.

Policelli, Frederick S.↗

A scalable messaging system for accelerating discovery from large scale scientific simulations

Emerging scientific and engineering simulations running at scale on leadership-class High End Computing (HEC) environments are producing large volumes of data, which has to be transported and analyzed before any insights can result from these simulations. The complexity and cost (in terms of time and energy) associated with managing and analyzing this data have become significant challenges, and are limiting the impact of these simulations. Recently, data-staging approaches along with in-situ and in-transit analytics have been proposed to address these challenges by offloading I/O and/or moving data processing closer to the data. However, scientists continue to be overwhelmed by the large data volumes and data rates. In this paper we address this latter challenge. Specifically, we propose a highly scalable and low-overhead associative messaging framework that runs on the data staging resources within the HEC platform, and builds on the staging-based online in-situ/in- transit analytics to provide publish/subscribe/notification-type messaging patterns to the scientist. Rather than having to ingest and inspect the data volumes, this messaging system allows scientists to (1) dynamically subscribe to data events of interest, e.g., simple data values or a complex function or simple reduction (max()/min()/avg()) of the data values in a certain region of the application domain is greater/less than a threshold value, or certain spatial/temporal data features or data patterns are detected; (2) define customized in-situ/in-transit actions that are triggered based on the events, such as data visualization or transformation; and (3) get notified when these events occur. The key contribution of this paper is a design and implementation that can support such a messaging abstraction at scale on high- end computing (HEC) systems with minimal overheads. We have implemented and deployed the messaging system on the Jaguar Cray XK6 machines at Oak Ridge National Laboratory and the Lonestar system at the Texas Advanced Computing Center (TACC), and we present the experimental performance evaluation using these HEC platforms in the paper.

Jin, Tong↗

Sparse Data Management in HDF5

The importance of sparse data management is growing with data produced by large-scale experimental and observational facilities that contain small amounts of non-zero values. In this document, we explore different design options to support sparse data in HDF5, one of the most popular highperformance I/O libraries and file formats used for scientific data. We discuss several use cases and requirements. Our main hard design constraint is that any changes to the HDF5 dataset API would be a burden on users and not acceptable. The remaining options are discussed below. We provide rationale for what we believe is the strongest candidate and describe how its potential benefits can be simulated with the existing HDF5 library. We have conducted a set of computational experiments the results of which are reported here. They show that our candidate meets all relevant requirements and gives us a certain degree of confidence for an HDF5 library-native implementation.

Mainzer, John↗

Microbial Risk Assessment

Historically, microbiological spaceflight requirements have been established in a subjective manner based upon expert opinion of both environmental and clinical monitoring results and the incidence of disease. The limited amount of data, especially from long-duration missions, has created very conservative requirements based primarily on the concentration of microorganisms. Periodic reevaluations of new data from later missions have allowed some relaxation of these stringent requirements. However, the requirements remain very conservative and subjective in nature, and the risk of crew illness due to infectious microorganisms is not well defined. The use of modeling techniques for microbial risk has been applied in the food and potable water industries and has exceptional potential for spaceflight applications. From a productivity standpoint, this type of modeling can (1) decrease unnecessary costs and resource usage and (2) prevent inadequate or inappropriate data for health assessment. In addition, a quantitative model has several advantages for risk management and communication. By identifying the variable components of the model and the knowledge associated with each component, this type of modeling can: (1) Systematically identify and close knowledge gaps, (2) Systematically identify acceptable and unacceptable risks, (3) Improve communication with stakeholders as to the reasons for resource use, and (4) Facilitate external scientific approval of the NASA requirements. The modeling of microbial risk involves the evaluation of several key factors including hazard identification, crew exposure assessment, dose-response assessment, and risk characterization. Many of these factors are similar to conditions found on Earth; however, the spaceflight environment is very specialized as the inhabitants live in a small, semi-closed environment that is often dependent on regenerative life support systems. To further complicate modeling efforts, microbial dose-response characteristics may be affected by a potentially dysfunctional crew immune system during a mission. In addition, microbial virulence has been shown to change under certain conditions during spaceflight, further complicating dose-response characterization. An initial study of the applicability of microbial risk assessment techniques was performed using Crew Health Care System (CHeCS) operational data from the International Space Station potable water systems. The risk of infection from potable water was selected as the flight systems and microbial ecology are well defined. This initial study confirmed the feasibility of using microbial risk assessment modeling for spaceflight systems. While no immediate threat was detected, the study identified several medically significant microorganisms that could pose a health risk if uncontrolled. The study also identified several specific knowledge gaps in making a risk assessment and noted that filling these knowledge gaps is essential as the risk estimates may change by orders of magnitude depending on the answers. The current phase of the microbial risk assessment studies focuses on the dose-response relationship of specific infectious agents, focusing on Salmonella enterica Typhimurium, Pseudomonas spp., and Escherichia coli, as their evaluation will provide a better baseline for determining the overall hazard characterization. The organisms were chosen as they either have been isolated on spacecraft or have an identified route of infection during a mission. The characterization will utilize dose-response models selected either from the peer-reviewed literature and/or by using statistical approaches. Development of these modeling and risk assessment techniques will help to optimize flight requirements and to protect the safety, health, and performance of the crew.

Ott, C. M.↗

OpenNEX: An open collaboration platform for the earth science community

Satellite data from the past several decades provide the most consistent record of land-surface processes that form the basis for scientific assessments of the impacts of climate variations and changes on the environment and human social-economical activities. During this time, scientific research on the characterization and assessment of environmental changes had tended to focus on large-scale land-surface changes with significant social-economic impacts. Increasingly, attention is shifting toward changes that occur more locally and that most directly relate to the everyday life of the majority of the population. In addition, there has been needs to develop management and policy decision support systems that are based on local environmental information. Almost at the same time, the advancement in sensor technology has allowed us to collect an unprecedented volume of environmental data. These data must be curated and analyzed to extract useful information for research and decision support purposes. Established in 2013 and funded by NASA, the Open NASA Earth eXchange (OpenNEX; https://opennex.org/ , Jia et al., 2019 ) project partnered with Amazon Web Services (AWS) to make available a large amount of Earth observing data, modeling results, and analysis tools on the AWS. OpenNEX provides researchers, developers, educators, and ordinary users with easy access to an integrated Earth science computational and data platform, enabling citizen scientists and application developers to realize the full value of NASA data assets and software tools. To encourage the public's engagement in this project, NASA ran virtual workshops and prize competitions. The virtual workshops provided online lectures and tutorials about how prominent scientists used the data in their research and the tutorials gave examples how to use the tools to interrogate the data in the Amazon cloud. Finally, the prize competitions allowed much wider participation in the OpenNEX project and enable testing the non-traditional projects and out-of-box ideas. OpenNEX has continued to evolve and mature. Here, we highlight new features and functionalities available to the community.

Jian Zhang↗

LandScan Global 30 Arcsecond Annual Global Gridded Population Datasets from 2000 to 2022

Abstract Oak Ridge National Laboratory (ORNL) annually develops the LandScan Global (LSG) dataset, a 30 arcsecond global gridded population dataset representing global ambient human population distribution. This multivariable dasymetric model disaggregates census counts within administrative boundaries using ancillary data. Each country’s distribution reflects cultural and socioeconomic patterns; manual validations yield a unique global dataset for assessing populations at risk. For over two decades, LSG has been a standard for estimating populations at risk, aiding U.S. federal government, academia and humanitarian organizations. During disasters such as the 2004 Indian Ocean tsunami and the 2010 Haiti earthquake and geopolitical crises such as the Syrian civil war and the 2022 Russian invasion of Ukraine, LSG supported scientific and operational communities in emergency response and recovery. In 2022, LSG datasets from 2000 onward were made publicly available through ORNL’s LandScan Portal. This data descriptor details our methodology and the application of geospatial science and machine learning to geographic and demographic data, highlighting uses in urban resiliency, emergency management, disaster response, and human health and security.

Science & Technology - Other Topics↗

Position Papers for the 2024 ASCR Workshop on Energy-Efficient Computing for Science

On behalf of the Advanced Scientific Computing Research (ASCR) program in the US Department of Energy (DOE) Office of Science, we are organizing a Workshop on Energy-Efficient Computing for Science (EECS). Energy efficiency involves coordination across all the interoperating components of a computing system—in particular, applications, algorithms, system software, programming models, data management, and the hardware on which they run. Looking 10-15 years into the future, the goal is to dramatically lower the energy costs of the computational platforms (from the data center to the edge) serving DOE science while expanding the capabilities of these systems, broadening their applicability to science challenges of interest to DOE and the nation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

SPAN: Ocean science

The Space Physics Analysis Network (SPAN) is a multi-mission, correlative data comparison network which links space and Earth science research and data analysis computers. It provides a common working environment for sharing computer resources, sharing computer peripherals, solving proprietary problems, and providing the potential for significant time and cost savings for correlative data analysis. This is one of a series of discipline-specific SPAN documents which are intended to complement the SPAN primer and SPAN Management documents. Their purpose is to provide the discipline scientists with a comprehensive set of documents to assist in the use of SPAN for discipline specific scientific research.

Thomas, Valerie L.↗

X-SAR: The X-band synthetic aperture radar on board the Space Shuttle

The X-band synthetic aperture radar (X-SAR) is the German/Italian contribution to the NASA/JPL Shuttle Radar Lab missions as part of the preparation for the Earth Observation System (EOS) program. The Shuttle Radar Lab is a combination of several radars: an L-band (1.2 GHz) and a C-band (5.3 GHz) multipolarization SAR known as SIR-C (Shuttle Imaging Radar); and an X-band (9.6 GHz) vertically polarized SAR which will be operated synchronously over the same target areas to deliver calibrated multifrequency and multipolarization SAR data at multiple incidence angles from space. A joint German/Italian project office at DARA (German Space Agency) is responsible for the management of the X-SAR project. The space hardware has been developed and manufactured under industrial contract by Dornier and Alenia Spazio. Besides supporting all the technical and scientific tasks, DLR, in cooperation with ASI (Agencia Spaziale Italiano) is responsible for mission operation, calibration, and high precision SAR processing. In addition, DLR developed an airborne X-band SAR to support the experimenters with campaigns to prepare for the missions. The main advantage of adding a shorter wavelength (3 cm) radar to the SIR-C radars is the X-band radar's weaker penetration into vegetation and soil and its high sensitivity to surface roughness and associated phenomena. The performance of each of the three radars is comparable with respect to radiometric and geometric resolution.

Werner, Marian U.↗

SEMS: System for Environmental Monitoring and Sustainability

The goal of this project was to establish a computational and data management system, SEMS, building on our existing system and MTPE-related research. We proposed that the new system would help support Washington University's efforts in environmental sustainability through use in: (a) Problem-based environmental curriculum for freshmen and sophomores funded by the Hewlett Foundation that integrates scientific, cultural, and policy perspectives to understand the dynamics of wetland degradation, deforestation, and desertification and that will develop policies for sustainable environments and economies; (b) Higher-level undergraduate and graduate courses focused on monitoring the environment and developing policies that will lead to sustainable environmental and economic conditions; and (c) Interdisciplinary research focused on the dynamics of the Missouri River system and development of policies that lead to sustainable environmental and economic floodplain conditions.

Arvidson, Raymond E.↗

Visualization of Computational Fluid Dynamics

Scientific visualization serves the dual purpose of exploration and exposition of the results of numerical simulations of fluid flow. Along with the basic visualization process which transforms source data into images, there are four additional components to a complete visualization system: Source Data Processing, User Interface and Control, Presentation, and Information Management. The requirements imposed by the desired mode of operation (i.e. real-time, interactive, or batch) and the source data have their effect on each of these visualization system components. The special requirements imposed by the wide variety and size of the source data provided by the numerical simulation of fluid flow presents an enormous challenge to the visualization system designer. We describe the visualization system components including specific visualization techniques and how the mode of operation and source data requirements effect the construction of computational fluid dynamics visualization systems.

Gerald-Yamasaki, Michael↗

Societal Benefits of Ocean Altimetry Data

The NASA/CNES Jason satellite, follow-on to the highly successful TOPEX/Poseidon mission, continues to provide oceanographers and marine operators across the globe with a continuous twelve-year, high quality stream of sea surface height data. The mission is expected to extend through 2007, when the NASA/NOAA/CNES follow-on mission, OSTM, will be launched with the wide-swath ocean altimeter on board. This unprecedented resource of valuable ocean data is being used to map sea surface height, geostrophic velocity, significant wave height, and wind speed over the global oceans. Altimeter data products are currently used by hundreds of researchers and operational users to monitor ocean circulation and improve our understanding of the role of the oceans in climate and weather. Ocean altimeter data has many societal benefits and has proven invaluable in many practical applications including; a) Ocean forecasting systems; b) Climate research and forecasting; c) Ship routing; d) Fisheries management; e) Marine mammal habitat monitoring; f) Hurricane forecasting and tracking; g) Debris tracking; and h) Precision marine operations such as cable-laying and oil production. The data has been cited in nearly 2,000 research and popular articles since the launch of TOPEX/Poseidon in 1992, and almost 200 scientific users receive the global coverage altimeter data on a monthly basis. In addition to the scientific and operational uses of the data, the educational community has seized the unique concepts highlighted by these altimeter missions as a resource for teaching ocean science to students from grade school through college. This presentation will highlight societal benefits of ocean altimetry data in the areas of climate studies, marine operations, marine research, and non-ocean investigations.

sea surface height↗