Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

APMS: An Integrated Suite of Tools for Measuring Performance and Safety

This is a report of work in progress. In it, I summarize the status of the research and development of the Aviation Performance Measuring System (APMS) for managing, processing, and analyzing digital flight-recorded data. The objectives of the NASA-FAA APMS research project are to establish a sound scientific and technological basis for flight-data analysis, to define an open and flexible architecture for flight-data-analysis systems, and to articulate guidelines for a standardized database structure on which to continue to build future flight-data-analysis extensions . APMS will offer to the air transport community an open, voluntary standard for flight-data-analysis software, a standard that will help to ensure suitable functionality, and data interchangeability, among competing software programs. APMS will develop and document the methodologies, algorithms, and procedures for data management and analyses to enable users to easily interpret the implications regarding safety and efficiency of operations. APMS does not entail the implementation of a nationwide flight-data-collection system. It is intended to provide technical tools to ease the large-scale implementation of flight-data analyses at both the air-carrier and the national-airspace levels in support of their Flight Operations and Quality Assurance (FOQA) Programs and Advanced Qualifications Programs (AQP). APMS cannot meet its objectives unless it develops tools that go substantially beyond the capabilities of the current commercially available software and supporting analytic methods that are mainly designed to count special events. These existing capabilities, while of proven value, were created primarily with the needs of air crews in mind. APMS tools must serve the needs of the government and air carriers, as well as air crews, to fully support the FOQA and AQP programs. They must be able to derive knowledge not only through the analysis of single flights (special-event detection), but through statistical evaluation of the performance of large groups of flights. This paper describes the integrated suite of tools that will assist analysts in evaluating the operational performance and safety of the national air transport system, the air carrier, and the air crew.

Statler, Irving C.↗

APMS: An Integrated Suite of Tools for Measuring Performance and Safety

This is a report of work in progress. In it, I summarize the status of the research and development of the Aviation Performance Measuring System (APMS) for managing, processing, and analyzing digital flight-recorded data. The objectives of the NASA-FAA APMS research project are to establish a sound scientific and technological basis for flight-data analysis, to define an open and flexible architecture for flight-data-analysis systems, and to articulate guidelines for a standardized database structure on which to continue to build future flight-data-analysis extensions. APMS will offer to the air transport community an open, voluntary standard for flight-data-analysis software, a standard that will help to ensure suitable functionality, and data interchangeability, among competing software programs. APMS will develop and document the methodologies, algorithms, and procedures for data management and analyses to enable users to easily interpret the implications regarding safety and efficiency of operations. APMS does not entail the implementation of a nationwide flight-data-collection system. It is intended to provide technical tools to ease the large-scale implementation of flight-data analyses at both the air-carrier and the national-airspace levels in support of their Flight Operations and Quality Assurance (FOQA) Programs and Advanced Qualifications Programs (AQP). APMS cannot meet its objectives unless it develops tools that go substantially beyond the capabilities of the current commercially available software and supporting analytic methods that are mainly designed to count special events. These existing capabilities, while of proven value, were created primarily with the needs of air crews in mind. APMS tools must serve the needs of the government and air carriers, as well as air crews, to fully support the FOQA and AQP programs. They must be able to derive knowledge not only through the analysis of single flights (special-event detection), but through statistical evaluation of the performance of large groups of flights. This paper describes the integrated suite of tools that will assist analysts in evaluating the operational performance and safety of the national air transport system, the air carrier, and the air crew.

Statler, Irving C.↗

Issues in using landscape indicators to assess land changes

Landscape indicators, when combined with information about environmental conditions (such as habitat potential, biodiversity, carbon and nutrient cycling, and erosion) and socioeconomic forces, can provide insights about changing ecosystem services. They also provide information about opportunities for improving natural resources management. Landscape indicators rely on data regarding land cover, land management and land functionality. Challenges in using landscape indicators to assess change and effects include (1) measures of land management and attributes that are reliable, robust and consistent for all areas on the Earth do not exist, and thus land cover is more frequently utilized; (2) multiple types of land cover and management are often found within a single landscape and are constantly changing, which complicates measurement and interpretation; and (3) while causal analysis is essential for understanding and interpreting changes in indicator values, the interactions among multiple causes and effects over time make accurate attribution among many drivers of change particularly difficult. Because of the complexity, sheer number of variables, and limitations of empirical data on land changes, models are often used to illustrate and estimate values for landscape indicators, and those models have several problems. Recommendations to improve our ability to assess the effects of changes in land management include refinement of questions to be more consistent with available information and the development of data sets based on systematic measurement over time of spatially explicit land qualities such as carbon and nutrient stocks, water and soil quality, net primary productivity, habitat and biodiversity. Well-defined and consistent land-classification systems that are capable of tracking changes in these and other qualities that matter to society need to be developed and deployed. Because landscapes are so dynamic, it is crucial to develop ways for the scientific community to work together to collect data and develop tools that will enable better analysis of causes and effects and to develop robust management recommendations that will increases land s capacity to meet societal needs in a changing world.

Kline, Keith L↗

Scalable Multi-Facility Workflows for Artificial Intelligence Applications in Climate Research

Earth observation satellites and earth system models are sources of vast, multi-modal datasets that are invaluable for advancing climate and environmental research. However, their scale and complexity pose significant challenges for processing and analysis. In this paper we discuss our experiences in developing and using a scientific research application using an automated multi-facility workflow that orchestrates data collection, preprocessing, artificial intelligence (AI) inferencing, and data movement across diverse computational resources, leveraging the Advanced Computing Ecosystem Testbed at the Oak Ridge Leadership Computing Facility (OLCF). We demonstrate that our workflow can be seamlessly integrated and orchestrated across research facilities managed by different federal agencies, thus allowing users to extract new scientific insights from climate datasets. The experimental results indicate that the multi-facility workflow significantly reduces processing time, enhances scalability, and maintains high efficiency across varying workloads. Notably, our workflow processes 12,000 high-resolution satellite images in just 44 seconds using 80 workers distributed across 10 nodes on the OLCF systems. Such high throughput is essential for dynamic tokenization and sharding of petascale satellite data for distributed AI model training and inferencing at scale across thousands of GPUs.

Kurihana, Takuya [ORNL] (ORCID:0000000156698565)↗

ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

Scientific applications at exascale generate and analyze massive amounts of data. A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems. As such, parallel I/O, the key technology enables moving data between compute nodes and storage, faces monumental challenges from new applications, memory, and storage architectures considered in the designs of exascale systems. As the storage hierarchy is expanding to include node-local persistent memory, burst buffers, etc., as well as disk-based storage, data movement among these layers must be efficient. Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond. In this paper, we describe new capabilities we have developed in Hierarchical Data Format version 5 (HDF5), the most popular parallel I/O library for scientific applications. HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems. The state-of-the-art features we describe include: Virtual Object Layer (VOL), Data Elevator, asynchronous I/O, full-featured single-writer and multiple-reader (Full SWMR), and parallel querying. In this paper, we introduce these features, their implementations, and the performance and feature benefits to applications and other libraries.

97 MATHEMATICS AND COMPUTING↗

Organizing Large Data Sets for Efficient Analyses on HPC Systems

Upcoming exascale applications could introduce significant data management challenges due to their large sizes, dynamic work distribution, and involvement of accelerators such as graphical processing units, GPUs. In this work, we explore the performance of reading and writing operations involving one such scientific application on two different supercomputers. Our tests showed that the Adaptable Input and Output System, ADIOS, was able to achieve speeds over 1TB/s, a significant fraction of the peak I/O performance on Summit. We also demonstrated the querying functionality in ADIOS could effectively support common selective data analysis operations, such as conditional histograms. In tests, this query mechanism was able to reduce the execution time by a factor of five. More importantly, ADIOS data management framework allows us to achieve these performance improvements with only a minimal amount of coding effort.

Gu, Junmin↗

Nuclear Physics Network Requirements Review (Final Report)

The Energy Sciences Network (ESnet) is the high-performance network user facility for the US Department of Energy (DOE) Office of Science (SC) and delivers highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the DOE science mission by connecting all its laboratories and facilities in the US and abroad. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) program and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet interconnects DOE national laboratories, user facilities, and major experiments so that scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large datasets, and access distributed data repositories. ESnet is specifically built to provide Between July 2023 and October 2023, ESnet and the Nuclear Physics program (NP) of the DOE SC organized an ESnet requirements review of NP-supported activities. Preparation for these events included identification of key stakeholders: program and facility management, research groups, and technology providers. Each stakeholder group was asked to prepare formal case study documents about its relationship to the NP program to build a complete understanding of the current, near-term, and long-term status, expectations, and processes that will support the science going forward.

97 MATHEMATICS AND COMPUTING↗

CNR LARA project, Italy: Airborne laboratory for environmental research

The increasing interest for the environmental problems and the study of the impact on the environment due to antropic activity produced an enhancement of remote sensing applications. The Italian National Research Council (CNR) established a new laboratory for airborne hyperspectral imaging, the LARA Project (Laboratorio Aero per Ricerche Ambientali - Airborne Laboratory for Environmental Research), equipping its airborne laboratory, a CASA-212, mainly with the Daedalus AA5000 MIVIS (Multispectral Infrared and Visible Imaging Spectrometer) instrument. MIVIS's channels, spectral bandwidths, and locations are chosen to meet the needs of scientific research for advanced applications of remote sensing data. MIVIS can make significant contributions to solving problems in many diverse areas such as geologic exploration, land use studies, mineralogy, agricultural crop studies, energy loss analysis, pollution assessment, volcanology, forest fire management and others. The broad spectral range and the many discrete narrow channels of MIVIS provide a fine quantization of spectral information that permits accurate definition of absorption features from a variety of materials, allowing the extraction of chemical and physical information of our environment. The availability of such a hyperspectral imager, that will operate mainly in the Mediterranean area, at the present represents a unique opportunity for those who are involved in environmental studies and land-management to collect systematically large-scale and high spectral-spatial resolution data of this part of the world. Nevertheless, MIVIS deployments will touch other parts of the world, where a major interest from the international scientific community is present.

Bianchi, R.↗

The PI-Mode of Project Management

The PI-Mode is NASA's new approach to project management. It responds to the Agency's new policy to develop scientific missions that deliver the highest quality science for a fixed cost. It also attempts to provide more research opportunities by reducing project development times and increasing the number of launches per year. In order to accomplish this, the Principal Investigator is placed at the helm of the project with full responsibility over all aspects of the mission, including instrument and spacecraft development, as well as mission operations and data analysis. This paper intends to study the PI-Mode to determine the strengths and weaknesses of such a new project management technique. It also presents an analysis of its possible impact on the scientific community and its relations with industry, NASA, and other institutions.

Isaac, Dan↗

Experiential findings for sustainable software ecosystems to support experimental and observational science

In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. Here, the study was conducted among 38 ORNL participants between June and July of 2021 and followed the required guidelines for human subjects training. We plan to use the collected information to help guide a vision for sustainable, community-based, and reusable scientific software ecosystems that need to adapt effectively to: (i) the evolving landscape of heterogeneous hardware in the next generation of instruments and computing (e.g. edge, distributed, accelerators), and (ii) data management requirements for data-driven science using artificial intelligence.

97 MATHEMATICS AND COMPUTING↗

Scientific Programming Using Java: A Remote Sensing Example

This paper presents results of a project to port remote sensing code from the C programming language to Java. The advantages and disadvantages of using Java versus C as a scientific programming language in remote sensing applications are discussed. Remote sensing applications deal with voluminous data that require effective memory management, such as buffering operations, when processed. Some of these applications also implement complex computational algorithms, such as Fast Fourier Transformation analysis, that are very performance intensive. Factors considered include performance, precision, complexity, rapidity of development, ease of code reuse, ease of maintenance, memory management, and platform independence. Performance of radiometric calibration code written in Java for the graphical user interface and of using C for the domain model are also presented.

Prados, Don↗

RISK COMMUNICATION FOR ES&H PROFESSIONALS AND LINE SUPERVISORS PARTICIPANT MANUAL COURSE NUMBER 004111 REV 0.1 JULY 2011

This course will help you successfully apply risk communication principles when interacting with workers regarding work hazards. What Is the Difference Between a Risk and a Hazard? In many cases, the terms 'risk' and 'hazard' are incorrectly used interchangeably. A hazard is a physical condition or practice with the potential for causing harm/adverse effects. A risk is the probability of harm/adverse effects occurring from an exposure to a hazard. It is modified by the severity of harm (the consequence). Once a hazard is identified, a risk assessment is conducted to determine the severity of the risk. It uses scientific methods and rigorous tests to determine the effects of the risk on people and the environment. Risk Management takes data from the risk assessment and writes policies to help protect people and the environment against the risk. Risk communication is an ongoing process that starts soon after a hazard is identified and continues through the writing of policies. In the past, governing agencies did not do a good job of communicating risk; they would only communicate about risk after studies had been conducted and policies written.

99 GENERAL AND MISCELLANEOUS↗

Field Test to Evaluate Deep Borehole Disposal.

The U.S. Department of Energy (DOE) has embarked on the Deep Borehole Field Test (DBFT), which will investigate whether conditions suitable for disposal of radioactive waste can be found at a depth of up to 5 km in the earth’s crust. As planned, the DBFT will demonstrate drilling and construction of two boreholes, one for initial scientific characterization, and the other at a larger diameter such as could be appropriate for waste disposal (the DBFT will not involve radioactive waste). A wide range of geoscience activities is planned for the Characterization Borehole, and an engineering demonstration of test package emplacement and retrieval is planned for the larger Field Test Borehole. Characterization activities will focus on measurements and samples that are important for evaluating the long-term isolation capability of the Deep Borehole Disposal (DBD) concept. Engineering demonstration activities will focus on providing data to evaluate the concept’s operational safety and practicality. Procurement of a scientifically acceptable DBFT site and a site management contractor is now underway. The concept of deep borehole disposal (DBD) for radioactive wastes is not new. It was considered by the National Academy of Science (NAS 1957) for liquid waste, studied in the 1980’s in the U.S. (Woodward–Clyde 1983), and has been evaluated by European waste disposal R&D programs in the past few decades (for example, Grundfelt and Crawford 2014; Grundfelt 2010). Deep injection of wastewater including hazardous wastes is ongoing in the U.S. and regulated by the Environmental Protection Agency (EPA 2001). The DBFT is being conducted with a view to use the DBD concept for future disposal of smaller-quantity, DOE-managed wastes from nuclear weapons production (i.e., Cs/Sr capsules and granular solid wastes). However, the concept may also have broader applicability for nations that have a need to dispose of limited amounts of spent fuel from nuclear power reactors. For such nations the cost for disposing of volumetrically limited waste streams could be lower than mined geologic repositories.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

INL WTB Overview Capabilities Future v14_040419

INL Wireless Test Bed Mission Provide highly-regarded cellular, wireless and communications technical expertise, systems and environment to enable: Test & Demonstrate full scale wireless systems and equipment With and without controlled interference Terrestrial, Subterranean and Airborne Demonstrate new & emerging wireless technologies Develop and Train to Tactics, Techniques and Procedures (TTPs) for wireless systems In a real world environment and configurations at real frequencies and real power levels to make informed decision State-of-the-Art INL WTB Assets and Capabilities Instrumentation: RF waveform analyzers, generators Protocol analyzers Noise generators for controlled interference Established Services & Processes: Spectrum approval & monitoring Safety, Medical, Fire, Security (physical) Resource management - personnel, networks, configuration control Secure, IP protected multi-user facility Broadband data access – entire INL Range Hardware prototyping, scientific labs Visitor – US citizen & foreign national

2G↗

GeneLab: Multi-Omics Investigation of Rodent Research-1 Bio-Banked Tissues

NASAs Rodent Research (RR) project is playing a critical role in advancing biomedical research on the physiological effects of space environments. Due to the limited resources for conducting biological experiments aboard the International Space Station (ISS), it is imperative to use crew time efficiently while maximizing high-quality science return. NASAs GeneLab project has as its primary objectives to 1) further increase the value of these experiments using a multi-omics, systems biology-based approach, and 2) disseminate these data without restrictions to the scientific community. The current investigation assessed viability of RNA, DNA, and protein extracted from archived RR-1 tissue samples for epigenomic, transcriptomic, and proteomic assays. During the first RR spaceflight experiment, a variety of tissue types were harvested from subjects, snap-frozen or RNAlater-preserved, and then stored at least a year at -80OC after return to Earth. They were then prioritized for this investigation based on likelihood of significant scientific value for spaceflight research. All tissues were made available to GeneLab through the bio-specimen sharing program managed by the Ames Life Science Data Archive and included mouse adrenal glands, quadriceps, gastrocnemius, tibialis anterior, extensor digitorum longus, soleus, eye, and kidney. We report here protocols for and results of these tissue extractions, and thus, the feasibility and value of these kinds of omics analyses. In addition to providing additional opportunities for investigation of spaceflight effects on the mouse transcriptome and proteome in new kinds of tissues, our results may also be of value to program managers for the prioritization of ISS crew time for rodent research activities. Support from the NASA Space Life and Physical Sciences Division and the International Space Station Program is gratefully acknowledged.

GeneLab↗

What Comes First, the OWL or the Bean? Creating Reusable Scientific Software with OWL/RDF Vocabularies.

In the spring of 2013 the U.S. White House by executive order mandated: “Government information shall be managed as an asset throughout its life cycle to promote interoperability and openness, and, wherever possible and legally permissible, to ensure that data are released to the public in ways that make the data easy to find, accessible, and usable.” Key for the reusability of any scientific data is hereby the availability of metadata describing the published data in a vocabulary that is familiar to its potential users. The objective of this paper is to help scientific application developers who want to adopt the continuous stream of new community vocabularies to help make their data sharable, self-describable, and easily understood. To achieve this we suggest semantic vocabulary and application integration best practices and discuss the tradeoffs of encoding vocabularies through code versus deriving code from vocabularies.

Stephan, Eric G.↗

Reimagining the potential of Earth observations for ecosystem service assessments

The benefits nature provides to people, called ecosystem services, are increasingly recognized and accounted for in assessments of infrastructure development, agricultural management, conservation prioritization, and sustainable sourcing. These assessments are often limited by data, however, a gap with tremendous potential to be filled through Earth observations (EO), which produce a variety of data across spatial and temporal extents and resolutions. Despite widespread recognition of this potential, in practice few ecosystem service studies use EO. Here, we identify challenges and opportunities to using EO in ecosystem service modeling and assessment. Some challenges are technical, related to data awareness, processing, and access. These challenges require systematic investment in model platforms and data management. Other challenges are more conceptual but still systemic; they are byproducts of the structure of existing ecosystem service models and addressing them requires scientific investment in solutions and tools applicable to a wide range of models and approaches. We also highlight new ways in which EO can be leveraged for ecosystem service assessments, identifying promising new areas of research. More widespread use of EO for ecosystem service assessment will only be achieved if all of these types of challenges are addressed. Furthermore, this will require non-traditional funding and partnering opportunities from private and public agencies to promote data exploration, sharing, and archiving. Investing in this integration will be reflected in better and more accurate ecosystem service assessments worldwide.

54 ENVIRONMENTAL SCIENCES↗

Open-Source Data Engineering at NASA: CCMC's Approach to Managing Petabyte-Scale Heliophysics Data

The Community Coordinated Modeling Center (CCMC) at NASA Goddard Space Flight Center (GSFC) leads heliophysics research by providing open access to numerous models and their outputs. Our resources are available on-demand and continuously updated with real-time data, covering sun-earth interactions across multiple domains. These domains include coronal, heliosphere, inner and global magnetosphere, ionosphere, thermosphere, and lower atmosphere interactions. Operating in a hybrid environment, CCMC utilizes both self-owned hardware and Amazon Web Services (AWS) cloud infrastructure. Managing petabytes of data across multiple locations necessitates robust data engineering solutions. To address this challenge, CCMC has adopted industry-standard and open-source tools. We use Apache Airflow as our primary data engineering platform, Python for scripting and data processing, and GitLab for version control and CI/CD. Additionally, we employ Kubernetes for containerized services, Grafana and Prometheus for metrics and monitoring, and Terraform and Puppet for reproducible infrastructure as code. This presentation will discuss lessons learned from our data engineering experiences, platforms evaluated but found unsuitable for our scientific data requirements, and specific techniques developed to enhance data transfer speed and reliability. By using these technologies effectively, CCMC continues to advance heliophysics research through efficient data management and open-access modeling.

space weather↗