Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Enhancing Discoverability and Management of Atmospheric Data at Scale: Solutions from the ARM Data Center

The Atmospheric Radiation Measurement (ARM) is a multi-laboratory and multi-institutional U.S. Department of Energy (DOE) Office of Science National User Facility. The ARM Data Center (ADC), located at Oak Ridge National Laboratory, collects, archives, and shares vast atmospheric data crucial for climate research. The ADC manages over 7 PB of data from 460 instruments worldwide, processing it into more than 11,000 diverse data products using the Network Common Data Form (NetCDF) for machine-independent accessibility. The primary challenge addressed in this paper is the efficient management and distribution of vast and diverse datasets essential for the climate research community, enhancing accessibility through advanced tools like Data Discovery. The ADC has developed advanced infrastructure and software architecture to handle the continuous influx of heterogeneous data to enhance data discoverability, resulting in increased scientific collaboration. In 2023, users from over 34 countries downloaded and utilized ARM data, resulting in 1,455 publications. The ADC’s efforts have significantly improved the discoverability and usability of atmospheric data, fostering extensive scientific research and collaboration. This paper details the solutions implemented by the ADC team for efficient data discovery and distribution, and it demonstrates ARM’s capability of staging processed data for scientific analysis.

Shah, Chirag [ORNL] (ORCID:0000000203145737)↗
DOE OSTI2020

Modeling Water Quality in Watersheds: From Here to the Next Generation

In this synthesis, we assess present research and anticipate future development needs in modeling water quality in watersheds. We first discuss areas of potential improvement in the representation of freshwater systems pertaining to water quality, including representation of environmental interfaces, in-stream water quality and process interactions, soil health and land management, and (peri-)urban areas. In addition, we provide insights into the contemporary challenges in the practices of watershed water quality modeling, including quality control of monitoring data, model parameterization and calibration, uncertainty management, scale mismatches and provisioning of modeling tools. Finally, we make three recommendations to provide a path forward for improving watershed water quality modeling science, infrastructure and practices. These include building stronger collaborations between experimentalists and modelers, bridging gaps between modelers and stakeholders, and cultivating and applying procedural knowledge to better govern and support water quality modeling processes within organizations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A KBase Case Study on Genome-wide Transcriptomics and Plant Primary Metabolism in Sorghum

A better understanding of the genetic and metabolic mechanisms that confer stress resistance and tolerance in plants is key to engineering new crops through advanced breeding technologies. This requires a systems biology approach that builds on a genome-wide understanding of the regulation of gene expression, plant metabolism, physiology and growth. In this study, we examine the response to drought stress in Sorghum, as we leverage the tools for transcriptomics and plant metabolic modeling we have implemented at the U.S. Department of Energy Systems Biology Knowledgebase (KBase). KBase enables researchers worldwide to collaborate and advance research by allowing them to upload private or public data into the KBase Narrative Interface, empowering them to analyze this data using a rich, extensible array of computational and data-analytics tools, and allows them to securely share scientific workflows and conclusions. We demonstrate how to use the current RNA-seq tools in KBase, applicable to both plants and microbes, to assemble and quantify long transcripts and identify differentially expressed genes effectively. More specifically, we demonstrate the utility of the platform by identifying key genes that are differentially expressed during drought-stress in Sorghum bicolor, which is an important sustainable production crop plant. We then show how to use KBase tools to predict the membership of genes in metabolic pathways and examine expression data in the context of metabolic subsystems. We demonstrate the power of the platform by making the data analysis and interpretation available to the biologists in the reproducible, re-usable, point-and-click format of a KBase Narrative thus promoting FAIR (Findable, Accessible, Interoperable and Reusable) guiding principles for scientific data management and stewardship.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Integrating science for water security governance

Hydrological extremes are intensifying globally, increasing the complexity of decisions required to ensure water security. Advances in hydrological science, modeling, and data systems have expanded the technical frontier of water research, yet uptake of scientific insights in policy and management decisions remains limited. This persistent science–policy gap is not primarily a failure of knowledge generation or robustness, but an institutional challenge shaped by how scientific and governance systems are organized, coordinated, and connected to support the effective use of scientific knowledge. These challenges are particularly pronounced in multi-level and transboundary water governance, where decisions span jurisdictions and require coordination across institutional and political boundaries. We synthesize research at the science–policy interface and evidence from water security initiatives to show how institutional arrangements, scientific tool development, and research practices enable or constrain the sustained use of scientific knowledge in water-security governance processes. Building on these insights, we develop ‘shared decision infrastructure’ as a framing to describe how scientific knowledge is embedded within the institutional, relational, and procedural arrangements that connect science to decision-making processes over time. We translate this framing into a practical intervention roadmap centered on institutional design, tool translation, sustained co-production, and outcome-oriented evaluation to support the integration of science into ongoing governance processes. By positioning science as shared decision infrastructure, the roadmap clarifies how researchers can design scientific efforts that support more coordinated, accountable, and adaptive water security decisions amid deepening uncertainty.

M whitney, Kristen [NASA Goddard Space Flight Cent↗
DOE OSTI2020

LDM-148: Rubin Observatory Data Management System Design

The Rubin Observatory LSST Data Management System (DMS) is a set of services employing a variety of software components running on computational and networking infrastructure that combine to deliver science data products to the observatory's users and support observatory operations. This document describes the components, their service instances, and their deployment environments as well as the interfaces among them, the rest of the LSST system, and the outside world.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

A standards perspective on genomic data reusability and reproducibility

Genomic and metagenomic sequence data provides an unprecedented ability to re-examine findings, offering a transformative potential for advancing research, developing computational tools, enhancing clinical applications, and fostering scientific collaboration. However, effective and ethical reuse of genomics data is hampered by numerous technical and social challenges. The International Microbiome and Multi’Omics Standards Alliance (IMMSA, https://www.microbialstandards.org/) and the Genomic Standards Consortium (GSC, https://gensc.org) hosted a 5-part seminar series “A Year of Data Reuse” in 2024 to explore challenges and opportunities of data reuse and reproducibility across disparate domains of the genomic sciences. Addressing these challenges will require a multifaceted approach, including common metadata reporting, clear communication, standardized protocols, improved data management infrastructure, ethical guidelines, and collaborative policies that prioritize transparency and accessibility. We offer strategies to enable responsible and technically feasible data reuse, recognition of data reproducibility challenges, and emphasizing the importance of cross-disciplinary efforts in the pursuit of open science and data-driven innovation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

The Last Journey. I. An Extreme-scale Simulation on the Mira Supercomputer

The Last Journey is a large-volume, gravity-only, cosmological N-body simulation evolving more than 1.24 trillion particles in a periodic box with a side length of 5.025 Gpc. It was implemented using the HACC simulation and analysis framework on the BG/Q system Mira. The cosmological parameters are chosen to be consistent with the results from the Planck satellite. A range of analysis tools have been run in situ to enable a diverse set of science projects and, at the same time, keep the resulting data amount manageable. Analysis outputs have been generated starting at redshift z ~ 10 to allow for construction of synthetic galaxy catalogs using a semianalytic modeling approach in postprocessing. As part of our in situ analysis pipeline, we employ a new method for tracking halo substructures, introducing the concept of subhalo cores. The production of multiwavelength synthetic sky maps is facilitated by generating particle light cones in situ, also beginning at z ~ 10. We provide an overview of the simulation setup and generated data products; a first set of analysis results is presented. A subset of the data is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Community-Driven Methods for Open and Reproducible Software Tools for Analyzing Datasets from Atom Probe Microscopy

Atom probe tomography, and related methods, probe the three-dimensional architecture of a material. The software tools that microscopists use, and how these tools are connected into workflows, makes a substantial contribution to the accuracy and precision of such a material characterization experiment. Typically, we adapt methods from other communities like mathematics, data science, computational geometry, artificial intelligence, or scientific computing. We also realize that improving on research data management is a challenge when it comes to align with the FAIR data stewardship principles. Faced with this global challenge, we are convinced that collaborating is useful. Here, we report the results and challenges with an inter-laboratory call for developing test cases for several types of atom probe software tools. The results support why defining detailed recipes of software workflows and sharing these recipes is necessary and rewarding: Open source tools and (meta)data exchange can help to make our day-to-day data processing tasks become more efficient, the training of new users and knowledge transfer become easier, and assist us with automated quantification of uncertainties to gain access to substantiated results.

36 MATERIALS SCIENCE↗
DOE OSTI2025

LandScan Global 30 Arcsecond Annual Global Gridded Population Datasets from 2000 to 2022

Abstract Oak Ridge National Laboratory (ORNL) annually develops the LandScan Global (LSG) dataset, a 30 arcsecond global gridded population dataset representing global ambient human population distribution. This multivariable dasymetric model disaggregates census counts within administrative boundaries using ancillary data. Each country’s distribution reflects cultural and socioeconomic patterns; manual validations yield a unique global dataset for assessing populations at risk. For over two decades, LSG has been a standard for estimating populations at risk, aiding U.S. federal government, academia and humanitarian organizations. During disasters such as the 2004 Indian Ocean tsunami and the 2010 Haiti earthquake and geopolitical crises such as the Syrian civil war and the 2022 Russian invasion of Ukraine, LSG supported scientific and operational communities in emergency response and recovery. In 2022, LSG datasets from 2000 onward were made publicly available through ORNL’s LandScan Portal. This data descriptor details our methodology and the application of geospatial science and machine learning to geographic and demographic data, highlighting uses in urban resiliency, emergency management, disaster response, and human health and security.

Science & Technology - Other Topics↗
DOE OSTI2024

Guest Editorial for Nondestructive Testing and Evaluation (NDT&E) Special Section

Nondestructive testing and evaluation (NDT&E) are interdisciplinary fields that require a significant amount of interconnection between fundamental physics, measurement techniques, data processing, decision making, and reporting to have a significant impact on industry. NDT&E practitioners are challenged to keep up with the fast-paced evolution of materials, structures, processing, and manufacturing technologies. The development of engineered materials, complex structures and composites, and novel forming techniques require NDT&E to rapidly evolve to meet the needs of industry.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Science Capsule: Towards Sharing and Reproducibility of Scientific Workflows

Workflows are increasingly processing large volumes of data from scientific instruments, experiments and sensors. These workflows often consist of complex data processing and analysis steps that might include a diverse ecosystem of tools and also often involve human-in-the-loop steps. Sharing and reproducing these workflows with collaborators and the larger community is critical but hard to do without the entire context of the workflow including user notes and execution environment. In this paper, we describe Science Capsule, which is a framework to capture, share, and reproduce scientific workflows. Science Capsule captures, manages and represents both computational and human elements of a workflow. It automatically captures and processes events associated with the execution and data life cycle of workflows, and lets users add other types and forms of scientific artifacts. Science Capsule also allows users to create `workflow snapshots' that keep track of the different versions of a workflow and their lineage, allowing scientists to incrementally share and extend workflows between users. Our results show that Science Capsule is capable of processing and organizing events in near real-time for high-throughput experimental and data analysis workflows without incurring any significant performance overheads.

Ghoshal, Devarshi↗
DOE OSTI2024

A Microservices Architecture Toolkit for Interconnected Science Ecosystems

Microservices architecture is a promising approach for developing reusable scientific workflow capabilities for inte- grating diverse resources, such as experimental and observational instruments and advanced computational and data management systems, across many distributed organizations and facilities. In this paper, we describe how the INTERSECT Open Architec- ture leverages federated systems of microservices to construct interconnected science ecosystems, review how the INTERSECT software development kit eases microservice capability develop- ment, and demonstrate the use of such capabilities for deploying an example multi-facility INTERSECT ecosystem.

Brim, Michael↗
DOE OSTI2025

A Grassroots Network and Community Roadmap for Interconnected Autonomous Science Laboratories for Accelerated Discovery

Scientific discovery is being revolutionized by AI and autonomous systems, yet current autonomous laboratories remain isolated islands unable to collaborate across institutions. We present the Autonomous Interconnected Science Lab Ecosystem (AISLE), a grassroots network transforming fragmented capabilities into a unified system that shorten the path from ideation to innovation to impact and accelerates discovery from decades to months. AISLE addresses five critical dimensions: (1) cross-institutional equipment orchestration, (2) intelligent data management with FAIR compliance, (3) AI-agent driven orchestration grounded in scientific principles, (4) interoperable agent communication interfaces, and (5) AI/ML-integrated scientific education. By connecting autonomous agents across institutional boundaries, autonomous science can unlock research spaces inaccessible to traditional approaches while democratizing cutting-edge technologies. This paradigm shift toward collaborative autonomous science promises breakthroughs in sustainable energy, materials development, and public health.

Ferreira da Silva, Rafael [Oak Ridge National Labo↗
DOE OSTI2021

The FEWSION for Community Resilience (F4R) Process: Building Local Technical and Social Capacity for Critical Supply Chain Resilience

Local business leaders, policy makers, elected officials, city planners, emergency managers, and private citizens are responsible for, and deeply affected by, the performance of critical supply chains and related infrastructures. At the center of critical supply chains is the food-energy-water nexus (FEW); a nexus that is key to a community’s wellbeing, resilience, and sustainability. In the 21st century, managing a local FEW nexus requires accurate data describing the function and structure of a community’s supply chains. However, data is not enough; we need data-informed conversation and technical and social capacity building among local stakeholders to utilize the data effectively. There are some resources available at the mesoscale and for food, energy, or water, but many communities lack the data and tools needed to understand connections and bridge the gaps between these scales and systems. As a result, we currently lack the capacity to manage these systems in small and medium sized communities where the vast majority of people, decisions, and problems reside. This study develops and validates a participatory citizen science process for FEW nexus capacity building and data-driven problem solving in small communities at the grassroots level. The FEWSION for Community Resilience (F4R) process applies a Public Participation in Scientific Research (PPSR) framework to map supply chain data for a community’s FEW nexus, to identify the social network that manages the nexus, and then to generate a data-informed conversation among stakeholders. F4R was piloted and co-developed with participants over a 2-year study, using a design-based research process to make evidence-based adjustments as needed. Results show that the F4R model was successful at improving volunteers’ awareness about nexus and supply chain issues, at creating a network of connections and communication with stakeholders across state, regional, and local organizations, and in facilitating data-informed discussion about improvements to the system. In this paper we describe the design and implementation of F4R and discuss four recommendations for the successful application of the F4R model in other communities: 1) embed opportunities for co-created PPSR, 2) build social capital, 3) integrate active learning strategies with user-friendly digital tools, and 4) adopt existing materials and structure.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

2021 Annual Report: Atmospheric Radiation Measurement

In fiscal year 2021 (FY2021), the COVID-19 pandemic continued to have an impact on activities within the Atmospheric Radiation Measurement (ARM) user facility. Travel was limited, which affected field activities and forced the continuation of virtual meetings. However, the ability to travel expanded significantly through the year, bringing some return to normalcy, and throughout the year, there was a great deal of activity dedicated to advancing the facility. Because of COVID, ARM twice delayed the TRacking Aerosol Convection interactions ExpeRiment (TRACER) in the Houston, Texas, area. Originally planned to launch in the spring, TRACER started October 1, 2021, one month after the Surface Atmosphere Integrated Field Laboratory (SAIL) campaign began near Crested Butte, Colorado. ARM teams worked onsite and remotely to make sure both campaigns could launch on this new schedule that was set early in the year. Data from SAIL and TRACER will be critical to improving earth system models, each contributing to different sets of issues. SAIL measurements will provide insights into how precipitation forms and water travels through the Upper Colorado River Basin. TRACER scientists want to know whether tiny atmospheric particles can influence the severity of thunderstorms. In October 2020, the Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition ended after 13 months. MOSAiC data from ARM and other organizations are already helping scientists better understand how ice, atmosphere, and ocean systems are connected in the central Arctic. This report discusses some early investigations from MOSAiC, along with other ARM campaigns and activities generating prolific research. In this report, you will also learn how research activities moved forward during the pandemic at ARM’s fixed-location atmospheric observatories. There were important science applications from measurements across the facility. ARM spent a significant amount of FY2021 looking back—and ahead. In November 2020, ARM had its Triennial Review. This review is held every three years to evaluate ARM’s effectiveness in science, operations, and management. Overall, the reviewers had positive feedback regarding the breadth and impact of science activities using ARM data and the way ARM strives to meet changing user needs. The reviewers also provided some recommendations for strengthening ARM going forward. Incorporating feedback from the review, we finalized an updated Decadal Vision document that will help guide ARM priorities in the coming years. This report describes the four themes driving ARM’s Decadal Vision.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

dCache – Efficient Message Encoding For Inter-Service Communication in dCache: Evaluation of Existing Serialization Protocols as a Replacement for Java Object Serialization

As a well established, large-scale distributed storage system, dCache is used to manage and serve huge amounts of data collected by high energy physics, astrophysics and photon science experiments. Based on a microservices-like architecture, dCache is built as a modular distributed system, where each component provides a different core functionality. These services communicate by passing serialized messages to each other, a core behavior whose performance properties can consequently affect the entire system.This paper compares and evaluates different data serialization protocols in computer science with the objective of replacing and improving upon Java Object Serialization (JOS), which has increasingly presented itself as no longer being sufficiently performant for encoding messages. The criteria for choosing a new framework are collected, analyzed and formalized. The primary motivation for replacing Java serialization for encoding dCache messages is increasing the general speed of message-passing and thereby reducing the round-trip time for user requests. Emphasis is also placed on schema evolution capabilities and framework usability. Approaches for generalizing (de)serialization speed and size measurements based on data structure complexity are introduced, criteria for measuring documentation, learning curve, maintainability and introduction effort are defined. Finally, several selected serialization protocols are evaluated and compared accordingly, concluding with a recommendation for a suitable JOS replacement.

Morschel, Lea↗
DOE OSTI2024

2023 Atmospheric Radiation Measurement (ARM) (Annual Report)

For more than 30 years, the Atmospheric Radiation Measurement (ARM) user facility, which is managed by the U.S. Department of Energy (DOE) Office of Science, has provided freely available data and resources to scientists worldwide. Providing these services effectively requires getting out in the field and engaging with the science community. With COVID travel restrictions behind us, we are focused on connecting in person with ARM users, establishing new partnerships, and supporting fieldwork that might have waited during the pandemic.

54 ENVIRONMENTAL SCIENCES↗