Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data integrity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

VA EDH Advanced Software Pipeline Framework Report: Enhancing Automation and Scalability

The VA Environmental Determinants of Health (EDH) Advanced Software Pipeline Framework is designed to enhance the efficiency, scalability, and security of geospatial data processing workflows. This framework integrates modern data orchestration and containerization technologies, including Prefect for workflow automation, Docker for containerization, and PostgreSQL/PostGIS for geospatial data storage and analysis. It ensures standardized, reproducible, and automated data processing, supporting VA objectives related to substance use risk assessment and recovery research. The pipeline addresses key scalability and performance challenges through horizontal and vertical scaling, high-performance computing (HPC) integration, parallel processing, task caching, and dynamic resource allocation. These optimizations improve throughput and reduce latency, allowing the system to efficiently manage large and complex datasets. Additionally, security and compliance measures—such as data encryption (SSL), Role-Based Access Control (RBAC), and adherence to GDPR and HIPAA standards—safeguard sensitive information throughout data transmission and storage. A key implementation of this framework includes the automation of shelter list geolocation workflows, ensuring that up-to-date data is readily available for VA decision-making. Lessons learned from this project include the transition from in-memory processing to incremental storage writes, improving resource management and reliability. Future enhancements aim to expand automation, integrate AI-driven anomaly detection, and incorporate high-performance computing resources. This framework provides a scalable, secure, and adaptable solution for managing geospatial datasets, reinforcing the VA’s ability to support clinical and strategic initiatives through data-driven decision-making.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Congruity of genomic and epidemiological data in modelling of local cholera outbreaks

Cholera continues to be a global health threat. Understanding how cholera spreads between locations is fundamental to the rational, evidence-based design of intervention and control efforts. Traditionally, cholera transmission models have used cholera case-count data. More recently, whole-genome sequence data have qualitatively described cholera transmission. Integrating these data streams may provide much more accurate models of cholera spread; however, no systematic analyses have been performed so far to compare traditional case-count models to the phylodynamic models from genomic data for cholera transmission. Here, we use high-fidelity case-count and whole-genome sequencing data from the 1991 to 1998 cholera epidemic in Argentina to directly compare the epidemiological model parameters estimated from these two data sources. We find that phylodynamic methods applied to cholera genomics data provide comparable estimates that are in line with established methods. Our methodology represents a critical step in building a framework for integrating case-count and genomic data sources for cholera epidemiology and other bacterial pathogens.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2009

Integrating and Visualizing Tropical Cyclone Data Using the Real Time Mission Monitor

The Real Time Mission Monitor (RTMM) is a visualization and information system that fuses multiple Earth science data sources, to enable real time decision-making for airborne and ground validation experiments. Developed at the NASA Marshall Space Flight Center, RTMM is a situational awareness, decision-support system that integrates satellite imagery, radar, surface and airborne instrument data sets, model output parameters, lightning location observations, aircraft navigation data, soundings, and other applicable Earth science data sets. The integration and delivery of this information is made possible using data acquisition systems, network communication links, network server resources, and visualizations through the Google Earth virtual globe application. RTMM is extremely valuable for optimizing individual Earth science airborne field experiments. Flight planners, scientists, and managers appreciate the contributions that RTMM makes to their flight projects. A broad spectrum of interdisciplinary scientists used RTMM during field campaigns including the hurricane-focused 2006 NASA African Monsoon Multidisciplinary Analyses (NAMMA), 2007 NOAA-NASA Aerosonde Hurricane Noel flight, 2007 Tropical Composition, Cloud, and Climate Coupling (TC4), plus a soil moisture (SMAP-VEX) and two arctic research experiments (ARCTAS) in 2008. Improving and evolving RTMM is a continuous process. RTMM recently integrated the Waypoint Planning Tool, a Java-based application that enables aircraft mission scientists to easily develop a pre-mission flight plan through an interactive point-and-click interface. Individual flight legs are automatically calculated "on the fly". The resultant flight plan is then immediately posted to the Google Earth-based RTMM for interested scientists to view the planned flight track and subsequently compare it to the actual real time flight progress. We are planning additional capabilities to RTMM including collaborations with the Jet Propulsion Laboratory in the joint development of a Tropical Cyclone Integrated Data Exchange and Analysis System (TC IDEAS) which will serve as a web portal for access to tropical cyclone data, visualizations and model output.

Goodman, H. Michael↗
DOE OSTI2022

A Simple and Accurate Energy-Detector-Based Transient Waveform Detection for Smart Grids: Real-World Field Data Performance

Integration of distributed energy sources, advanced meshed operation, sensors, automation, and communication networks all contribute to autonomous operations and decision-making processes utilized in the grid. Therefore, smart grid systems require sophisticated supporting structures. Furthermore, rapid detection and identification of disturbances and transients are a necessary first step towards situationally aware smart grid systems. This way, high-level monitoring is achieved and the entire system kept operational. Even though smart grid systems are unavoidably sophisticated, low-complexity algorithms need to be developed for real-time sensing on the edge and online applications to alert stakeholders in the event of an anomaly. In this study, the simplest form of anomaly detection mechanism in the absence of any a priori knowledge, namely, the energy detector (also known as radiometer in the field of wireless communications and signal processing), is investigated as a triggering mechanism, which may include automated alerts and notifications for grid anomalies. In contrast to the mainstream literature, it does not rely on transform domain tools; therefore, utmost design and implementation simplicity are attained. Performance results of the proposed energy detector algorithm are validated by real power system data obtained from the DOE/EPRI National Database of power system events and the Grid Signature Library.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Big Data For Operation and Maintenance Cost Reduction

The purpose of this research is to develop a first-of-a-kind framework for integrating Big Data capability into the daily activities of our current fleet of nuclear power plants. Big Data is traditionally defined as data sets with high volume, velocity, and heterogeneity, and the existing Big Data analytics capabilities are now widely popular in fields such as finance, weather, e-commerce, healthcare and sports. In the nuclear industry, while the volume and velocity of data may present computational challenges for existing analytics capabilities, data heterogeneity are seen to present the major challenge. This research project mainly focuses on incorporating the wide range of data heterogeneities in nuclear power plants into an integrated Big Data Analytics capability. The primary end-product of this project is a Big Data framework that is capable of dealing with the large volume and heterogeneity of the data found in nuclear power plants to extract timely and valuable information on equipment performance. The framework can generate system insights that are actionable relations between measurable impacts and the corresponding maintenance action plans and enable optimization of plant operation and maintenance based on the extracted information. The developed framework is capable of handling heterogeneous data including both image data and time-series sensor data. Specifically, this developed framework includes the following components. The first component is an overarching maintenance ontology which includes system insights required by maintenance optimization. The maintenance ontology interacts with other components in the developed framework. The second component handles Piping & Instrumentation Diagram (P&ID) data. It can be used to extract system components and their relations automatically from the P&IDs. This extracted information is stored in the first component, i.e., maintenance ontology, and is also used as input to the third component, i.e., a tool for generating the fault tree for the corresponding system. The generated fault tree in turn is stored in the ontology for assessing risk that is used as a criterion in maintenance policy optimization. The fourth component is a tool for inferring the parameters in the Markov degradation model for a nuclear system. It uses basic information from the ontology. The fifth component is a tool for assessing the degradation level using sensor measurement data, for example, pressure, flowrate. This tool can be used for determining corrective maintenance actions. The results obtained from components four and five are returned to the ontology. The sixth component of the framework is a tool for optimizing the maintenance policy for a nuclear system of interest. It takes certain basic information from the ontology, e.g., costs of maintenance actions and system failures, as input, and returns the optimal maintenance policy to the ontology. This tool can be used for determining predictive maintenance actions. A set of experiments have also been conducted to verify the algorithms developed in this project for nuclear system degradation monitoring. The experiments are based on four solenoid valves, similar to the ones used in nuclear power plants. The analyses based on the experimental data using two algorithms, i.e., the Randomized Window Decomposition (RWD) algorithm and the particle filtering algorithm, and the results are introduced in the report. The Big Data framework developed in this project can be used as a support tool in daily activities of plant operation and maintenance and will reduce current costs while maintaining or improving safety levels. Overall, the project will not only benefit existing reactors, however it will open new frontiers to realize the long overdue value of Big Data Analytics in the nuclear sphere.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1977

MIUS Integration and Subsystem Test (MIST) data system

A data system for use in testing integrated subsystems of a modular integrated utility system (MIUS) is presented. The MIUS integration and subsystem test (MIST) data system is reviewed from its conception through its checkout and operation as the controlling portion of the MIST facility. The MIST data system provides a real time monitoring and control function that allows for complete evaluation of the performance of the mechanical and electrical subsystems, as well as controls the operation of the various components of the system. In addition to the aforementioned capabilities, the MIST data system provides computerized control of test operations such that minimum manpower is necessary to set up, operate, and shut down subsystems during test periods.

Pringle, L. M.↗
NASA NTRS1989

International standards activities in image data compression

Integrated Services Digital Network (ISDN); coding for color TV, video conferencing, video conferencing/telephone, and still color images; ISO color image coding standard; and ISO still picture standard are briefly discussed. This presentation is represented by viewgraphs only.

Haskell, Barry↗
NASA NTRS2017

Principal Modes of Precipitation Variability from Preliminary Series of IMERG Data

The Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement (GPM) mission, IMERG, is the unified U.S. algorithm that provides merged Microwave Infrared (IR) satellite precipitation product for the U.S. GPM team. Even though IMERG record is still very short, 2014-2016, it is tempting to test if it captures ENSO and NAO signals as compared to the popular, still on-going, TRMM Multi-satellite Precipitation Analysis, TMPA. El Nino Southern Oscillation (ENSO) is the most significant mode of interannual variability of tropical ocean atmosphere. North Atlantic Oscillation (NAO) impact is on monthly scales and is mostly an atmospheric mode in the North Atlantic. There exist well-defined, multivariate, indexes that represent ENSO and NAO conditions and phase.

climate variability↗
NASA NTRS2018

An “EPIIC” Vision to Evolve Project Integration, Innovation, and Collaboration with Broad Impact for How NASA Executes Complex Projects

Evolving Project Integration, Innovation, and Collaboration (EPIIC) is a vision defined to transform the way projects manage information to support real-time decisions, capture best practices and lessons learned, perform assessments, and manage risk across a portfolio of projects. The foundational project management needs for data and information will be revolutionized through innovations on how we manage and access that data, implement configuration control, and certify compliance. The embedded intelligence of new interactive data interfaces integrate technical and programmatic data such that near real time analytics can be accomplished to more efficiently and accurately complete systems engineering and project management tasks. The system-wide data analytics that are integrated into customized data interfaces allows the growing team of engineers and managers required to develop and implement major NASA missions the ability to access authoritative source(s) of system information while greatly reducing the labor required to complete system assessments. This would allow, for example, much of what is accomplished in a scheduled design review to take place as needed, between any team members, at any time. An intelligent data interface that rigorously integrates systems engineering and project management information in near real time can provide substantially greater insight for systems engineers, project managers, and the large diverse teams required to complete a complex project. System engineers, programmatic personnel (those who focus on cost, schedule, and risk), the technical engineering disciplines, and project management can realize immediate benefit from the shared vision described herein. Implementation of the vision also enables significant improvements in the performance of the engineered system being developed.

Moreland, Robert J.↗
NASA NTRS2015

NASA's ATM Ontology: Semantic Integration and Querying Across NAS Data Sources

NASA is developing an Air Traffic Management (ATM) Ontology as part of an advanced prototyping activity that demonstrates the utility of semantic technologies for integrating, querying, and searching over various sources of heterogeneous ATM data. The ontology encodes an overarching data model that functions as the backbone upon which to overlay data from multiple sources published by FAA, NOAA, NASA, and others. The integrated data can be queried to produce results not achievable using any single source alone. The ontology incorporates flight data, weather data, traffic management advisory data, airport delay data, and national airspace infrastructure data for a very limited spatial and temporal slice of airspace operations (one day of operations at a major airport).

air traffic management↗
NASA NTRS2017

Data Reduction Pipeline for the CHARIS Integral-Field Spectrograph I: Detector Readout Calibration and Data Cube Extraction

We present the data reduction pipeline for CHARIS, a high-contrast integral-field spectrograph for the Subaru Telescope. The pipeline constructs a ramp from the raw reads using the measured nonlinear pixel response and reconstructs the data cube using one of three extraction algorithms: aperture photometry, optimal extraction, or chi-squared fitting. We measure and apply both a detector flatfield and a lenslet flatfield and reconstruct the wavelength- and position-dependent lenslet point-spread function (PSF) from images taken with a tunable laser. We use these measured PSFs to implement a chi-squared-based extraction of the data cube, with typical residuals of approximately 5 percent due to imperfect models of the under-sampled lenslet PSFs. The full two-dimensional residual of the chi-squared extraction allows us to model and remove correlated read noise, dramatically improving CHARIS's performance. The chi-squared extraction produces a data cube that has been deconvolved with the line-spread function and never performs any interpolations of either the data or the individual lenslet spectra. The extracted data cube also includes uncertainties for each spatial and spectral measurement. CHARIS's software is parallelized, written in Python and Cython, and freely available on github with a separate documentation page. Astrometric and spectrophotometric calibrations of the data cubes and PSF subtraction will be treated in a forthcoming paper.

Brandt, Timothy D.↗
DOE OSTI2026

Web-based Preprocessing and Visualization of 3D FIB Tomography Data for Nuclear Fuel Characterization

Three-dimensional (3D) focused ion beam (FIB) tomography enables reconstruction of internal nuclear fuel features that can't be fully evaluated through surface imaging alone. This capability supports characterization of fuel constituents and defects under thermal and irradiation conditions relevant to microreactor development. However, large tomography datasets can create data-handling, loading, and visualization challenges, especially when image-stack preparation and file conversion must be completed with separate tools. The Computational Ultraspatial Tomography Toolkit for High-Resolution Object Analysis Tools (CUTTRHOAT) is an open-source web application being developed to display FIB tomography datasets available through the Nuclear Research Data System (NRDS). The current alpha version requires prepared HDF5 datasets and has limited integrated data-preparation capabilities. This project improves CUTTHROAT by adding dataset-folder selection, automatic input detection, dataset scanning, missing-slice identification, blank-slice insertion, and image-stack-to-HDF5 conversion. Two applications will be compared: the baseline CUTTHROAT alpha workflow and the updated application containing the integrated data-handling and preprocessing functions. Evaluation will consider dataset detection accuracy, conversion success, loading time, rendering responsiveness, application stability, and user interaction. Preliminary results demonstrate successful loading of existing HDF5 files and converted image stacks, while testing also identified performance reductions caused by excessive blank-slice generation. The updated workflow reduces reliance on external preparation tools and supports more direct movement from image stacks to color-code 3D visualization. Future work includes refining missing-slice handling, integrating additional preprocessing functions, like a denoising feature, parsing TIFF metadata for automatic voxel scaling, and adding manual X, Y, and Z voxel-spacing inputs for PNG and JPEG.

36 - MATERIALS SCIENCE↗
DOE OSTI2025

Data for "Discovery, Characterization, and Application of Chromosomal Integration Sites in the Hyperthermophilic Archaeon Sulfolobus islandicus"

Sulfolobus islandicus , an emerging archaeal model organism, offers unique advantages for metabolic engineering and synthetic biology applications owing to its ability to thrive in extreme environments. Although several genetic tools have been established for this organism, the lack of well-characterized chromosomal integration sites has limited its potential as a cellular factory. Here, we systematically identified and characterized 13 artificial CRISPR RNAs targeting eight integration sites in S. islandicus using the CRISPR-COPIES pipeline and a multi-omics-informed computational workflow. We leveraged the endogenous CRISPR-Cas system to integrate the reporter gene lacS and validated heterologous expression through a β-galactosidase assay, revealing significant positional effects. As a proof of concept, we utilized these sites to genetically manipulate lipid ether composition by overexpressing glycerol dibiphytanyl glycerol tetraether (GDGT) ring synthase B (GrsB). This study expands the genetic toolbox for S. islandicus and advances its potential as a robust platform for archaeal synthetic biology and industrial biotechnology.

AI/ML↗
DOE OSTI2022

Computational prediction of plant metabolic pathways

Uncovering genes encoding enzymes responsible for the biosynthesis of diverse plant metabolites is essential for metabolic engineering and production of plant metabolite-derived medicine. With the availability of multi-omics data for an ever-increasing number of plant species and the development of computational approaches, the metabolic pathways of many important plant compounds can be predicted, complementing a more traditional genetic and/or biochemical approach. In this work, we summarize recent progress in predicting plant metabolic pathways using genome, transcriptome, proteome, interactome, and/or metabolome data, and the utility of integrating these data with machine learning to further improve metabolic pathway predictions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Deep Learning Advances Arctic River Water Temperature Predictions

The accelerated warming in the Arctic poses serious risks to freshwater ecosystems by altering streamflow and river thermal regimes. However, limited research on Arctic River water temperatures exists due to data scarcity and the absence of robust methodologies, which often focus on large, major river basins. To address this, we leveraged the newly released, extensive AKTEMP data set and advanced machine learning techniques to develop a Long Short-Term Memory (LSTM) model. By incorporating ERA5-Land reanalysis data and integrating physical understanding into data-driven processes, our model advanced river water temperature predictions in ungauged, snow- and permafrost-affected basins in Alaska. Our model outperformed existing approaches in high-latitude regions, achieving a median Nash-Sutcliffe Efficiency of 0.95 and root mean squared error of 1.0°C. The LSTM model learned air temperature, soil temperature, solar radiation, and thermal radiation—factors associated with energy balance—were the most important drivers of river temperature dynamics. Soil moisture and snow water equivalent were highlighted as critical factors representing key processes such as thawing, melting, and groundwater contributions. Glaciers and permafrost were also identified as important covariates, particularly in seasonal river water temperature predictions. Our LSTM model successfully captured the complex relationships between hydrometeorological factors and river water temperatures across varying timescales and hydrological conditions. This scalable and transferable approach can be potentially applied across the Arctic, offering valuable insights for future conservation and management efforts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Analyzing the Multiscale Impacts of Implementing Energy-Efficient HVAC Improvements Through Energy Audits and Economic Input–Output Analysis

Abstract Heating, ventilation, and air-conditioning (HVAC) systems are usually an industry’s highest consumer of energy, most of which goes toward space cooling in buildings. Industrial energy-efficiency audits not only benefit manufacturers but also generate significant economic and environmental benefits to localities, states, and the nation. This article analyzes the micro- and macro scale impacts of implementing energy-efficient HVAC systems by integrating the industrial building energy data with the macroeconomic regional economic flow model. Micro-scale data include 10 years of historical energy, cost, and carbon dioxide savings achieved from energy-efficient HVAC implementation offered to manufacturers through industrial energy audits. The data were integrated into the macroeconomic modeling framework to illuminate the cascading regional economic impacts of implementing energy-efficient HVAC recommendations in manufacturing facilities. Results show that if recommendations had been implemented throughout all manufacturers in the region, $656 M energy costs would have been directly saved, 7.8 million metric tons of carbon dioxide emissions would have been avoided, and 4387 jobs could have been created, resulting in a total annual economic impact of $899 M stemming from direct, indirect, and induced impacts. The results offer insight into how industrial energy systems can be designed and provide models for how communities can accomplish a net-zero society.

Energy & Fuels↗
DOE OSTI2021

Geophysical Retrievals in an Artificial Intelligence (AI) Framework for Illuminating Processes Controlling Water Cycle

Focal Area(s): This white paper responds to Focal Area #3: Insight gleaned from complex data (both observed and simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI. Science Challenge: This white paper addresses the water-cycle and data-model integration grand challenge. It leverages data from the Atmospheric Radiation Measurement (ARM) Climate Research Facility, and Next-Generation Ecosystem Experiment (NGEE), and Science Focus Area (SFA). The white paper focuses on controlling cloud, precipitation, and radiative properties as observed and simulated by the Earth System Models (ESM). The described framework can be readily applied to any other ensemble of instruments, including satellites and other ground-based networks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Data Fusion to Enhance Quality Control and Analysis with Instruments at the Marine and Coastal Research Laboratory

Deploying environmental monitoring instruments in the marine environment can be challenging, facing challenges around device survivability, biofouling and corrosion, and consistent data collection. This project explores the use of data fusion – the process of integrating multiple data sources to produce more consistent, accurate, and useful information – to build a consistent long-term monitoring system at the Marine and Coastal Research Laboratory (MCRL) in Sequim, Washington. Unused instruments that had been acquired from past projects were inventoried and deployments planned on the MCRL pier and floating dock. A total of 8 instruments were deployed including a tide gauge, hydrophone, acoustic Doppler current profiler (ADCP), photosynthetically active radiation (PAR) sensors, meteorological station, and three water quality sensors. Deployments were planned to be well-protected around the pier structure and a maintenance schedule was created for cleaning and recalibration. An automated data pipeline was created to aggregate data on edge computers that push data to Amazon Web Services (AWS) cloud storage every 15 minutes, performing automated quality control and data transformations using the Time Series Data Analytical Toolkit (TSDAT). Continued efforts are underway to maintain this system into the future, take a data-driven approach to maintenance scheduling, improve the reliability of the system, and share the data with a variety of end-users.

54 ENVIRONMENTAL SCIENCES↗