Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data requirements”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

International interlaboratory compilation of trace element concentrations in the CUP-2 uranium ore concentrate standard

Herein, a nuclear forensics investigation involving a uranium ore concentrate relies on accurate and precise analysis of impurities. Analytical data defensibility requires the use of reference materials as part of quality control. This study presents a compilation of trace element concentration results of the CUP-2 Uranium Ore Concentrate Standard measured by 11 different laboratories. The laboratories employed various dissolution methods, analytical preparation methods, and instrumental platforms. The data presented here contain concentrations of 66 impurities with up to 138 individual data points for each impurity. Consensus values have been assigned to each impurity following a statistical analysis of the data set.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2025

Networked Microgrid Energy Management Considering Ownership and Control Structures: A Comparison

With the growing deployment of microgrids, networked microgrids have emerged for their additional advantages of economy, reliability and resilience by coordinating the operation of multiple microgrids. As microgrids are operated with different ownership, objectives and functionalities, the formed networked microgrids show characteristics of mixed ownership, inconsistent objectives and various functionalities. To enable the coordinated operation of networked microgrids, three control structures, i.e., centralized, distributed and decentralized, have been constructed in the literature. However, the data sharing enabling these different paradigms and the resulted value propositions are not well defined, leading to poor resource management and resilience, etc. To solve this issue, a complete comparison of networked microgrid energy management under centralized, distributed and decentralized structures are performed. As a novel contribution, the required minimum data exchange of networked microgrid energy management under three control structures are identified, respectively. The value propositions are calculated and compared against each other by the results of case studies.

control structures↗
DOE OSTI2024

Automating data analysis for hydrogen/deuterium exchange mass spectrometry using data-independent acquisition methodology

We present a hydrogen/deuterium exchange workflow coupled to tandem mass spectrometry (HX-MS 2 ) that supports the acquisition of peptide fragment ions alongside their peptide precursors. The approach enables true auto-curation of HX data by mining a rich set of deuterated fragments, generated by collisional-induced dissociation (CID), to simultaneously confirm the peptide ID and authenticate MS 1 -based deuteration calculations. The high redundancy provided by the fragments supports a confidence assessment of deuterium calculations using a combinatorial strategy. The approach requires data-independent acquisition (DIA) methods that are available on most MS platforms, making the switch to HX-MS 2 straightforward. Importantly, we find that HX-DIA enables a proteomics-grade approach and wide-spread applications. Considerable time is saved through auto-curation and complex samples can now be characterized and at higher throughput. We illustrate these advantages in a drug binding analysis of the ultra-large protein kinase DNA-PKcs, isolated directly from mammalian cells.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

NETL CO2U LCA Documentation Spreadsheet

The NETL CO2U LCA Documentation Spreadsheet is part of the NETL CO2U LCA Guidance Toolkit for Carbon Utilization funding recipients to meet their LCA requirements. The toolkit includes the following files: NETL CO2U LCA Guidance Document, NETL CO2U openLCA LCI Database, NETL CO2U openLCA Results Contribution Tool, NETL CO2U LCA Documentation Spreadsheet, and NETL CO2U LCA Report Template. The NETL CO2U LCA Documentation Spreadsheet is available to funding recipients to help meet their LCA data documentation requirements, especially when using a spreadsheet model rather than LCA software to complete the LCA requirements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Power modeling of degraded PV systems: Case studies using a dynamically updated physical model (PV-Pro)

Power modeling, widely applied for health monitoring and power prediction, is crucial for the efficiency and reliability of Photovoltaic (PV) systems. The most common approach for power modeling uses a physical equivalent circuit model, with the core challenge being the estimation of model parameters. Traditional parameter estimation either relies on datasheet information, which does not reflect the system's current health status, especially for degraded PV systems, or requires additional I-V characterization, which is generally unavailable for large-scale PV systems. Thus, we build upon our previously developed tool, PV-Pro (originally proposed for degradation analysis), to enhance its application for power modeling of degraded PV systems. PV-Pro extracts model parameters from production data without requiring I-V characterization. This dynamic model, periodically updated, can closely capture the actual degradation status, enabling precise power modeling. PV-Pro is compared with popular power modeling techniques, including persistence, nominal physical, and various machine learning models. The results indicate that PV-Pro achieves outstanding power modeling performance, with an average nMAE of 1.4 % across four field-degraded PV systems, reducing error by 17.6 % compared to the best alternative technique. Furthermore, PV-Pro demonstrates robustness across different seasons and severities of degradation. The tool is available as a Python package at https://github.com/DuraMAT/pvpro.

14 SOLAR ENERGY↗
DOE OSTI2023

Data, figures, animations, and scripts associated with the manuscript "Integrated Effects of Site Hydrology and Vegetation on Exchange Fluxes and Carbon Cycling at the Coastal Terrestrial Aquatic Interface"

This package contains the data, figures, animations, and scripts used in "Integrated Effects of Site Hydrology and Vegetation on Exchange Fluxes and Carbon Cycling at the Coastal Terrestrial Aquatic Interface" (Li et al., 2023). This study examines the interactions between soil, vegetation and hydrologic conditions in coastal areas, with a focus on the Chesapeake Bay region. The researchers used a mechanistic model called ATS-PFLOTRAN to explore how these interactions affect exchange process in different regions and the carbon/nitrogen cycle across the terrestrial-aquatic interface (TAI). Simulation scenarios isolate the effects of control factors and reaction constants derived from laboratory experiments. The results show a carbon cycle "hot zone" in coastal wetlands and in the transition zone between wetlands and uplands. Transpiration enhances fluxes between the surface and subsurface domains and increases dissolved oxygen in TAI. The decomposition of leaf-derived organic carbon provides an additional source of carbon for aerobic respiration and denitrification in the TAI. Microbial activity plays a key role in controlling redox conditions and their variability. This modeling study improves the understanding of complex TAI interactions and facilitates the representation of coastal ecosystems in larger-scale Earth system models.Several files can be found from this data package.1. Readme,md: This file describe the Title, Target Journal, Target submission date, Co-author, Science Questions, Hypotheses, Key words, Key message, Model and Data, Repo Structure. The user can read this file first and then go to details.2. mesh.zip: This file contains the mesh file for simulation cases. 3. Figure.zip: This file contains figures used in the manuscript . 4. animation.zip: This file contains animations used in the manuscript . 5. Data.zip: This file contains all required input data, such as concentration and flow boundaries. Also, it contains the DEM and processes results on area fraction. 6. Simulation_setup_and_results.zip: This file contains setup of cases used in the manuscript . The result files are too large, if you need that data, please contact to the Author. 7. notebooks.zip: This file contains the Jupyter notebooks for performing sensitivity analysis and other pre-process and post-process analyses.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Out-of-Pile Test of LVDT-Based Creep Test Rig at PWR Prototypical Conditions

New and improved materials are being considered to support the existing nuclear reactors and future next-generation reactors. The materials can significantly degrade and limit their properties in harsh reactor environments. To accurately understand the material’s degradation, real-time data is required under prototypic-irradiation conditions. Additionally, understanding the creep behavior of materials under harsh environments is essential for an evaluation of safety concerns. To provide these capabilities, the Idaho National Laboratory’s (INL’s) High Temperature Test Laboratory (HTTL) has developed several instrumented test rigs to obtain real-time data from specimens in well-controlled pressurized water reactor (PWR) coolant conditions in the Materials and Test Reactors (MTRs). This technical report focuses on INL’s efforts to evaluate and enhance the former prototype creep test rig that relied on linear variable differential transformers in laboratory settings. Specifically, the test rig is capable of detecting changes in the length of tensile specimen, which is useful for measuring thermal expansion and creep loading.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Error-Bounded Learned Scientific Data Compression with Preservation of Derived Quantities

Scientific applications continue to grow and produce extremely large amounts of data, which require efficient compression algorithms for long-term storage. Compression errors in scientific applications can have a deleterious impact on downstream processing. Thus, it is crucial to preserve all the “known” Quantities of Interest (QoI) during compression. To address this issue, most existing approaches guarantee the reconstruction error of the original data or primary data (PD), but cannot directly control the problem of preserving the QoI. In this work, we propose a physics-informed compression technique that is composed of two parts: (i) reduction of the PD with bounded errors and (ii) preservation of the QoI. In the first step, we combine tensor decompositions, autoencoders, product quantizers, and error-bounded lossy compressors to bound the reconstruction error at high levels of compression. In the second step, we use constraint satisfaction post-processing followed by quantization to preserve the QoI. To illustrate the challenges of reducing the reconstruction errors of the PD and QoI, we focus on simulation data generated by a large-scale fusion code, XGC, which can produce tens of petabytes in a single day. The results show that our approach can achieve a high compression amount while accurately preserving the QoI within scientifically acceptable bounds.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Accelerating cosmological inference with Gaussian processes and neural networks – an application to LSST Y1 weak lensing and galaxy clustering

ABSTRACT Studying the impact of systematic effects, optimizing survey strategies, assessing tensions between different probes and exploring synergies of different data sets require a large number of simulated likelihood analyses, each of which cost thousands of CPU hours. In this paper, we present a method to accelerate cosmological inference using emulators based on Gaussian process regression and neural networks. We iteratively acquire training samples in regions of high posterior probability which enables accurate emulation of data vectors even in high dimensional parameter spaces. We showcase the performance of our emulator with a simulated 3×2 point analysis of LSST-Y1 with realistic theoretical and systematics modelling. We show that our emulator leads to high-fidelity posterior contours, with an order of magnitude speed-up. Most importantly, the trained emulator can be re-used for extremely fast impact and optimization studies. We demonstrate this feature by studying baryonic physics effects in LSST-Y1 3×2 point analyses where each one of our MCMC runs takes approximately 5 min. This technique enables future cosmological analyses to map out the science return as a function of analysis choices and survey strategy.

Astronomy & Astrophysics↗
DOE OSTI2025

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗
DOE OSTI2021

When to use rsync

We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency ≦ 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Scaling and merging time-resolved pink-beam diffraction with variational inference

Time-resolved x-ray crystallography (TR-X) at synchrotrons and free electron lasers is a promising technique for recording dynamics of molecules at atomic resolution. While experimental methods for TR-X have proliferated and matured, data analysis is often difficult. Extracting small, time-dependent changes in signal is frequently a bottleneck for practitioners. Recent work demonstrated this challenge can be addressed when merging redundant observations by a statistical technique known as variational inference (VI). However, the variational approach to time-resolved data analysis requires identification of successful hyperparameters in order to optimally extract signal. In this case study, we present a successful application of VI to time-resolved changes in an enzyme, DJ-1, upon mixing with a substrate molecule, methylglyoxal. We present a strategy to extract high signal-to-noise changes in electron density from these data. Furthermore, we conduct an ablation study, in which we systematically remove one hyperparameter at a time to demonstrate the impact of each hyperparameter choice on the success of our model. We expect this case study will serve as a practical example for how others may deploy VI in order to analyze their time-resolved diffraction data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

GPU-based Image Compression for Efficient Compositing in Distributed Rendering Applications

Visualizations of large-scale data sets are often created on graphics clusters that distribute the rendering task amongst many processes. When using real-time GPU-based graphics algorithms, the most time-consuming aspect of distributed rendering is typically the com-positing phase - combining all partial images from each rendering process into the final visualization. Compo siting requires image data to be copied off the GPU and sent over a network to other processes. While compression has been utilized in existing distributed rendering compositors to reduce the data being sent over the network, this compression tends to occur after the raw images are transferred from the GPU to main memory. In this paper, we present work that leverages OpenGL / CUDA interoperability to compress raw images on the GPU prior to transferring the data to main memory. This approach can significantly reduce the device-to-host data transfer time, thus enabling more efficient compositing of images generated by distributed rendering applications.

Lipinksi, Riley↗
DOE OSTI2023

Automating the Solar Interconnection Technical Evaluation Process: PREconfiguring and Controlling Inverter SEt-Points (PRECISE)

Utilities are receiving an increasing number of interconnection requests for distributed solar photovoltaic (PV) systems from their customers, requiring a solution to quickly and intelligently perform technical assessments of these requests and to manage these new renewable energy assets. In this paper, we present a stand-alone interconnection tool, PREconfiguring and Controlling Inverter SEt-points (PRECISETM), and how it overcomes the challenges of accelerating technical evaluations and leverages smart inverter functionality to provide local voltage support on an as-needed basis. The paper details the purpose and architecture of PRECISE, including tools, data source requirements, the model-based interconnection evaluation process, and technical results. PRECISE is an integrated software tool that customizes PV inverter settings for utilities by modeling and assessing impacts on local voltages and the need for advanced inverter functions (AIFs) (e.g., volt-VAR and volt-watt). PRECISE maximizes the use of readily available utility data sets to make fast online assessments of incoming PV interconnection requests and makes accept/reject recommendations along with custom settings for AIFs.

ENERGY PLANNING, POLICY, AND ECONOMY,SOLAR ENERGY↗
DOE OSTI2022

Auto-generating databases of Yield Strength and Grain Size using ChemDataExtractor

Abstract The emerging field of material-based data science requires information-rich databases to generate useful results which are currently sparse in the stress engineering domain. To this end, this study uses the’materials-aware’ text-mining toolkit, ChemDataExtractor, to auto-generate databases of yield-strength and grain-size values by extracting such information from the literature. The precision of the extracted data is 83.0% for yield strength and 78.8% for grain size. The automatically-extracted data were organised into four databases: a Yield Strength, Grain Size, Engineering-Ready Yield Strength and Combined database. For further validation of the databases, the Combined database was used to plot the Hall-Petch relationship for, the alloy, AZ31, and similar results to the literature were found, demonstrating how one can make use of these automatically-extracted datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE's) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and helping users access data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data. This paper provides an update on recent improvements made to the GDR's data lakes and automated data pipelines, including: (1) streamlining the data lake intake process, (2) better educating users on the process and requirements through a new data lakes page, (3) adding data lake direct access links to GDR data lake submission pages, (4) implementing a DAS data pipeline to convert DAS data uploaded in SEG-Y format to a standardized hierarchical data format v5 (HDF5), (5) extending this pipeline to encompass data in the GDR data lake, (6) adding metadata requirements for geospatial data, (7) making user interface/user experience (UX) enhancements to the data pipelines' documentation pages, and (8) improving the GDR's data standards and pipelines pages to better guide users in ensuring that their data is standardized by the GDR's automated data pipelines. 2024 Geothermal Resources Council. All rights reserved.

accessibility↗
DOE OSTI2025

SpectraCodec: A Hilbert curve-based method for encoding metadata in mass spectra for machine learning applications (SpectraCodec) v1

Machine learning approaches to mass spectrometry (MS) data analysis require structured metadata for optimal performance. However, current MS file formats necessitate external metadata sources, creating integration challenges that impede analytical workflows. Here, we present a novel approach for encoding metadata directly within mzML files using one-hot encoding of ASCII characters mapped via Hilbert space-filling curves. This strategy embeds metadata in the first spectrum's m/z-intensity space, ensuring persistence with the primary data, eliminating the need for external metadata files, and maintaining compatibility with existing MS software. We demonstrate that the Hilbert curve mapping efficiently utilizes the two-dimensional spectral space while maintaining robust data recovery. This method offers a practical solution for machine learning applications in mass spectrometry by ensuring metadata and spectral data remain unified through all stages of analysis.

Bowen, Benjamin [Lawrence Berkeley National Labora↗
DOE OSTI2025

Machine learning for domain transfer between simulated and experimental 2D X-ray diffraction patterns using generative adversarial networks

X-ray diffraction (XRD) is a well-established technique for analyzing materials at an atomic level. Dynamic compression experiments (DCE), in which materials are subject to extreme pressures, can provide fundamental understanding to pressure-induced phase transitions and compression of the crystal lattice. The analysis of XRD patterns from highly compressed samples is non-trivial given the sparsity of data, high experimental costs, and the fact that the data is often marred with X-ray background and other artifacts. While accurate computational frameworks exist, they solve the forward problem—from structures and orientations to XRD patterns. Solving the inverse problem for 2D experimental diffraction patterns is currently a complex manual process of matching and comparing experimentally observed patterns to computationally generated ones. Machine learning is a promising tool for automating the matching process but often requires data-intensive architectures. Here, in this study, we use a CycleGAN to translate the domain of limited experimental data to a domain in which there is readily available simulated data. This domain shift allows data-intensive machine learning models that have only been trained on simulated XRD patterns to be used in the analysis of experiments.

Brozak, Samantha Jean [Sandia National Laboratorie↗