Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data staging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

KSpaceNavigator (KSN)

Intuitive GUI for manipulating microscope stages, allowing to align crystallographic data with stages, allowing to align crystallographic data with stage coordinates and microscope images. Simulates kinematic diffraction patterns and Kikuchi line patterns. Simulated patterns can be displayed as overlay to actually measured data, allowing manual fingerprinting and angular alignment. Crystallographic data is fed to the program in form of CIF (crystallographic information file) files, which are available from many databases and cover virtually all crystal structure ever reported in any journal. Actual goniometer scales can be linearized by lookup tables, program can be used with any microscope goniometer, double tilt and tilt-rotation type. Software requirements: Win32 platform (XP); Compiler/Version: Borland C++ Builder 5; Type of files: Executable modules; Hardware requirement: PC

Duden, Thomas↗
DOE OSTI2023

A fast two-stage algorithm for non-negative matrix factorization in smoothly varying data

This article reports the study of algorithms for non-negative matrix factorization (NMF) in various applications involving smoothly varying data such as time or temperature series diffraction data on a dense grid of points. Utilizing the continual nature of the data, a fast two-stage algorithm is developed for highly efficient and accurate NMF. In the first stage, an alternating non-negative least-squares framework is used in combination with the active set method with a warm-start strategy for the solution of subproblems. In the second stage, an interior point method is adopted to accelerate the local convergence. The convergence of the proposed algorithm is proved. The new algorithm is compared with some existing algorithms in benchmark tests using both real-world data and synthetic data. Furthermore, the results demonstrate the advantage of the algorithm in finding high-precision solutions.

interior point method↗
DOE OSTI2025

The Carbon Storage Technical Viability Approach (CS TVA)

The Carbon Storage Technical Viability Approach (CS TVA) StoryMap provides an in-depth overview of the products created during the CS TVA research effort. In detail, the StoryMap addresses the CS TVA Matrix, Database Version 2.0, Database Catalog, Workflow, and Data Availability Result Database, discussing how each was developed and implemented. Information on how the matrix, database, and database catalog are interconnected, and their usage is also explained. The workflow section provides information on the CS TVA product development from the data-gathering stage to the final data availability results. A section on an expansion of the CS TVA workflow that utilizes Natural Language processing (NLP) section was included. Finally, the Data Availability Results Database is discussed. These results provide data science-informed insights into potential data gaps when assessing the viability of carbon storage in a given area or region.

Carbon Storage↗
DOE OSTI2020

Distributed Computing for the Project 8 Experiment

The Project 8 collaboration aims to measure the absolute neutrino mass or improve on the current limit by measuring the tritium beta decay electron spectrum. We present the current distributed computing model for the Project 8 experiment. Project 8 is in its second phase of data taking with a near continuous data rate of 1Gbps. The current computing model uses DIRAC (Distributed Infrastructure with Remote Agent Control) for its workflow and data management. A detailed meta-data assignment using the DIRAC File Catalog is used to automate raw data transfers and subsequent stages of data processing. The DIRAC system is deployed on containers managed using a Kubernetes cluster to provide a scalable infrastructure. A modified DIRAC Site Director provides the ability to submit jobs using Singularity on opportunistic High-Performance Computing (HPC) sites.

Distributed Computing, Kubernetes, DIRAC, Project ↗
DOE OSTI2022

Data from the Development Evolution of a Vehicle for Custom Control

In order to develop custom controllers intended to operate vehicles on a live highway, a series of data collection-focused tests were performed at increasing stages of complexity. Modern vehicles with features like Adaptive Cruise Control (ACC) feature a rich set of sensors and drive-by-wire mechanisms. The presented stages of data collection begins with the analysis of raw data provided by various vehicles, and eventually results in spoofing Controller Area Network (CAN) protocols for sending control commands to operate a vehicle. This paper covers the data and technical efforts needed at various stages. The raw data and tools to plot the data are also publicly available.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Transitioning from Stage-III to Stage-IV: cosmology from galaxy×CMB lensing and shear×CMB lensing

ABSTRACT We examine the cosmological constraining power from two cross-correlation probes between galaxy and cosmic microwave background (CMB) surveys: the cross-correlation of lens galaxy density with CMB lensing convergence 〈δgκCMB〉, and source galaxy weak lensing shear with CMB lensing convergence 〈γκCMB〉. These two cross-correlation probes provide an independent cross-check of other large-scale structure constraints and are insensitive to galaxy-only or CMB-only systematic effects. In addition, when combined with other large-scale structure probes, the cross-correlations can break degeneracies in cosmological and nuisance parameters, improving both the precision and robustness of the analysis. In this work, we study how the constraining power of 〈δgκCMB〉 + 〈γκCMB〉 changes from Stage-III (ongoing) to Stage-IV (future) surveys. Given the flexibility in selecting the lens galaxy sample, we also explore systematically the impact on cosmological constraints when we vary the redshift range and magnitude limit of the lens galaxies using mock galaxy catalogs. We find that in our setup, the contribution to cosmological constraints from 〈δgκCMB〉 and 〈γκCMB〉 are comparable in the Stage-III data sets; but in Stage-IV surveys, the noise in 〈δgκCMB〉 becomes subdominant to cosmic variance, preventing 〈δgκCMB〉 to further improve the constraints. This implies that to maximize the cosmological constraints from future 〈δgκCMB〉 + 〈γκCMB〉 analyses, we should focus more on the requirements on 〈γκCMB〉 instead of 〈δgκCMB〉. Furthermore, the selection of the lens sample should be optimized in terms of our ability to characterize its redshift or galaxy bias instead of its number density.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Transitioning from Stage-III to Stage-IV: Cosmology from galaxy×CMB lensing and shear×CMB lensing (Final Report for Grant SC0021429 "Cosomology with Galaxy and CMB Surveys: Mitigating Systematic Effects Through Combining Datasets")

We examine the cosmological constraining power from two cross-correlation probes between galaxy and cosmic microwave background (CMB) surveys: the cross-correlation of lens galaxy density with CMB lensing convergence $\langle δ_g$$^κ\text{CMB}\rangle$, and source galaxy weak lensing shear with CMB lensing convergence $\langle^{γκ}$$_{\text{CMB}}\rangle$. These two cross-correlation probes provide an independent cross-check of other large-scale structure constraints and are insensitive to galaxy-only or CMB-only systematic effects. In addition, when combined with other large-scale structure probes, the cross-correlations can break degeneracies in cosmological and nuisance parameters, improving both the precision and robustness of the analysis. In this work, we study how the constraining power of $\langle δ_g$$^κ\text{CMB}\rangle + \langle^{γκ}$$_{\text{CMB}}\rangle$ changes from Stage-III (ongoing) to Stage-IV (future) surveys. Given the flexibility in selecting the lens galaxy sample, we also explore systematically the impact on cosmological constraints when we vary the redshift range and magnitude limit of the lens galaxies using mock galaxy catalogs. We find that in our setup, the contribution to cosmological constraints from $\langle δ_g$$^κ\text{CMB}\rangle$ and $\langle^{γκ}$$_{\text{CMB}}\rangle$ are comparable in the Stage-III data sets; but in Stage-IV surveys, the noise in$\langle δ_g$$^κ\text{CMB}\rangle$ becomes subdominant to cosmic variance, preventing $\langle δ_g$$^κ\text{CMB}\rangle$ to further improve the constraints. This implies that to maximize the cosmological constraints from future $\langle δ_g$$^κ\text{CMB}\rangle$ + $\langle^{γκ}$$_{\text{CMB}}\rangle$ analyses, we should focus more on the requirements on$^{γκ}$$_{\text{CMB}}\rangle$ instead of$\langle δ_g$$^κ\text{CMB}\rangle$. Furthermore, the selection of the lens sample should be optimized in terms of our ability to characterize its redshift or galaxy bias instead of its number density.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Calorimeter Pileup Deconvolution for Online Trigger Primitives

In high energy physics experiment, as the luminosity increases, pile-up issues on detectors such as calorimeters become non-negligible. Deconvolution approaches with mathematic pre-assumptions such as Sparse Representation are developed for data analysis stage. For online computation tasks such as for trigger primitive creation, signal availability is significantly different as in offline data analysis stage, and therefore, different (yet simpler) algorithms should be explored. In this document, several approaches of deconvolution suitable for FPGA implementation are discussed.

Wu, Jin-yuan [Fermilab] (ORCID:0000000344329521)↗
DOE OSTI2024

High-accuracy emulators for observables in ΛCDM, N eff, Σ m ν, and w cosmologies

ABSTRACT We use the emulation framework CosmoPower to construct and publicly release neural network emulators of cosmological observables, including the cosmic microwave background (CMB) temperature and polarization power spectra, matter power spectrum, distance-redshift relation, baryon acoustic oscillation (BAO) and redshift-space distortion (RSD) observables, and derived parameters. We train our emulators on Einstein–Boltzmann calculations obtained with high-precision numerical convergence settings, for a wide range of cosmological models including ΛCDM, wCDM, ΛCDM + Neff, and ΛCDM + Σmν. Our CMB emulators are accurate to better than 0.5 per cent out to ℓ = 104, which is sufficient for Stage-IV data analysis, and our P(k) emulators reach the same accuracy level out to $k=50 \, \, \mathrm{Mpc}^{-1}$, which is sufficient for Stage-III data analysis. We release the emulators via an online repository (CosmoPower Organisation), which will be continually updated with additional extended cosmological models. Our emulators accelerate cosmological data analysis by orders of magnitude, enabling cosmological parameter extraction analyses, using current survey data, to be performed on a laptop. We validate our emulators by comparing them to class and camb and by reproducing cosmological parameter constraints derived from Planck TT, TE, EE, and CMB lensing data, as well as from the Atacama Cosmology Telescope Data Release 4 CMB data, Dark Energy Survey Year-1 galaxy lensing and clustering data, and Baryon Oscillation Spectroscopic Survey Data Release 12 BAO and RSD data.

Astronomy & Astrophysics↗
DOE OSTI2024

MultiPEM Toolbox: User Manual [Rev. 2]

This document explains use of the Multi-Phenomenology Explosion Monitoring (Multi PEM) Toolbox, a collection of R scripts for estimating the unknown device parameters of a new event with uncertainty quantification. The methodology and application used for illustration in this user manual are fully documented in a Los Alamos National Laboratory technical report hereafter designated “WPA” for reference. Additional details on the application are found in a recent journal article. Two assessment types are available: rapid and complete. Rapid assessments are conducted in two stages, as described in Section 2. In the first stage, calibration data are used to estimate forward and error model parameters (WPA, §5.1) and (if relevant) errors-in-variables yield values for calibration sources (WPA, §3, Equation (3)). In the second stage, new event data are used to estimate the unknown new event device parameters (WPA, §5.2) with uncertainty quantification. Two options for treating the inferred first stage parameters in second stage Bayesian analysis are available: fixing them at their maximum likelihood estimate (default), or multiple imputation. Multiple imputation involves utilizing several posterior samples (imputations) of the first stage parameters as fixed values in the second stage posterior sampling of the new event device parameters. Second stage sampling is conducted across imputations in parallel to improve computational efficiency. This method produces improved uncertainty quantification of the new event device parameters compared with the default treatment of the first stage parameters, at the expense of additional computation. Complete assessments are conducted in a single stage, as described in Section 3. Calibration and (if relevant) new event data are used simultaneously to estimate all forward model, error model, and (if relevant) new event device parameters with uncertainty quantification on the latter. As the name suggests, rapid assessments generally run substantially faster than complete assessments (even with multiple imputation), because the results of first stage analysis can be stored and incorporated into estimating a relatively low-dimensional space of new event device parameters whenever relevant new event data becomes available. On the other hand, complete assessments must be run on the full set of model and device parameters with calibration and new event data every time the latter becomes available.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Facilitating Staging-based Unstructured Mesh Processing to Support Hybrid In-Situ Workflows

In-situ and in-transit processing alleviate the gap between the computing and I/O capabilities by scheduling data analytics close to the data source. Hybrid in-situ processing splits data analytics into two stages: the data processing that runs in-situ aims to extract regions of interest, which are then transferred to staging services for further in-transit analytics. To facilitate this type of hybrid in-situ processing, the data staging service needs to support complex intermediate data representations generated/consumed by the in-situ tasks. Unstructured (or irregular) mesh is one such derived data representation that is typically used and bridges simulation data and analytics. However, how staging services efficiently support unstructured mesh transfer and processing remains to be explored. This paper investigates design options for transferring and processing unstructured mesh data using staging services. Using polygonal mesh data as an example, we show that hybrid in-situ workflows with staging-based unstructured mesh processing can effectively support hybrid in-situ workflows, and can significantly decrease data movement overheads.

data-driven↗
DOE OSTI2025

Performance of Convection-Permitting and Convection-Parameterized Models in Reproducing the Extreme Precipitation Intensity Relationship with Surface Conditions

Here, this study investigates the warm-season extreme precipitation–temperature scaling relationship in CONUS404, a convection-permitting (4 km) Weather Research and Forecasting (WRF) Model simulation over the conterminous United States for the past four decades, and compares it with the WRF-Thermodynamic Global Warming (WRF-TGW) historical simulation at a coarser resolution (12 km) using parameterized convection. We also analyze the NCEP stage IV and NASA Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement (IMERG) datasets as observational benchmarks. We examine how extreme precipitation intensity (EPI) varies with temperature and saturation deficit over representative regions based on hourly data. The stage IV and IMERG data show a similar pattern of EPI variation with temperature and saturation deficit, except that the EPI peak is lower in IMERG than in stage IV. Under dry and hot conditions, EPI decreases too rapidly with elevated saturation deficit in both CONUS404 and WRF-TGW compared to observations, but the performance of CONUS404 is superior to WRF-TGW. When the near-surface atmosphere is saturated or close to saturated, both CONUS404 and WRF-TGW produce higher peak values of EPI relative to the observational references; IMERG exhibits scaling rates close to the Clausius–Clapeyron (C–C) relationship, while CONUS404, WRF-TGW, and stage IV all demonstrate super-C–C scaling behaviors. Despite marked warming over the past four decades, in both CONUS404 and WRF-TGW, the scaling relationship between EPI and temperature in a saturated atmosphere remains stable and robust. This indicates a strong potential for the EPI–temperature scaling rate under saturation to be used as an emergent constraint in reducing uncertainties of future extreme precipitation projection.

Atmosphere↗
DOE OSTI2024

Enhancing Discoverability and Management of Atmospheric Data at Scale: Solutions from the ARM Data Center

The Atmospheric Radiation Measurement (ARM) is a multi-laboratory and multi-institutional U.S. Department of Energy (DOE) Office of Science National User Facility. The ARM Data Center (ADC), located at Oak Ridge National Laboratory, collects, archives, and shares vast atmospheric data crucial for climate research. The ADC manages over 7 PB of data from 460 instruments worldwide, processing it into more than 11,000 diverse data products using the Network Common Data Form (NetCDF) for machine-independent accessibility. The primary challenge addressed in this paper is the efficient management and distribution of vast and diverse datasets essential for the climate research community, enhancing accessibility through advanced tools like Data Discovery. The ADC has developed advanced infrastructure and software architecture to handle the continuous influx of heterogeneous data to enhance data discoverability, resulting in increased scientific collaboration. In 2023, users from over 34 countries downloaded and utilized ARM data, resulting in 1,455 publications. The ADC’s efforts have significantly improved the discoverability and usability of atmospheric data, fostering extensive scientific research and collaboration. This paper details the solutions implemented by the ADC team for efficient data discovery and distribution, and it demonstrates ARM’s capability of staging processed data for scientific analysis.

Shah, Chirag [ORNL] (ORCID:0000000203145737)↗
DOE OSTI2026

Numerical Modeling of a Two-Stage Ocean Current Turbine

The Equinox Ocean Turbines (EQOT) current energy converter has a unique design with power generation in two small-diameter turbines attached to the tips of a large-diameter passive rotor. This configuration offers some key advantages for capturing ocean currents. With no centrally placed generator, almost no reaction torque is required at the nacelle of the main large-diameter rotor, and the small-diameter tip turbine generators operate at a higher speed and lower torque. The physics that determine the performance and loads on the turbine are also unique. The interactions of the flow field between the two stages and the general architecture of the system cannot be captured with traditional mid-fidelity modeling tools. For design iterations and large sets of load cases, it is important to have mid-fidelity models that can capture the important phenomenon with enough accuracy to identify global trends. This work uses a limited set of high-fidelity computational fluid dynamics (CFD) simulations to help inform the selection of and construction of a custom mid-fidelity model. Mid-fidelity modeling approaches were verified by comparing key turbine performance quantities to those found with the CFD model. Hydrodynamic interactions of the two-stage rotor were identified through high-fidelity CFD modeling. This highlighted the impact of the main rotor tip vortex and wake on the secondary rotor apparent inflow. This results in a relative flow rotation and sharp deficit, that change the optimal secondary rotor rotation speed and adds unsteadiness to the blade loading respectively. Multiple mid-fidelity approaches were evaluated for their ability to capture these effects. A simple approximation of the combined-stage performance based on single-stage BEM provides a reasonable rough prediction, especially near the peak TSR values, with some larger discrepancy at higher TSRs. Predicting the combined-stage performance based on single-stage CFD data improves this prediction across the TSR range. Although the combined-stage modeling in OLAF was not successful in this stage of the project, it showed promise as a mid-fidelity method, assuming the parameters can be tuned to account for the significant differences in time and length scales between the main and secondary rotors. This may be addressed through code changes in future work. A significant finding from the OLAF work was the agreement between the vortex core radius values found independently via a parameter space search and via CFD. The technique of using single-stage secondary rotor BEM, with a custom inflow taken from single-stage main rotor CFD or OLAF, provides an efficient method to capture one-way coupled flow interactions. This method provided generally good predictions of the impact of the flow rotation on the secondary rotor but struggled to accurately predict the peaks of the unsteady load progression. Future work could include some superposition of a tuned main rotor trailing edge viscous wake into the custom inflow to better predict this interaction.

16 TIDAL AND WAVE POWER↗
DOE OSTI2020

Determining Compression Characteristics of Honeycomb Material - 19674

The objective is to determine compression test characteristics of stainless steel honeycomb material to be able to represent honeycomb structures accurately in analytical models used to simulate hypothetical accident scenarios of shipping packages. Honeycomb is a material used primary in the aerospace industry due to its high strength-to-weight ratio. Because it doesn't have a shelf life and can withstand high heats, it is an excellent candidate for a structural material in package designs. Honeycomb has orthotropic material properties. The honeycomb currently being evaluated is made of metal ribbons spot welded together to form hexagon pattern between metal plates. The hexagon ribbons are brazed to the metal plates. The complexity of the honeycomb significantly lengthens the simulation time, which is further complicated when the brazing, welding, and imperfections of the material is considered. This means to affectively represent the honeycomb in simulation programs, such as Abaqus, the material needs to be approximated as a uniform orthotropic material. This requires material properties in each of the three directions, T, W, and L. The T direction is defined as the 'strong' direction, perpendicular to the honeycomb sheet. The L direction is parallel the ribbon and the W direction is perpendicular to the ribbon. Honeycomb has 3 different compression stages. Stage 1 is the initial compression where the honeycomb maintains its structural integrity and does not permanently deform from loads from normal operations. Stage 2 is when the initial buckling causes irreversible damage to the honeycomb. Stage 2 is the one we are most interested in because it absorbs the most energy from a hypothetical accident scenario. During Stage 2, the honeycomb fails layer-by-layer, indicating that the more layers, the longer material crushing is sustained, and thus the more energy that is absorbed. Stage 3 is final compression, similar to compressing solid metal. During stage 3 the stress increases with a diminishing rate of elongation and the honeycomb is completely failed where the plates between the honeycomb core sandwich the crushed honeycomb ribbon. The stress-strain graph bellow shows compression test of 3 different samples in the T direction. The red lines divide the three stages. The far left is Stage 1, the middle stage 2, and the right stage 3. The data collected is displayed in the table below. The expressions in the table represent the regression formula that represents each stage of the compression on a stress-strain graph. Stage 1 is assumed to intersect with the origin; data for stage 3 for the crush in W and L directions where unable to be gathered due to the nature of failure for those directions. Stage 1 and 2 are linear regressions while stage 3 is a degree 2, to best match the curve. Linear regression is used on stage 2 because ultimately that represents the energy absorbed. With more data, a degree n x 2 regression would be a more appropriate regression for stage 2, where n is the number of layers. The table below is the approximation of each scenario and stage. Each stage starts and ends at the intersection of the next stage. There where two main objectives for these tests: firstly to understand how honeycomb performs under extreme compression, and secondly to be able to numerically represent the honeycomb structure. Both of these where completed to varying degrees. It is important understood how the honeycomb fails. If it is crushed in the T direction it retains its integrity even after being crushed, however when crushed in the L and W crush direction, if it fails, it disintegrates, and loses all integrity. Also, the more layers the more time the material spends in stage 2. Failure is started by buckling, thus if there is any imperfection, the stress will not spike but transition straight into stage 2. The data collected and aggregated can be used for initial simulation of honeycomb used in packages. The initial testing has set the ground work for more data to be collected in order to verify results and to allow more confidence in the simulation results. This is only the initial data collected. The next steps is to continue to collect more data to verify results and to test more variants of honeycomb, with different brazing, layers, and shape.

36 MATERIALS SCIENCE↗
DOE OSTI2024

HERO WEC 2024 Hydraulic Configuration Deployment Data

The following submission includes raw and processed data from the in water deployment of NREL's Hydraulic and Electric Reverse Osmosis Wave Energy Converter (HERO WEC), in the form of parquet files, TDMS files, CSV files, bag files and MATLAB workspaces. This dataset was collected in March 2024 at the Jennette's pier test site in North Carolina. This submission includes the following: - Data description document (HERO WEC FY24 Hydraulic Deployment Data Descriptions.doc) - This document includes detailed descriptions of the type of data and how it was processed and/or calculated. - Processed MATLAB workspace - The processed data is provided in the form of a single MATLAB workspace containing data from the full deployment. This workspace contains data from all sensors down sampled to 10 Hz along with all array Value Added Products (VAPs). - MATLAB visualization scripts - The MATLAB workspaces can be visualized using the file "HERO_WEC_2024_Hydraulic_Config_Data_Viewer.m/mlx". The user simply needs to download the processed MATLAB workspaces, specify the desired start and end times and run this file. Both the .m and .mlx file format has been provided depending on the user's preference. - Summary Data - The fully processed data was used to create a summary data set with averages and important calculations performed on 30-minute intervals to align with the intervals of wave resource data reported from nearby CDIP ocean observing buoys located 20km East of Jennette's pier and 40km Northeast of Jennette's pier. The wave resource data provided in this data set is to be used for reference only due the difference in water depth and proximity to shore between the Jennette's pier test site and the locations of the ocean observing buoys. This data is provided in the Summary Data zip folder, which includes this data set in the form of a MATLAB workspace, parquet file, and excel spreadsheet. - Processed Parquet File - The processed data is provided in the form of a single parquet file containing data from all HERO WEC sensors collected during the full deployment. Data in these files has been down sampled to 10 Hz and all array VAPs are included. - Interim Filtered Data - Raw data from each sensor group partitioned into 30-minute parquet files. These files are outputs from an intermediate stage of data processing and contain the raw data with no Quality Control (QC) or calculations performed in a format that is easier to use than the raw data. - Raw Data - Raw, unprocessed data from this deployment can be found in the Raw Data zip folder. This data is provided in the form of TDMS, CSV, and bag files in the original format output by the MODAQ system. - Python Data Processing Script - This links to an NREL public github repository containing the python script used to go from raw data to fully processed parquet files. Additional documentation on how to use this script is included in the github repository. This data set has been developed by the National Renewable Energy Laboratory, operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. Funding provided by the U.S. Department of Energy Office of Energy Efficiency and Renewable Energy Water Power Technologies Office.

16 TIDAL AND WAVE POWER↗