Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “analysis workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Buildings Sector Scenarios: Demand-side data to support energy system planning in the United States

The US energy system is in a period of high uncertainty about load growth, its implications for the energy generation mix, and downstream impacts on customer energy costs. In this context, there is a need for comprehensive, credible, and readily-customized projections of energy demand to ensure that planning decisions account for end-use management opportunities to improve system reliability and affordability. Here we introduce the Buildings Sector Scenarios (BSS) dataset, which includes a benchmark suite of such projections for the buildings sector — a key source of energy consumption, peak electricity demand, and consumer energy expenditures. The dataset contains projections through 2050 covering the contiguous United States (CONUS) resolved down to the county, hourly level by sector and end use for electricity demand and to the state, annual level by sector and end use for non-electric fuels. We summarize the BSS analysis workflow and the tools and datasets that support it, document key BSS scenario inputs and modeling assumptions, and outline BSS scenario outputs. We assess the technical quality of the dataset against historical surveys and projected estimates of buildings sector demand. Finally, we provide guidance on how stakeholders can access, use, and reproduce the dataset, and/or create new scenarios to explore their own analysis questions.

Langevin, Jared↗

Accelerating Multivariate Functional Approximation Computation with Domain Decomposition Techniques⋆

Modeling large datasets through Multivariate Functional Approximations (MFA) provide an elegant way to handle many visualization and scientific analysis workflows. The process necessitates scalable data partitioning methods to compute MFA representations efficiently without compromising the accuracy or continuity of the reconstructed solution. We propose a domain -decomposed method for computing the MFA with B -spline bases, which reduces the total work per task and uses a restricted Additive Schwarz (RAS) method to converge the control point data degrees -of -freedom along subdomain boundaries. We provide an in-depth analysis of the parallel approach with domain decomposition solvers, aiming to minimize local subdomain error residuals and recover high -order continuity at subdomain interfaces with appropriate choices of knot overlaps. The communication cost, determined by the overlap regions in the RAS implementation, is optimized to recover the numerical error profile of the single subdomain case. Our proposed method stands in contrast to previous methods, which typically only recover either C 0 or at best C 1 continuity for arbitrary B -spline degree expansions, or those that require post -processing to blend discontinuities in the reconstructed data. We demonstrate the effectiveness of our approach using analytical and real -world datasets in 1D, 2D, and 3D through both strong and weak scaling studies. The performance results indicate that the overall cost of computing the approximation is directly proportional to the underlying nearest -neighbor communication implementation, and is only weakly dependent on the overlap region size that determines the size of the messages. This finding underscores the efficiency and scalability of our proposed method, making it a promising solution for handling large datasets in scientific workflows.

additive Schwarz solvers↗

Energy transport analysis of NSTX plasmas with the TGLF turbulent and NEO neoclassical transport models

This work presents a study of plasma transport at low aspect ratio on the National Spherical Torus Experiment tokamak, where the turbulent and neoclassical energy fluxes calculated by the quasilinear Trapped Gyro Landau Fluid (TGLF) model and the multi species drift-kinetic Neoclassical solver (NEO) are validated against experimental data. The turbulent energy transport of two plasma discharges, one in the L-mode confinement regime and another in the H-mode regime, is dominated by electrostatic drift-wave instabilities, while the ion heat transport has a significant neoclassical contribution. The data analysis workflow is described in detail to understand how the variations of mapping and fitting of experimental data affect the power balance solution and subsequent flux-matching plasma profile predictions with the TGYRO solver. On average, the predicted plasma profiles are consistent with experimental data. However, the solutions are sensitive to various input parameters, including boundary conditions, and the electron-ion coupling. Linear gyrokinetic stability analysis demonstrates close agreement of the real frequencies of unstable modes between TGLF and CGYRO gyrokinetic simulations, but higher growth rates are predicted by TGLF, especially for the H-mode case. Estimates of the low-k modes' contributions to the total flux are consistent with linear stability analysis and the E × B suppression of turbulence in TGLF simulations with the SAT1 saturation model, while the SAT2 saturation model over-predicts the low-k modes' contribution in the H-mode case. Moreover, the results with SAT1 model are consistent with power balance analysis, which indicates only neoclassical ion energy fluxes inside ρ < 0.4 in the L-mode case and $\rho \unicode{x2A7D} 0.7$ in the H-mode case. The presence of multi-scale turbulence and ion-scale driven zonal flow mixing effects are also observed in TGLF scans of the electron turbulent heat flux over a range of temperature gradients and the electron-ion temperature ratio, which could explain the strong model sensitivity to variations of input parameters.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A gLite FTS based solution for managing user output in CMS

The CMS distributed data analysis workflow assumes that jobs run in a different location from where their results are finally stored. Typically the user output must be transferred across the network from one site to another, possibly on a different continent or over links not necessarily validated for high bandwidth/high reliability transfer. This step is named stage-out and in CMS was originally implemented as a synchronous step of the analysis job execution. However, our experience showed the weakness of this approach both in terms of low total job execution efficiency and failure rates, wasting precious CPU resources. The nature of analysis data makes it inappropriate to use PhEDEx, the core data placement system for CMS. As part of the new generation of CMS Workload Management tools, the Asynchronous Stage-Out system (AsyncStageOut) has been developed to enable third party copy of the user output. The AsyncStageOut component manages glite FTS transfers of data from the temporary store at the site where the job ran to the final location of the data on behalf of that data owner. The tool uses python daemons, built using the WMCore framework, and CouchDB, to manage the queue of work and FTS transfers. CouchDB also provides the platform for a dedicated operations monitoring system. In this paper, we present the motivations of the asynchronous stage-out system. We give an insight into the design and the implementation of key features, describing how it is coupled with the CMS workload management system. Finally, we show the results and the commissioning experience.

Cinquilli, M.↗

Current parallel I/O limitations to scalable data analysis.

This report describes the limitations to parallel scalability which we have encountered when applying our otherwise optimally scalable parallel statistical analysis tool kit to large data sets distributed across the parallel file system of the current premier DOE computational facility. This report describes our study to evaluate the effect of parallel I/O on the overall scalability of a parallel data analysis pipeline using our scalable parallel statistics tool kit [PTBM11]. In this goal, we tested it using the Jaguar-pf DOE/ORNL peta-scale platform on a large combustion simulation data under a variety of process counts and domain decompositions scenarios. In this report we have recalled the foundations of the parallel statistical analysis tool kit which we have designed and implemented, with the specific double intent of reproducing typical data analysis workflows, and achieving optimal design for scalable parallel implementations. We have briefly reviewed those earlier results and publications which allow us to conclude that we have achieved both goals. However, in this report we have further established that, when used in conjuction with a state-of-the-art parallel I/O system, as can be found on the premier DOE peta-scale platform, the scaling properties of the overall analysis pipeline comprising parallel data access routines degrade rapidly. This finding is problematic and must be addressed if peta-scale data analysis is to be made scalable, or even possible. In order to attempt to address these parallel I/O limitations, we will investigate the use the Adaptable IO System (ADIOS) [LZL+10] to improve I/O performance, while maintaining flexibility for a variety of IO options, such MPI IO, POSIX IO. This system is developed at ORNL and other collaborating institutions, and is being tested extensively on Jaguar-pf. Simulation code being developed on these systems will also use ADIOS to output the data thereby making it easier for other systems, such as ours, to process that data.

99 GENERAL AND MISCELLANEOUS↗

Development and implementation of high-throughput proteomic and metabolomics assays by using advanced chromatographic and mass spectrometric systems (CRADA Final Report)

The mission of this CRADA with Agilent was to couple powerful MS platforms (QQQ, IM-QTOFMS) with Agilent’s novel Ultra-High-Performance Liquid Chromatography (UHPLC) fast metabolomic workflows and perform ABF Machine Learning (ML) to generated datasets. Agilent transferred UHPLC methods to PNNL and LBNL and methods were implemented and demonstrated in both labs, achieving total acquisition times of < 10 min. Metabolites analyzed using Agilent’s shared methods included metabolites from central carbon metabolism, common across hosts, and metabolites unique to engineered strains. Standards were acquired in an UHPLC-Drift Tube Ion Mobility Mass Spectrometer (DTIMS) system for the first time within the context of ABF and methods were optimized based on Agilent’s protocols. Samples from ABF hosts Pseudomonas putida, Aspergillus pseudoterreus, Aspergillus niger and Rhodosporidium toruloides were analyzed using the UHPLC-DTIMS platform for a total of 276 runs. A data analysis workflow compatible with the Experimental Data Depot (EDD) and completely shareable was developed for the acquired UHPLC-DTIMS data. Samples were analyzed using a Data Independent Acquisition Approach (DIA), which for most of the standards provided more transitions therefore increasing detection confidence. Using the data acquired by PNNL, LBNL, and Agilent’s specifications from previous ML projects, SNL applied an ensemble ML strategy to pick the best performing model for automated LC-method selection. Finally, with the contribution of the participant labs and Agilent, SNL developed an Automated Method Selection (AMS) software tool to predict the best liquid chromatography method for analysis of any new molecules of interest. Samples with novel pathways and new metabolite targets of interest are generated at a high pace in the ABF. Overall, the project advanced rapid metabolomics by combining liquid chromatography, ion mobility spectrometry, and data-independent mass spectrometry with machine learning. This multidimensional approach uses retention time, collision cross-section, precursor mass, and fragment-ion information to distinguish chemically similar metabolites that can be difficult to resolve using conventional liquid- or gas-chromatography methods. The resulting workflow also provided automated metabolite-identification error estimates, addressing a recognized need for statistical confidence measures in metabolomics.

Petzold, Christopher [Lawrence Berkeley National L↗

SPAROW: Stochastic Programming and Related Optimization Workflows

SAND2026-16703O SPAROW: Stochastic Programming and Related Optimization Workflows is a Python library tool that facilitates the development and solution of stochastic programming problems. It provides a user-friendly class structure for defining stochastic programs through scenario-based representations of uncertainties. SPAROW incorporates multiple optimization strategies, including integer programming with all scenarios, progressive hedging, Benders decomposition, and Snoglode, a novel technique developed by Carnegie Mellon University. It also features interfaces to external solvers and functions that are commonly used in analysis workflows, making it applicable to a wide range of scientific and engineering design challenges, particularly in power grid planning. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Hart, William [Sandia National Lab. (SNL-NM), Albu↗

MILK : a Python scripting interface to MAUD for automation of Rietveld analysis

Modern diffraction experiments ( e.g. in situ parametric studies) present scientists with many diffraction patterns to analyze. Interactive analyses via graphical user interfaces tend to slow down obtaining quantitative results such as lattice parameters and phase fractions. Furthermore, Rietveld refinement strategies ( i.e. the parameter turn-on-off sequences) tend to be instrument specific or even specific to a given dataset, such that selection of strategies can become a bottleneck for efficient data analysis. Managing multi-histogram datasets such as from multi-bank neutron diffractometers or caked 2D synchrotron data presents additional challenges due to the large number of histogram-specific parameters. To overcome these challenges in the Rietveld software Material Analysis Using Diffraction ( MAUD ), the MAUD Interface Language Kit ( MILK ) is developed along with an updated text batch interface for MAUD . The open-source software MILK is computer-platform independent and is packaged as a Python library that interfaces with MAUD . Using MILK , model selection ( e.g. various texture or peak-broadening models), Rietveld parameter manipulation and distributed parallel batch computing can be performed through a high-level Python interface. A high-level interface enables analysis workflows to be easily programmed, shared and applied to large datasets, and external tools to be integrated with MAUD . Through modification to the MAUD batch interface, plot and data exports have been improved. The resulting hierarchical folders from Rietveld refinements with MILK are compatible with Cinema: Debye–Scherrer , a tool for visualizing and inspecting the results of multi-parameter analyses of large quantities of diffraction data. In this manuscript, the combined Python scripting and visualization capability of MILK is demonstrated with a quantitative texture and phase analysis of data collected at the HIPPO neutron diffractometer.

97 MATHEMATICS AND COMPUTING↗

Performance of Julia for High Energy Physics Analyses

We argue that the Julia programming language is a compelling alternative to currently more common implementations in Python and C++ for common data analysis workflows in high energy physics. We compare the speed of implementations of different workflows in Julia with those in Python and C++. Furthermore, our studies show that the Julia implementations are competitive for tasks that are dominated by computational load rather than data access. For work that is dominated by data access, we demonstrate an application with concurrent file reading and parallel data processing.

97 MATHEMATICS AND COMPUTING↗

Agilent AgileBioFoundry CRADA (Final Report)

The mission of this CRADA with Agilent was to couple powerful MS platforms (QQQ, IM-QTOF-MS) with Agilent’s novel Ultra-High-Performance Liquid Chromatography (UHPLC) fast metabolomic workflows and perform ABF Machine Learning (ML) to generated datasets. Agilent transferred UHPLC methods to PNNL and LBNL and methods were implemented and demonstrated in both labs, achieving total acquisition times of < 10 min. Metabolites analyzed using Agilent’s shared methods included metabolites from central carbon metabolism, common across hosts, and metabolites unique to engineered strains. Standards were acquired in an UHPLC-Drift Tube Ion Mobility Mass Spectrometer (DTIMS) system for the first time within the context of ABF and methods were optimized based on Agilent’s protocols. Samples from ABF hosts Pseudomonas putida, Aspergillus pseudoterreus, Aspergillus niger and Rhodosporidium toruloides were analyzed using the UHPLC-DTIMS platform for a total of 276 runs. A data analysis workflow compatible with the Experimental Data Depot (EDD) and completely shareable was developed for the acquired UHPLC-DTIMS data. Samples were analyzed using a Data Independent Acquisition Approach (DIA), which for most of the standards provided more transitions therefore increasing detection confidence. Using the data acquired by PNNL, LBNL, and Agilent’s specifications from previous ML projects, SNL applied an ensemble ML strategy to pick the best performing model for automated LC-method selection. Finally, with the contribution of the participant labs and Agilent, SNL developed an Automated Method Selection (AMS) software tool to predict the best liquid chromatography method for analysis of any new molecules of interest. Samples with novel pathways and new metabolite targets of interest are generated at a high pace in the ABF. Therefore, our accomplishment in this CRADA improved the efficiency and accuracy of strain testing by developing and implementing fast analytical methods, robust processing tools, and software for predicting the best methods for UHPLC analysis.

97 MATHEMATICS AND COMPUTING↗

Design-Space Data: Informing Common Design Decisions with Pre-Simulated Data

Design Space Exploration (DSE) analysis techniques represent a data-centric approach to integrating performance analysis in early design phases when there is the greatest potential to cheaply improve the energy efficiency of a building. We focus on a novel extension of DSE called Universal Design Space Exploration (UDSE), which leverages massive databases of pre-simulated analysis that represent all possible outcomes of common analysis workflows. These databases, called Design Spaces, become “universal” when a single pre-simulated design space can be re-applied to future unknown projects. Unlike current simulation methods, which require a design to exist before it can be analyzed and often take minutes or hours to simulate, UDSE leverages pre-simulation to deliver rapid and relevant insight as new designs are conceptualized. The data underpinning UDSE enables advanced statistical and Artificial Intelligence methods, allowing UDSE to deliver a greater understanding of the larger problem being explored, rather than simply delivering analysis of several pre-conceived design options. We believe that UDSE can provide instantaneous, relevant analysis for all building design projects at negligible cost. This paper has two main goals, to develop a relevant Universal Design Space that showcases the potential of UDSE and to release this data freely to industry and academia; thereby lowering the barrier to entry to digital literacy in statistics, ML and AI within the architecture, engineering, construction (AEC) industry.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Learning continuous scattering length density profiles from neutron reflectivities using convolutional neural networks

Interpreting neutron reflectivity (NR) data using ad hoc multi-layer models and physics-based models provides information about spatially resolved neutron scattering length density (NSLD) profiles. Recent improvements in data acquisition systems have allowed acquiring thousands of NR curves in a couple of hours, which has led to a need for automated data analysis tools to interpret NR measurements in real-time. Here, we present a machine learning analysis workflow that uses a series of models, based on a convolutional neural network (CNN), to learn the relation between the NSLDs and the NRs, and subsequently produce continuous NSLD profiles directly from NRs. The usefulness of our CNN-based models is demonstrated by constructing NSLDs from NRs of several films containing homopolymer polyzwitterions and diblock copolymers mixed with different types of salts. Comparisons of the NSLDs with those constructed using ad hoc multi-layer models reveal a very good agreement, suggesting the potential of CNN-based models for real-time automated data analysis of NRs.

36 MATERIALS SCIENCE↗

A Scoping Review of Mixed Initiative Visual Analytics in the Automation Renaissance

Artificial agents are increasingly integrated into data analysis workflows, carrying out tasks that were primarily done by humans. Our research explores how the introduction of automation recalibrates the dynamic between humans and automating technology. To explore this question, we conducted a scoping review encompassing twenty years of mixed-initiative visual analytic systems. To describe and contrast the relationship between humans and automation, we developed an integrated taxonomy to delineate the objectives of these mixed-initiative visual analytics tools, how much automation they support, and the assumed roles of humans. Here, we describe our qualitative approach of integrating existing theoretical frameworks with new codes we developed. Our analysis shows that the visualization research literature lacks consensus on the definition of mixed-initiative systems and explores a limited potential of the collaborative interaction landscape between people and automation. Our research provides a scaffold to advance the discussion of human-AI collaboration during visual data analysis. Our integrated taxonomy is available in the form of a web application on https://smonadjemi.github.io/miva.

Monadjemi, Shayan [ORNL] (ORCID:0000000293855969)↗

rabpro: global watershed boundaries, river elevation profiles, and catchment statistics

River and Basin Profiler (rabpro) is a Python package to delineate watersheds, extract river flowlines and elevation profiles, and compute watershed statistics for any location on the Earth’s surface. As fundamental hydrologically-relevant units of surface area, watersheds are areas of land that drain via aboveground pathways to the same location, or outlet. Delineations of watershed boundaries are typically performed on digital elevation models (DEMs) that represent surface elevations as gridded rasters. Depending on the resolution of the DEM and the size of the watershed, delineation may be very computationally expensive. With this in mind, we designed rabpro to provide user-friendly workflows to manage the complexity and computational expense of watershed calculations given an arbitrary coordinate pair. In addition to basic watershed delineation, rabpro will extract the elevation profile for a watershed’s mainchannel flowline. This enables the computation of river slope, which is a critical parameter in many hydrologic and geomorphologic models. Finally, rabpro provides a user-friendly wrapper around Google Earth Engine’s (GEE) Python API to enable cloud-computing of zonal watershed statistics and/or time-varying forcing data from hundreds of available datasets. Altogether, rabpro provides the ability to automate or semi-automate complex watershed analysis workflows across broad spatial extents.

54 ENVIRONMENTAL SCIENCES↗

OpenOA: An Open-Source Codebase For Operational Analysis of Wind Farms

OpenOA is an open source framework for operational data analysis of wind energy plants, implemented in the Python programming language. OpenOA provides a common data model, high level analysis workflows, and low-level convenience functions that engineers, analysts, and researchers in the wind energy industry can use to facilitate analytics workflows on operational data sets. OpenOA contains documentation, worked out examples in Jupyter notebooks, and a corresponding example dataset from the Engie Renewable’s La Haute Borne Dataset.

17 WIND ENERGY↗

Evaluation of Portable Programming Models to Accelerate LArTPC Detector Simulations

The Liquid Argon Time Projection Chamber (LArTPC) technology is widely used in high energy physics experiments, including the upcoming Deep Underground Neutrino Experiment (DUNE). Accurately simulating LArTPC detector responses is essential for analysis algorithm development and physics model interpretations. Accurate LArTPC detector response simulations are computationally demanding, and can become a bottleneck in the analysis workflow. Compute devices such as General-Purpose Graphics Processing Units (GPGPUs) have the potential to substantially accelerate simulations compared to traditional CPU-only processing. The software development that requires often carries the cost of specialized code refactorization and porting to match the target hardware architecture. With the rapid evolution and increased diversity of the computer architecture landscape, it is highly desirable to have a portable solution that also maintains reasonable performance. We report our ongoing effort in evaluating Kokkos as a basis for this portable programming model using LArTPC simulations in the context of the Wire-Cell Toolkit, a C++ library for LArTPC simulations, data analysis, reconstruction and visualization.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Evaluation of Portable Programming Models to Accelerate LArTPC Detector Simulations

The Liquid Argon Time Projection Chamber (LArTPC) technology is widely used in high energy physics experiments, including the upcoming Deep Underground Neutrino Experiment (DUNE). Accurately simulating LArTPC detector responses is essential for analysis algorithm development and physics model interpretations. Accurate LArTPC detector response simulations are computationally demanding, and can become a bottleneck in the analysis workflow. Compute devices such as General-Purpose Graphics Processing Units (GPGPUs) have the potential to substantially accelerate simulations compared to traditional CPU-only processing. The software development for these compute accelerators often carries the cost of specialized code refactorization and porting to match the target hardware architecture. With the rapid evolution and increased diversity of the computer architecture landscape, it is highly desirable to have a portable solution that also maintains reasonable performance. We report our ongoing effort in evaluating Kokkos as a basis for this portable programming model using LArTPC simulations in the context of the Wire-Cell Toolkit, a C++ library for LArTPC simulations, data analysis, reconstruction and visualization.

47 OTHER INSTRUMENTATION↗

Sensor cost-effectiveness analysis for data-driven fault detection and diagnostics in commercial buildings

Data-driven building fault detection and diagnostics (FDD) is heavily dependent on sensors. However, common sensors from Building Automation Systems are not optimized to maximize accuracy in FDD. Installing additional sensors that provide more detailed building system information is key to maximizing the performance of FDD solutions. Here in this paper, we present a sensor cost analysis workflow to quantify the economic implications of installing new sensors for FDD using the concept of sensor threshold marginal cost (STMC). STMC does not represent actual sensor cost. Rather, it represents a target cost based on the economic benefit that would be realized through improved FDD performance and one or more specified economic criteria. We calculate STMCs for multiple possible fault types and use fault prevalence information to aggregate STMCs into a single dollar value to determine the cost-effectiveness of a potential sensor investment. We conducted a case study using Oak Ridge National Laboratory's Flexible Research Platform (FRP) test facility as a reference. The case study demonstrates the feasibility of the analysis and highlights the key cost considerations in sensor selection for FDD. The results also indicate that identifying and installing the few key sensor(s) is critical to cost-effectively improve FDD performance.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗