Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software: data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

XRF-MAPS

x-ray fluorescence (XRF) imaging typically involves the creation and analysis of 3D data sets, where at each scan position a full energy dispersive x-ray spectrum is recorded. This allows one to later process the data in a variety of different approaches, e.g., by spectral region of interest (ROI) summation with or without background subtraction, principal component analysis, or fitting. XRF-Maps is a C++ open source software package that implements these functions to provide a tool set for the analysis of XRF data sets.

GLOWACKI, ARTHURT↗

SYMBIOSYS: A Methodology for Performance Analysis of Composable HPC Data Services

Microservices are a powerful new way of building, customizing, and deploying distributed services owing to their flexibility and maintainability. Several large-scale distributed platforms have emerged to serve the growing needs of data-centric workloads and services in commercial computing. Concurrently, high-performance computing (HPC) systems and software are rapidly evolving to meet the demands of diversified applications and heterogeneity. The interplay of hardware factors, software configuration parameters, and the flexibility offered with a microservice architecture makes it nontrivial to estimate the optimal service instantiation for a given application workload. Further, this problem is exacerbated when considering that these services operate in a dynamic and heterogeneous HPC environment. An optimally integrated service can be vastly more performant than a haphazardly integrated one. Existing performance tools for HPC either fail to understand the request-response model of communication inherent to microservices or they operate within a narrow scope, limiting the insight that can be gleaned from employing them in isolation. We propose a methodology for integrated performance analysis of HPC microservices frameworks and applications called SYMBIOSYS. We describe its design and implementation within the context of the Mochi framework. This integration is achieved by combining distributed callpath profiling and tracing with a performance data exchange strategy that collects fine-grained, low-level metrics from the RPC communication library and network layers. The result is a portable, low-overhead performance analysis setup that provides a holistic profile of the dependencies among microservices and how they interact with the Mochi RPC software stack. Using HEPnOS, a production-quality Mochi data service, we demonstrate the low-overhead operation of SYMBIOSYS at scale and use it to identify the root causes of poorly performing service configurations.

microservices↗

Machine Intelligence-Centered System for Automated Characterization of Functional Materials and Interfaces

Classic design of experiment relies on a time-intensive workflow that requires planning, data interpretation, and hypothesis building by experienced researchers. Here, in this paper, we describe an integrated, machine-intelligent experimental system which enables simultaneous dynamic tests of electrical, optical, gravimetric, and viscoelastic properties of materials under a programmable dynamic environment. Specially designed software controls the experiment and performs on-the-fly extensive data analysis and dynamic modeling, real-time iterative feedback for dynamic control of experimental conditions, and rapid visualization of experimental results. The system operates with minimal human intervention and enables time-efficient characterization of complex dynamic multifunctional environmental responses of materials with simultaneous data processing and analytics. The system provides a viable platform for artificial intelligence (AI)-centered material characterization, which, when coupled with an AI-controlled synthesis system, could lead to accelerated discovery of multifunctional materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

AXEAP (ARGONNE X-RAY EMISSION PACKAGE)

Argonne X-ray Emission Package (AXEAP), a singular purpose software package for processing X-ray emission (XES) images collected with a 2-dimensional position sensitive pixel array detector, has been developed. AXEAP can rapidly convert XES image files into a spectral form by applying parallel computation and unsupervised machine learning to compute vast amount of image data. Special focus has been placed on designing user-friendly-interface for processing multiple edges, non-resonant and resonant x-ray emission image analysis, in order to make data processing quick and easy. AXEAP is free software and is written in MATLAB, armed with powerful libraries and toolboxes. The software runs on all common operating systems such as Linux, Window, and Mac.

SUN, CHENGJUN↗

Near real-time streaming analysis of big fusion data

Experiments on fusion plasmas produce high-dimensional data time series with ever-increasing magnitude and velocity, but turn-around times for analysis of this data have not kept up. For example, many data analysis tasks are often performed in a manual, ad-hoc manner some time after an experiment. In this article, we introduce the Delta framework that facilitates near real-time streaming analysis of big and fast fusion data. By streaming measurement data from fusion experiments to a high-performance compute center, Delta allows computationally expensive data analysis tasks to be performed in between plasma pulses. This article describes the modular and expandable software architecture of Delta and presents performance benchmarks of individual components as well as of an example workflow. Focusing on a streaming analysis workflow where electron cyclotron emission imaging (ECEi) data is measured at KSTAR on the National Energy Research Scientific Computing Center's (NERSC's) supercomputer we routinely observe data transfer rates of about 4 Gigabit per second. In NERSC, a demanding turbulence analysis workflow effectively utilizes multiple nodes and graphical processing units and executes them in under 5 min. We further discuss how Delta uses modern database systems and container orchestration services to provide web-based real-time data visualization. For the case of ECEi data we demonstrate how data visualizations can be augmented with outputs from machine learning models. Here, by providing session leaders and physics operators, results of higher-order data analysis using live visualizations may make more informed decisions on how to configure the machine for the next shot.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

LogPath: Log data based energy consumption analysis enabling electric vehicle path optimization

Vehicle navigation and path optimization require a more meticulous approach when it deals with EVs (electric vehicles) and SDVs (software-defined vehicles), due to lengthy charging times and the lack of charging infrastructure. Long-distance freight EV trucking needs path guidance with accurate energy consumption estimates to prevent charging-related failures. We developed a novel energy consumption estimation approach that only uses battery log data to extract major vehicle parameters to increase EV navigation accuracy without additional sensors. This is enabled by extracting multiple drive modes from the log data for analysis. The system provides 1) routes, 2) charge locations, 3) charging times, and 4) optimal vehicle speeds that guarantee the shortest travel time. Here we successfully validated the system using log data collected from an EV and Tesla's Supercharging map in the US and compared it with the commercially available navigation system, Tesla's trip planner, whose capabilities solely include charging time and routing.

EV (Electric vehicles) navigation↗

Data Assimilation using Non-invasive Monte Carlo Sensitivity Analysis of Reactor Kinetics Parameters

Accurately predicting the criticality of an experiment before interacting with the experimental components is very important for criticality safety. Radiation transport software can be utilized to calculate the effective neutron multiplication factor of a nuclear system. Because of the integral nature of the effective neutron multiplication factor, the value calculated contains various sources of nuclear-data induced uncertainty. The sensitivity analysis and data assimilation technique presented in this paper exhibit one possible method of identifying and reducing the effective neutron multiplication factor nuclear-data induced uncertainty. The results presented in this work show that it is possible to use relative sensitivity coefficients of the prompt neutron decay constant and the effective delayed neutron fraction to 239 Pu nuclear data to reduce nuclear-data induced uncertainties in the effective neutron multiplication factor. This work has been utilized by members of the Los Alamos National Laboratory project EUCLID (Experiments Underpinned by Computational Learning for Improvements in Nuclear Data) for optimally designing a new experiment, which will be used to reduce compensating errors in 239 Pu nuclear data.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The U.S. CMS HL-LHC R&D Strategic Plan

The HL-LHC run is anticipated to start at the end of this decade and will pose a significant challenge for the scale of the HEP software and computing infrastructure. The mission of the U.S. CMS Software & Computing Operations Program is to develop and operate the software and computing resources necessary to process CMS data expeditiously and to enable U.S. physicists to fully participate in the physics of CMS. We have developed a strategic plan to prioritize R&D efforts to reach this goal for the HL-LHC. This plan includes four grand challenges: modernizing physics software and improving algorithms, building infrastructure for exabyte-scale datasets, transforming the scientific data analysis process and transitioning from R&D to operations. We are involved in a variety of R&D projects that fall within these grand challenges. In this talk, we will introduce our four grand challenges and outline the R&D program of the U.S. CMS Software & Computing Operations Program.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

pathSQE : an automated workflow for single-crystal inelastic neutron scattering data processing and analysis

Inelastic neutron scattering (INS) experiments utilizing modern time-of-flight spectrometers enable the comprehensive mapping of the energy (E)- and momentum (Q)-resolved dynamical structure factor of single crystals, probing both the lattice and magnetic excitations. Yet, the large size and complexity of four-dimensional INS data are challenging current analysis workflows, often resulting in an underutilization of the measured information. To help address this issue, this paper introduces new software interfaced with the Mantid framework, pathSQE, designed to streamline the processing, analysis and interpretation of 4D single-crystal INS data. By automating key tasks such as 1D/2D slicing, symmetrization, Brillouin zone folding, data visualization, prioritization and filtering, and comparisons with simulations, pathSQE facilitates and accelerates INS data analysis workflows. Here, this paper outlines the features and implementation and provides several illustrations of the use of pathSQE on data collected on single crystals using direct-geometry time-of-flight spectrometers at the Spallation Neutron Source, including Ge, FeSi, MnO and SnS single-crystal measurements on the ARCS, HYSPEC and CNCS neutron spectrometers. Beyond streamlining post-experiment data processing, pathSQE establishes an automated and modular processing pipeline that could support future real-time experiment steering.

36 MATERIALS SCIENCE↗

WISP: Watching grid Infrastructure Stealthily through Proxies (Final Technical Report)

The complex interdependencies of cyber systems (sensors and communications), physical grids and associated electricity market operations make protecting electric power grids a significant challenge. The energy sector is constantly under new, targeted, advanced and dangerous cyber-attacks that have the potential to result in the loss of human life. These threats are further exacerbated by our need to modernize the grid. One focus of cyber security research in smart grids is the securing of the SCADA system through advanced intrusion detection systems (IDS) and bad data detection algorithms in state estimation. These methods either require full knowledge of the system topology and parameters or fail to understand the physical behaviors under attack. WISP (Watching grid Infrastructure Stealthily through Proxies) is designed to provide additional protection to the power grid using only publicly available data. In particular, WISP exploits the spatio-temporal nature of the real time locational marginal prices (LMPs), in conjunction with other information such as bids, weather, outages and load data to analyze anomalous power pricing behaviors and then correlate those observations to localize regions of interest and identify potential cyber events. WISP is non-intrusive as the tool is deployed as a service in the Cloud or on premise and provides reliable information to system operators for enhanced situational awareness, without impeding energy delivery functions. The WISP technology comprises three modules: the data-driven anomaly detection core, the vulnerability and risk analysis and the root cause analysis. The data-driven anomaly detection core performs the tasks of feature selection, anomaly detection and attack region localization. The vulnerability and risk analysis module provides system level information of the vulnerable variables and times, assisting the operators in selecting monitoring and protection nodes. The root cause analysis module takes the detection results and identifies potential operational conditions that contribute to the detected anomalies. In Phase I, we have demonstrated the feasibility and effectiveness of WISP. We developed a realistic electricity market simulator capable of generating normal and attack market data under various operational conditions. We developed a series of cyber-attack detection and analysis algorithms and evaluated them under multiple data sources. Finally, we integrated all modules into an end-to-end software, providing functions for data management, data analytics and visualization. Specifically, we have achieved: (i) real-time data acceptance from external utility interfaces with >99% acceptance rate; (ii) high performance anomaly detection algorithms with >98% detection accuracy and <0.1% false alarm rate; and (iii) ultra-low computing delay <50 milliseconds. Additionally, our team developed algorithms to identify the vulnerable variables in electricity market operations and root cause analysis functions to identify major contributors to the price spikes. These ancillary modules are necessary when deploying WISP in real world industry environment. In Phase II, we have demonstrated the effectiveness of WISP software on realistic largescale power systems. We performed red team testing for the Phase I WISP software and identified software vulnerabilities and implemented corresponding mitigation solutions. We adapted the electricity market simulator for the Texas synthetic 2000-bus system and generated datasets for the false data injection attacks. We created database and visualization interfaces for the Texas system and the ISO New England system. We performed software optimization in terms of operation efficiency, computing speed and detection accuracy. Finally, we tested the software on the Texas system and the ISO New England system and evaluated the detection performance. Overall, we achieved above 89% detection rate, below 3% false alarm rate and below 37 seconds of end-to-end detection delay.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Ideas and perspectives: Enhancing research and monitoring of carbon pools and land-to-atmosphere greenhouse gases exchange in developing countries

Carbon (C) and greenhouse gas (GHG) research has traditionally required data collection and analysis using advanced and often expensive instruments, complex and proprietary software, and highly specialized research technicians. Partly as a result, relatively little C and GHG research has been conducted in resource-constrained developing countries. At the same time, these are often the same countries and regions in which climate change impacts will likely be strongest and in which major science uncertainties are centered, given the importance of dryland and tropical systems to the global C cycle. Increasingly, scientific communities have adopted appropriate technology and approach (AT&A) for C and GHG research, which focuses on low-cost and low-technology instruments, open-source software and data, and participatory and networking-based research approaches. Adopting AT&A can mean acquiring data with fewer technical constraints and lower economic burden and is thus a strategy for enhancing C and GHG research in developing countries. However, AT&A can have higher uncertainties; these can often be mitigated by carefully designing experiments, providing clear protocols for data collection, and monitoring and validating the quality of obtained data. For implementing this approach in developing countries, it is first necessary to recognize the scientific and moral importance of AT&A. At the same time, new AT&A techniques should be identified and further developed. All these processes should be promoted in collaboration with local researchers and through training local staff and encouraged for wide use and further innovation in developing countries.

Kim, Dong-Gill↗

Reducing Uncertainty of Fielded Photovoltaic Performance (Final Technical Report)

Improved analysis and reporting of photovoltaic (PV) field performance increases the certainty of owners and financiers that systems will perform as expected. Advanced module technologies (e.g., PERC, HJT, and bifacial) introduce new degradation mechanisms and performance characteristics. The FY19-21 Reducing Uncertainty project leveraged data from the ever-increasing PV fleet to develop models and understanding of the field performance of existing and new technologies. Specifically, we accomplished: report on field performance and degradation rates for high-efficiency silicon (HJT, PERC, IBC) and more conventional technologies; developed automated analysis techniques to quantify system performance (performance ratio, energy yield) and production shortfalls (soiling, degradation, availability); refined the RdTools software toolkit to bring standard, validated analysis techniques to bear on third-party data; analyzed and reported on large datasets including Treasury data and Lawrence Berkeley National Laboratory's Utility-Scale dataset to expand the high-quality degradation-rate histogram published previously; worked with industry partners and the DuraMAT data hub to enable private parties to share and aggregate PV production data anonymously, leveraging cloud-based data analysis infrastructure and publishing on US fleet-scale performance comprising over 7GW of operating systems. (https://www.nrel.gov/pv/fleet-performance-data-initiative.html). Through our industry collaborations we have engaged in NDA-covered data transfer with twelve PV fleet owners as of January 2022, with more agreements in negotiation. Our scalable cloud-based time series database contains over 30 billion rows (20TB) of PV time series data, representing over 1700 commercial and utility-scale systems, and over 7.2 GW of DC capacity (Fig 1). Initial field performance results have been distributed in several public reports. Because our fleet composition and data quality methods are continually improving, annual updates to these results are published to our PV Fleet webpage [ https://www.nrel.gov/pv/fleet-performance-data-initiative.html ] and DuraMAT data hub [DOI: 10.21948/1842958]. Another existing dissemination channel used for observed soiling losses is a map we maintain for soiling losses. Additional products developed include a report detailing fleet-wide performance index, availability, startup loss and snow loss factors, a detailed report on the 1603 grant dataset comprising over 100,000 PV systems with failure and performance details and a utility-scale report coauthored with LBNL on 31 GW of system performance.

14 SOLAR ENERGY↗

WizEM

The software provides a flexible pipeline for artificial intelligence (AI) assisted analysis of data collected from an electron microscope. This tool distinctly provides near real-time image quantification results during data collection, incorporating feedback from a user to tune performance.

Akers, Sarah [Pacific Northwest National Laborator↗

Bayes Inference Engine (BIE) Test Plan: TiltConeBeamCombo

The Bayes Inference Engine (BIE) is a general software tool intended to be used primarily in the analysis of radiographic data for density. An analysis is set up in the BIE by representing the problem as a collection of modules called glyphs. All glyphs to be used in creating the forward model of the experiment will be tested separately for a range of inputs. The testing of the adjoint code is performed in a separate document.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Demonstration and Evaluation of a Non-Invasive, Low-Cost, Strap-On Sensor for Natural Gas Meters

The U.S. General Services Administration (GSA) is interested in installing internet-connected, gas submeters to better understand gas consumption in its portfolio of buildings. The GSA in partnership with the National Renewable Energy Laboratory conducted a demonstration to assess a specific submeter technology. This technology was implemented at two GSA separate facilities located in Dallas, Texas. This demonstration evaluated hardware and software installations and integrations, data integrity and accuracy, and included an economic analysis. This demonstration evaluated a gas submeter technology provided by the vendor Vata Verks, who produces a non-invasive strap-on submeter. The company was founded with a mission to conserve water (and eventually gas) cheaply and simply, as explained on its website. The product intends to streamline submeter deployments for gas and water by eliminating most hardware costs while allowing for easy integration of submetered data into other systems such as the Building Automation System and removing tenant and building disruption This demonstration evaluated the product features when used on a gas utility meter.

03 NATURAL GAS↗

Physics-Informed Learning Machines for Multiscale and Multiphysics Problems (PHILMS) (Technical Report)

The research work at University of California Santa Barbara (UCSB) resulted in several new developments in the areas of scientific machine learning, numerical analysis, and practical methods for data-driven modeling, prediction, reductions, and simulation. Many of the projects were carried out in collaboration with members of the national laboratories at Sandia National Laboratories (SNL), Pacific Northwestern National Laboratories (PNNL), and other institutions. Results included developing new scientific machine learning methods, related theory and mathematical frameworks for analysis and training, data-driven numerical solvers, and related tools and software for scientific computation. During the support period, over 16+ papers were submitted for publication, and 4 open-source software packages were developed and released (available at http://atzberger.org/). In addition, 7+ students and 2 post-docs were mentored in collaboration with the laboratory staff for future careers in academia, government labs, and industry.

97 MATHEMATICS AND COMPUTING↗

Transient Data Library of Solar Grid Integrated Distributed System

This submission contains an open-source library of transient events in distributed system with high solar PV. The library includes the collected data, related documents and scripts for loading the data. The data library is built for transient event detection and machine learning based analysis algorithm development. The data was collected via both field test and software simulation. The units for the data are included in the data file headers for each data series. A text editor or spreadsheet software, such as Excel, and Matlab is required to view the data.

algorithms↗