Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

Solution scattering at the Life Science X-ray Scattering (LiX) beamline

This work reports the instrumentation and software implementation at the Life Science X-ray Scattering (LiX) beamline at NSLS-II in support of biomolecular solution scattering. For automated static measurements, samples are stored in PCR tubes and grouped in 18-position sample holders. Unattended operations are enabled using a six-axis robot that exchanges sample holders between a storage box and a sample handler, transporting samples from the PCR tubes to the X-ray beam for scattering measurements. The storage box has a capacity of 20 sample holders. At full capacity, the measurements on all samples last for ∼9 h. For in-line size-exclusion chromatography, the beamline-control software coordinates with a commercial high-performance liquid chromatography (HPLC) system to measure multiple samples in batch mode. The beamline can switch between static and HPLC measurements instantaneously. In all measurements, the scattering data span a wide q -range of typically 0.006–3.2 Å −1 . Functionalities in the Python package py4xs have been developed to support automated data processing, including azimuthal averaging, merging data from multiple detectors, buffer scattering subtraction, data storage in HDF5 format and exporting the final data in a three-column text format that is acceptable by most data analysis tools. These functionalities have been integrated into graphical user interfaces that run in Jupyter notebooks, with hooks for external data analysis software.

60 APPLIED LIFE SCIENCES↗

Visual Analytics for Deep Embeddings of Large Scale Molecular Dynamics Simulations

Molecular Dynamics (MD) simulation have been emerging as an excellent candidate for understanding complex atomic and molecular scale mechanism of bio-molecules that control essential bio-physical phenomenon in a living organism. But this MD technique produces large-size and long-timescale data that are inherently high-dimensional and occupies many terabytes of data. Processing this immense amount of data in a meaningful way is becoming increasingly difficult. Therefore, specific dimensionality reduction algorithm using deep learning technique has been employed here to embed the high-dimensional data in a lower-dimension latent space that still preserves the inherent molecular characteristics i.e. retains biologically meaningful information. Subsequently, the results of the embedding models are visualized for model evaluation and analysis of the extracted underlying features. However, most of the existing visualizations for embeddings have limitations in evaluating the embedding models and understanding the complex simulation data. We propose an interactive visual analytics system for embeddings of MD simulations to not only evaluate and explain an embedding model but also analyze various characteristics of the simulations. Our system enables exploration and discovery of meaningful and semantic embedding results and supports the understanding and evaluation of results by the quantitatively described features of the MD simulations (even without specific labels).

Chae, Junghoon↗

HumoNet: A Framework for Realistic Modeling and Simulation of Human Mobility Network

Understanding, analyzing, and predicting human mobility and dynamics are valuable to solving pressing problems, developing effective plans, and prescribing timely remedies. As a computational approach, realistic human mobility simulations allow us to understand, analyze, and predict complex systems, including human societies. Accurate simulations rely on (1) the model that captures interactions and behaviors of myriad entities in our society and (2) the mapping of model instances to real-world entities. Taking this into account, this paper introduces the Human Mobility Network simulation framework (HumoNet), an integrated patterns of life (POL) simulation framework that leverages real-world data layers including transportation networks, points of interest, populations, popularity, and human trajectories. HumoNet is a data informed model in which agents are equipped with activities, locomotion, and planning capabilities. To simulate realistic kinematic maneuvers of individuals in transportation networks, HumoNet harnesses a microscopic traffic simulator that provides interaction among vehicles and traffic objects. In this paper, we describe the framework, outline our methodologies, and discuss the data processing and challenges of each data layer. Through experiments, we demonstrate that our simulations capture key features of human mobility by comparing them to the literature and real data using standard measures of human mobility (i.e., the radius of gyration, number of locations visited, level of exploration) and metrics scoring (i.e., Jensen-Shannon divergence). We envision that the synthetic data produced by HumoNet will serve as a benchmark for analyzing epidemics, deploying EV charging networks, and validating AI/ML tasks such as location prediction.

Kim, Joon-Seok↗

A Multi-layer, Data-driven Advanced Reasoning Tool for Intelligent Data Mining and Analysis for Smart Grids

This paper presents the multi-layer, data-driven advanced reasoning tool (M-DART), a proof-of-principle decision support tool for improved power system operation. M-DART will cross-correlate and examine different data sources to assess anomalies, infer root causes, and anneal data into actionable information. By performing higher-level reasoning “triage” of diverse data sources, M-DART focuses on early detection of emerging power system events and identifies highest priority actions for the human decision maker. M-DART represents a significant advancement over today’s grid monitoring technologies that apply offline analyses to derive model-based guidelines for online real-time operations and use isolated data processing mechanisms focusing on individual data domains. The development of the M-DART will bridge these gaps by reasoning about results obtained from multiple data sources that are enabled by the smart grid infrastructure. This hybrid approach integrates a knowledge base that is trained offline but tuned online to capture model-based relationships while revealing complex causal relationships among data from different domains.

data mining, reasoning, smart grid, AMI, power gen↗

Parallel Harness for Informatic Stream Hashing

PHISH is a lightweight framework which a set of independent processes can use to exchange data as they run on the same desktop machine, on processors of a parallel machine, or on different machines across a network. This enables them to work in a coordinated parallel fashion to perform computations on either streaming, archived, or self-generated data. The PHISH distribution includes a simple, portable library for performing data exchanges in useful patterns either via MPI message-passing or ZMQ sockets. PHISH input scripts are used to describe a data-processing algorithm, and additional tools provided in the PHISH distribution convert the script into a form that can be launched as a parallel job.

Plimpton, Steve↗

Imaging Bragg Edge Analysis TooLs for Engineering Structures (iBeatles)

The Spallation Neutron Source (SNS) at Oak Ridge National Laboratory (ORNL) provides pulsed neutrons with energies varying from epithermal to cold. In preparation for VENUS, the neutron imaging beamline to be located at beam port 10, we have performed a series of experiments focused on wavelength-dependent radiography and computed tomography for a broad range of applications, from materials science to biological tissues.One of the time-of-flight (TOF) techniques that is of interest to the scientific community is the 2-dimensional mapping of phases and average crystalline plane orientation in samples both ex-situ and during applied stresses such as tensile loading and heating. This technique is known as Bragg edgeimaging and relies on the identification of changes of transmission values, fitting of the edge to measure its displacement, and thus identify the shift in lattice parameter due to stresses. One of the challenges of TOF imaging measurements is the amount of data and the inability to observe Bragg edge shifts in real time during an experiment. Thus, we have been focusing on creating a Python-based interface that allows fast data processing and instantaneous mapping and fitting of the Bragg edges, and their evolution through time. Python libraries and Jupyter notebooks have been implemented to facilitate decision making during an experiment. The advantage of the notebooks is the possibility to guide an experiment as they can quickly process and display Bragg edge data. These notebooks can be used independently, or can be combined in a Python Graphical User Interface (GUI) tool called iBeatles. This interface permits visualization and fitting of the Bragg edges, and ultimately back-projects the fitting results onto the radiographs to display a strain map. Assuming data collection has sufficient statistics, the strain mapping analysis can be performed on a pixel-by-pixel basis. This development is a step forward toward a better user experience at the future VENUS beamline in terms of live feedback and productivity. Analysis that used to take days of switching between different applications can now be done in minutes within the

Bilheux, JeanChristophe [Oak Ridge National Labora↗

Leaf structural and chemical traits, and BNL field campaign sample details, San Lorenzo, Panama, 2020

This data package includes leaf traits, canopy traits and sample details for leaves from 71 species sampled from the San Lorenzo forest canopy crane site, Panama (PA-SLZ) during the BNL field campaign in January to March 2020. Each leaf sample is described with species, phenological stage and location within vertical canopy profiles. Leaf area index (LAI) and height is presented for each canopy profile location. Leaf mass per area (LMA), leaf water content (LWC) and leaf carbon and nitrogen content are included for a subset of the samples. This data package includes sample details, processed data for leaf traits and LAI (*.csv), LAI raw data (compressed as *.zip) and digital camera images (*.jpg, compressed as *.zip) of the leaf samples. Metadata files include data descriptions (_dd.csv) for tabular data, a list of all species sampled during the campaign (*.csv) and a detailed description of the field campaign protocol and methods (*.pdf). See related datasets for leaf gas exchange, leaf water potential and leaf spectral measurements made on the samples described here.

54 ENVIRONMENTAL SCIENCES↗

The Consortium for Advanced Sorghum Phenomics (CASP). Final report

The goal of CASP was to accelerate breeding of biomass sorghum [Sorghum bicolor (L.) Moench] by identifying genotypes exhibiting high yield under well-watered, pre- or post-drought and/or salinity-stress conditions. We did this by combining high-throughput, non-invasive drone phenotyping with genomics and molecular profiling. Field-based phenotyping utilized a multi-modal sensor suite of LiDAR, multispectral cameras, and thermal cameras mounted on a commercial drone to detect traits required for yield prediction and selection of drought and saline tolerant lines of sorghum. Traits of interest included plant height (PH), leaf area index (LAI), wet biomass (BMW), and biomass at 65% moisture (BM65) and were measured from emergence to harvest on a weekly basis over three growing seasons. The final output were measurements of traits on a plot-by-plot basis, identified by the plot ID used by the Proprietary data processing software enabled raw field data to be turned into plant traits and delivered to the PNNL and JGI within the same workday.

09 BIOMASS FUELS↗

Wireless Sensor Modalities at a Nuclear Plant Site to Collect Vibration Data

One of the major contributors to the total operating costs of domestic nuclear fleet of reactors today is the operation and maintenance (O&M) costs. These include labor-intense preventive maintenance programs involving manually-performed inspection, calibration, testing, and maintenance of plant assets at periodic frequency and time-based replacement of assets at periodic frequency, irrespective of their conditions. This has resulted in a labor-centric business model to achieve high capacity factors. To build an optimal maintenance program, it’s time to transition from this labor-centric business model to a technology-centric business model. Fortunately, there are technologies (advanced sensor, data analytics, and risk assessment methodologies) that will support this transition. The technology-centric business model will result in significant plant life extension and reduction of time-based maintenance activities. This will drive down O&M costs as labor is a rising cost and technology is a declining cost. This approach will lay the foundation for real-time condition assessment of plant assets, allowing condition-based maintenance to enhance plant safety, reliability, and economics of operation. The goal of this project is to address challenges in the area of digital monitoring, i.e., the application of advanced sensor technologies (particularly wireless sensor technologies) and science-based data analytic capabilities to advance online monitoring and predictive maintenance in nuclear plants to improve plant performance (efficiency gain and economic competitiveness). To achieve the project goal, in partnership with Exelon Generating Company (Exelon), researchers from Idaho National Laboratory (INL) and Oak Ridge National Laboratory (ORNL) are performing research and development (R&D) to demonstrate application of wireless sensors using the distributed antenna system and advanced data analytics to achieve predictive maintenance. In the report, wireless vibration sensors, vibration data and its indicator are described. The wireless vibration sensors presented in this report support three types of wireless communication, namely, Wi-Fi, cellular, and 900 MHz. These wireless vibration sensors are considered by partner plant site for installation on plant asset to enable online vibration monitoring to replace periodic measurements. These vibration data along with other plant process data will be utilized to develop diagnostic and prognostic models.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

gvr (s1)

The G-band vapor radiometer (GVR) provides time-series measurements of brightness temperatures from four double sideband channels around the 183.31-GHz water vapor line. Atmospheric emission in this spectral region is primarily due to water vapor, with some influence from liquid water. The GVR is approximately 30 times more sensitive to water vapor than the two-channel microwave radiometer (MWR) for a precipitable water vapor (PWV) amount of less than 2.5 mm. Measurements from this instrument are, therefore, especially useful during low-humidity conditions (PWV < 5 mm). Data products include mentor-processed data.

54 ENVIRONMENTAL SCIENCES↗

Datasets for "Excessive rainfall leads to maize yield loss of a comparable magnitude to extreme drought in the United States"

Updated on 2021/07/01 Add python codes for data processing of the NASS yield data, PRISM climate data, RMA insurance data, agmip yield simulation data, and related geospatial analysis-------------------This repository is for the datasets and codes for “Excessive rainfall leads to maize yield loss of a comparable magnitude to extreme drought in the United States” (Li et al. 2019 GCB). The datasets include maize yield, climate data, maize yield loss from crop insurance data in the US, etc. The datasets can reproduce the results of the paper and they can also be reused to explore other topics. Please cite Li et al (2019) or acknowledge this dataset (DOI: 10.6084/m9.figshare.7581473) when using it.

Agricultural spatial analysis and modelling↗

Parallel Harness for Informatic Stream Hashing

PHISH is a lightweight framework which a set of independent processes can use to exchange data as they run on the same desktop machine, on processors of a parallel machine, or on different machines across a network. This enables them to work in a coordinated parallel fashion to perform computations on either streaming, archived, or self-generated data. The PHISH distribution includes a simple, portable library for performing data exchanges in useful patterns either via MPI message-passing or ZMQ sockets. PHISH input scripts are used to describe a data-processing algorithm, and additional tools provided in the PHISH distribution convert the script into a form that can be launched as a parallel job.

Steve Plimpton, Tim Shead↗

A Visual Analytics Framework for Reviewing Streaming Performance Data

Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data without missing important changes or patterns. To support streaming data analysis, we introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization. The data management module collects various computing and communication performance metrics from the monitored system using streaming data processing techniques and feeds the data to the other two modules. The analysis module automatically identifies important changes and patterns at the required latency. In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results. Finally, the interactive visualization module provides the analysts with a coherent view of the changes and patterns in the continuously captured performance data. Through a multi-faceted case study on performance analysis of parallel discrete-event simulation, we demonstrate the effectiveness of our framework for identifying bottlenecks and locating outliers.

Human-centered computing↗

A Ceph S3 Object Data Store for HEP

We present a novel data format design that obviates the need for data tiers by storing individual event data products in column objects. The objects are stored and retrieved through Ceph S3 technology, with a layout designed to minimize metadata volume and maximize data processing parallelism. Performance benchmarks of data storage and retrieval are presented.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

dbprocessing

dbprocessing is a python framework for automating data processing pipelines. The dbprocessing package uses a configuration file to define the input, intermediate, and output data products for a particular data type as well as the processes that connect those data products. In addition, dbprocessing uses python scripts called inspectors to determine if a particular file matches a configured data product. If a particular data product is found, it is ingested into the dbprocessing database (sqlite3 or postgres) which then triggers all of the chained processes to the final data product. While the framework is written in python, it can run software in any language but may require a “wrapper” to translate the dbprocessing command line arguments to the form expected by the software.

Walker, Andrew↗

Quantifying Transient Uncertainty in the BEAVRS Benchmark Using Time Series Analysis Methods

Advances in computation have brought about significant improvements in creating fast-running high-fidelity simulations of nuclear cores. The BEAVRS benchmark is a highly-detailed PWR specification with two cycles of measured operational data used to validate high-fidelity core analysis methods. This PWR depletion benchmark captures the fine details of the LWR fuel assemblies, burnable absorbers, in-core fission detectors, core loading and shuffling patterns. Specifically, 58 of the 193 assemblies contain in-core detectors with measurements taken over 61 axial positions every month. These detectors are U-235 fission chambers with slightly varying mass of U-235. The collected signals are normalized on a given assembly permitting full core comparisons. The fuel layout for cycle 1 and instrument tube locations for the reactor are given in figures 1 and 2 respectively. Through a series of data processing and comparisons, it was shown that axially integrated radial maps of reaction rates were in close agreement between provided detector data and calculated data. More recently, the focus of the BEAVRS project has been on quantifying uncertainty to further assess the validity of aforementioned results. A close investigation of sources of error shows that uncertainties from operational nuclear data arise primarily from data measurements and processing. While the errors in axial realignment and interpolation seem significant, these sources affect only a small subset of data points near grid spacers and do not persist throughout the entire core. Instead, measurement uncertainty from detector count rates and radial integration dominate the uncertainty in measured data are determined as the most significant sources of error. This uncertainty analysis treated each given burnup step as independent of comparable data at neighboring burnup steps. More recent work has been targeted at understanding how reaction rates vary over time, in order to determine whether calculated reaction rates follow any observable trends. This abstract hones in on transient uncertainty quantification, in order to compare observed data against models for transient behavior. The first section of the paper uses linear regression tools to fit operational data, while the latter portion of the paper explores more complex simulation tools to fit operational data. Ultimately, the BEAVRS benchmark aims to serve as a true non-proprietary international benchmark for the validation of high-fidelity tools. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Damage Pseudo Cross Section Generator

This software is a python script for generating group damage pseudo-cross-sections for specific elements, and tabulating them in forms useful to python or to office-open workbooks (xlsx). The damage pseudo-cross-section is needed for calculating how much neutron radiation damage a structural material receives in a reactor measured in displacements per atom (DPA). The process of generating these pseudo-cross-sections is laborious as it requires running the nuclear data processing code, NJOY, for hundreds of isotopes, and then post-processing the data. This also allows many opportunities for human error to be involved. This script eliminates these problems by fully automating the process.

Gale, MicahD.↗

Neutron Thermal Cross Sections, Westcott Factors, Resonance Integrals, Maxwellian Averaged Cross Sections and Astrophysical Reaction Rates Calculated from the ENDF/B-VII.1, JEFF-3.1.2, JENDL-4.0, ROSFOND-2010, CENDL-3.1 and EAF-2010 Evaluated Data Libraries

We present calculations of neutron thermal cross sections, Westcott factors, resonance integrals, Maxwellian-averaged cross sections and astrophysical reaction rates for 843 ENDF materials using data from the major evaluated nuclear libraries and European activation file. Extensive analysis of newly-evaluated neutron reaction cross sections, neutron covariances, and improvements in data processing techniques motivated us to calculate nuclear industry and neutron physics quantities, produce s-process Maxwellian-averaged cross sections and astrophysical reaction rates, systematically calculate uncertainties, and provide additional insights on currently available neutron-induced reaction data. Nuclear reaction calculations are discussed and new results are presented. Due to space limitations, the present paper contains only calculated Maxwellian-averaged cross sections and their uncertainties. The complete data sets for all results are published in the Brookhaven National Laboratory report.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗