Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data engineering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62

On reading Youden: Learning about the practice of statistics and applied statistical research from a master applied statistician

From reading William John “Jack” Youden’s books and articles, Youden (1900-1971), an analytical chemist, becomes an applied statistician by the time he joins the National Bureau of Standards (NBS) in 1948. Here, this article traces his transition from chemist to applied statistician and what his body of work mostly at NBS (1948-1965) demonstrates about his practice of statistics and the role that applied statistical research plays in it. There is much we can learn from a master applied statistician.

42 ENGINEERING↗

Group-equivariant autoencoder for identifying spontaneously broken symmetries

We introduce the group-equivariant autoencoder (GE autoencoder), a deep neural network (DNN) method that locates phase boundaries by determining which symmetries of the Hamiltonian have spontaneously broken at each temperature. We use group theory to deduce which symmetries of the system remain intact in all phases, and then use this information to constrain the parameters of the GE autoencoder such that the encoder learns an order parameter invariant to these “never-broken” symmetries. This procedure produces a dramatic reduction in the number of free parameters such that the GE-autoencoder size is independent of the system size. We include symmetry regularization terms in the loss function of the GE autoencoder so that the learned order parameter is also equivariant to the remaining symmetries of the system. By examining the group representation by which the learned order parameter transforms, we are then able to extract information about the associated spontaneous symmetry breaking. We test the GE autoencoder on the 2D classical ferromagnetic and antiferromagnetic Ising models, finding that the GE autoencoder (1) accurately determines which symmetries have spontaneously broken at each temperature; (2) estimates the critical temperature in the thermodynamic limit with greater accuracy, robustness, and time efficiency than a symmetry-agnostic baseline autoencoder; and (3) detects the presence of an external symmetry-breaking magnetic field with greater sensitivity than the baseline method. Lastly, we describe various key implementation details, including a quadratic-programming-based method for extracting the critical temperature estimate from trained autoencoders and calculations of the DNN initialization and learning rate settings required for fair model comparisons.

42 ENGINEERING↗

PATHA: Performance Analysis Tool for HPC Applications

Large science projects rely on complex workflows to analyze terabytes or petabytes of data. These jobs are often running over thousands of CPU cores and simultaneously performing data accesses, data movements, and computation. It is difficult to identify bottlenecks or to debug the performance issues in these large workflows. In order to address these challenges, we have developed Performance Analysis Tool for HPC Applications (PATHA) using the state-of-art open source big data processing tools. Our framework can ingest system logs to extract key performance measures, and apply the most sophisticated statistical tools and data mining methods on the performance data. Furthermore, it utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of PATHA, we conduct a case study on the workflows from an astronomy project known as the Palomar Transient Factory (PTF). This study processed 1.6 TB of system logs collected on the NERSC supercomputer Edison. Using PATHA, we were able to identify performance bottlenecks, which reside in three tasks of PTF workflow with the dependency on the density of celestial objects.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Performance and Emission Investigations of Jatropha and Karanja Biodiesels in a Single-Cylinder Compression-Ignition Engine Using Endoscopic Imaging

Biofuels produced from non-edible sources that are cultivated on marginal lands represent a viable source of renewable and carbon-neutral energy. In this context, biodiesel obtained from Jatropha and Karanja oil seeds have received significant interest, especially in South Asian subcontinent. Both of these fuels are produced from non-edible plant seeds with high oil content, which can be grown on marginal lands. In this research, we have investigated the performance and emission characteristics of Jatropha and Karanja methyl esters (biodiesel) and their blends with diesel. Another objective is to examine the effect of long-term storage on biodiesel’s oxidative stability. The biodiesels were produced at Indian Institute of Technology Kanpur, (IIT Kanpur), India, and the engine experiments were performed in a single cylinder, 4-stroke, compression ignition engine at Argonne National Laboratory (ANL), Chicago. An endoscope was used to visualize in-cylinder combustion events and examine the soot distribution. The effects of fuel and start of injection (SOI) on engine performance and emissions were investigated. Results indicated that ignition delay was shorter with biodiesel. Consequently the cylinder pressure and premixed heat release were higher for diesel compared to biodiesel. Engine performance data for biodiesel (J100, K100) and biodiesel blends (J30, K30) showed an increase in break thermal efficiency (BTE) (10.9%, 7.6% for biodiesel and blend, respectively), BSFC (13.1% and 5.6%), and NOx emission (9.8% and 12.9%), and a reduction in BSHC (8.64% and 12.9%), and BSCO (15.56% and 4.0%). The soot analysis from optical images qualitatively showed that biodiesel and blends produced less soot compared to diesel. The temperature profiles obtained from optical imaging further supported higher NOx in biodiesels and their blends compared to diesel. Additionally, the data indicated that retarding the injection timing leads to higher BSFC, but lower flame temperatures and NOx levels along with higher soot formation for all test fuels. The physicochemical properties such as fatty acid profile, cetane number, and oxygen content in biodiesels support the observed combustion and emission characteristics of the fuels tested in this study. Finally, the effect of long-term storage is found to increase the glycerol content, acid value and cetane number of the two biodiesels, indicating some oxidation of unsaturated fatty acids in the fuels.

09 BIOMASS FUELS↗

Complex-valued universal linear transformations and image encryption using spatially incoherent diffractive networks

As an optical processor, a diffractive deep neural network (D2NN) utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing, completing its tasks at the speed of light propagation through thin optical layers. With sufficient degrees of freedom, D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light. Similarly, D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination; however, under spatially incoherent light, these transformations are nonnegative, acting on diffraction-limited optical intensity patterns at the input field of view. Here, we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light. Through simulations, we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products, a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination. The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.

36 MATERIALS SCIENCE↗

Relaxation of nonequilibrium populations after a pump: the breaking of Mathiessen's rule

From the early days of many-body physics, it was realized that the self-energy governs the relaxation or lifetime of the retarded Green’s function. So it seems reasonable to directly extend those results into the nonequilibrium domain. But experiments and calculations of the response of quantum materials to a pump show that the relationship between the relaxation and the self-energy only holds in special cases. Experimentally, the decay time for a population to relax back to equilibrium and the linewidth measured in a linear-response angle-resolved photoemission spectroscopy differ by large amounts. Theoretically, aside from the weak-coupling regime where the relationship holds, one also finds deviations and additionally one sees violations of Mathiessen’s rule. In this study, we examine whether looking at an effective transport relaxation time helps to analyze the decay times of excited populations as they relax back to equilibrium. We conclude that it may do a little better, but it has a fitting parameter for the overall scale which must be determined.

42 ENGINEERING↗

Randomized Projection for Rank-Revealing Matrix Factorizations and Low-Rank Approximations

Rank-revealing matrix decompositions provide an essential tool in spectral analysis of matrices, including the Singular Value Decomposition (SVD) and related low-rank approximation techniques. QR with Column Pivoting (QRCP) is usually suitable for these purposes, but it can be much slower than the unpivoted QR algorithm. For large matrices, the difference in performance is due to increased communication between the processor and slow memory, which QRCP needs in order to choose pivots during decomposition. Our main algorithm, Randomized QR with Column Pivoting (RQRCP), uses randomized projection to make pivot decisions from a much smaller sample matrix, which we can construct to reside in a faster level of memory than the original matrix. This technique may be understood as trading vastly reduced communication for a controlled increase in uncertainty during the decision process. Furthermore, for rank-revealing purposes, the selection mechanism in RQRCP produces results that are the same quality as the standard algorithm, but with performance near that of unpivoted QR (often an order of magnitude faster for large matrices). Additionally, we also propose two formulas that facilitate further performance improvements. The first efficiently updates sample matrices to avoid computing new randomized projections. The second avoids large trailing updates during the decomposition in truncated low-rank approximations. Our truncated version of RQRCP also provides a key initial step in our truncated SVD approximation, TUXV. These advances open up a new performance domain for large matrix factorizations that will support efficient problem-solving techniques for challenging applications in science, engineering, and data analysis.

97 MATHEMATICS AND COMPUTING↗

Making Data Prefetch Smarter: Adaptive Prefetching on POWER7

Hardware data prefetch engines are integral parts of many general purpose server-class microprocessors in the eld to- day. Some prefetch engines allow the user to change some of their parameters. The prefetcher, however, is usually enabled in a default conguration during system bring-up and dynamic reconguration of the prefetch engine is not an autonomic feature of current machines. Conceptually, however, it is easy to infer that commonly used prefetch algorithms, when applied in a xed mode will not help per- formance in many cases. In fact, they may actually de- grade performance due to useless bus bandwidth consump- tion and cache pollution. In this paper, we present an adap- tive prefetch scheme that dynamically modies the prefetch settings in order to adapt to the workload requirements. We implement and evaluate adaptive prefetching in the con- text of an existing, commercial processor, namely the IBM POWER7. Our adaptive prefetch mechanism improves per- formance with respect to the default prefetch setting up to 2.7X and 30% for single-threaded and multiprogrammed workloads, respectively. Categories

Jimenez, Victor↗

Adaptive prefetching on POWER7: Improving performance and power consumption

Hardware data prefetch engines are integral parts of many general purpose server-class microprocessors in the field today. Some prefetch engines allow users to change some of their parameters. But, the prefetcher is usually enabled in a default configuration during system bring-up, and dynamic reconfiguration of the prefetch engine is not an autonomic feature of current machines. Conceptually, however, it is easy to infer that commonly used prefetch algorithms—when applied in a fixed mode—will not help performance in many cases. In fact, they may actually degrade performance due to useless bus bandwidth consumption and cache pollution, which in turn, will also waste power. We present an adaptive prefetch scheme that dynamically modifies the prefetch settings in order to adapt to workloads

adaptive system↗

Securing Federated Learning Against Active Reconstruction Attacks

Federated Learning (FL) has amassed notable attention for its ability to preserve user privacy while emphasizing the retainment of model training efficiency. Due to this potential, FL has been integrated in many domains, such as healthcare, finance, law, and industrial engineering, where data cannot be easily exchanged due to sensitive information and strict privacy laws. However, current research has indicated that FL protocols are easily compromised by active data reconstruction attacks employed by actively dishonest servers. The malicious modification of global model parameters allows an actively dishonest server to obtain a direct copy of users’ private data via gradient inversion. Here, this class of attacks is highly underexplored and continues to be a major challenge due to the intense threat model. In this paper, we propose OASIS as a scalable and modality-agnostic defense based on data augmentation that counteracts active data reconstruction attacks while preserving model performance. To generalize our defense, we uncover the intuition behind gradient inversion that enables these attacks and theoretically establish the conditions by which the defense can be considered robust regardless of attack design. From this, we formulate our defense with data augmentation that illustrates its ability to undermine the attack principle. We evaluate OASIS on five real-world datasets–two image-based (ImageNet and CIFAR100) and three text-based (Wikitext, Stack Overflow, and Shakespeare)–which span diverse uses cases such as vision tasks and language modeling. Comprehensive evaluations on these datasets exhibit the efficacy of OASIS and highlight its feasibility as a solution.

97 MATHEMATICS AND COMPUTING↗

SetGo: Metadata Readiness for Scientific AI Datasets

Scientific datasets intended for AI use require both computational readiness for model training and metadata readiness for discovery, sharing, and reuse. The Readiness Engine for Data Integration (REDI) addresses computational readiness, but no corresponding tool evaluates whether a dataset’s metadata are sufficiently complete, governed, and standards-compliant for publication and agent-based consumption. Existing FAIR assessors operate only on published repository records, and no single system covers FAIR compliance, licensing, provenance, governance, reproducibility, and catalog readiness together. We present SetGo, an open-source Python toolkit that assesses and repairs metadata readiness across these six dimensions before a dataset is published or archived. Applied to four scientific corpora, SetGo surfaces deficiencies that general-purpose tools do not detect: ERA5 climate metadata scores 4% on ACDD 1.3 compliance; materials datasets fail OPTIMADE species-definition requirements; and PDB-derived proteomics data carries licensing terms incompatible with standard SPDX identifiers. Guided enrichment raises overall FAIR scores from 52–57% to 81–91%, and a single setgo publish command pushes to Hugging Face Hub, CKAN, or OpenMetadata with ML Commons Croissant 1.0 metadata sidecars. To support interactive and automated workflows, SetGo integrates with coding agents powered by large language models (LLMs) through a /setgo skill that enables natural-language execution of the full assess–enrich–publish loop, with user involvement limited to supplying missing metadata values.

Wilkinson, Sean [ORNL] (ORCID:0000000214437479)↗

BuildingSync® [SWR-18-28]

BuildingSync® is a standardized language for commercial building energy audit data that software developers can use to exchange data between audit tools. Taking the form of an XML schema, BuildingSync can be required by building owners and audit program managers to allow data analysis and aggregation across multiple buildings in order to evaluate program performance and analyze trends. BuildingSync was developed using the standard energy data terminology defined in the Building Energy Data Exchange Specification (BEDES). BuildingSync is one of the first implementations of BEDES Version 1.0, and it was developed in close collaboration with the BEDES working groups. BuildingSync standardizes the format of the energy audit data itself. It is not intended to standardize the calculations performed on these data by engineers, analysts, software packages, or other audit tools. For more, see: https://buildingsync.net/

Long, Nicholas↗

Autodataman

The autodataman tool is intended to be a very lightweight climate data database and data distribution engine.

Ordonez, AnaC↗

Data from Biodiesel Production from Engineered Sugarcane Lipids under Uncertain Feedstock Compositions: Process Design and Techno-Economic Analysis

In this study, different process schemes were designed and evaluated for biodiesel production from engineered cane lipids with uncertain fatty acid compositions. Four different process schemes were compared under (i) thermal glycerolysis and (ii) enzymatic glycerolysis approaches. These schemes were based on the biodiesel yield and economic indicators such as the net present value (NPV) and the minimum selling price (MSP) of biodiesel. A scheme with polar lipid separation under thermal glycerolysis resulted in the maximum NPV ($96.5 million) and minimum MSP ($1107/ton biodiesel), respectively. Through local sensitivity analysis, it was concluded that the cane lipid percentage is the most significant factor influencing process economics. A conjoint analysis of the lipid procurement price and cane lipid percent suggested that 15% cane lipids with a low lipid procurement price ($0.536/kg) results in a positive NPV. When the cane lipid price is higher (>$0.80/kg), a 20% lipid content should be considered to achieve a positive NPV. At 20% cane lipids, the worst-case and best-case scenarios were evaluated by analyzing the interplay of the three most important parameters, The best-case scenario revealed that the minimum NPV under any process scheme could yield more than $100 million (or MSP: $0.80/L), and the worst-case analysis showed that losses incurred by the plant could be as high as $80 million (MSP: $1.36/L). A Monte Carlo simulation indicated that there is a 70% chance of the plant being profitable (NPV > 0).

Conversion↗

VERAView User's Guide

VERAView has been developed as an interactive graphical interface for the visualization and engineering analyses of output data from VERA. The python-based software is easy to install and intuitive to use, and provides instantaneous 2D and 3D images, 1D plots, and alpha-numeric data from VERA multi-physics simulations. This document provides a brief overview of the software and some description of the major features of the application, including examples of each of the encapsulated ‘widgets’ that have been implemented thus far. VERAView is still under major development and large changes in the software and this document are still anticipated.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

BigData Express: Toward Predictable, Schedulable, and High-Performance Data Transfer [PowerPoint]

Topics include: • BigData Express: a schedulable, predictable, and high-performance data transfer service – A peer-to-peer, scalable, and extensible data transfer model – A visually appealing, easy-to-use web portal – A high-performance data transfer engine – A time-constraint-based scheduler – On-demand provisioning of end-to-end network paths with guaranteed QoS – Robust and flexible error handling – CILogon-based security

97 MATHEMATICS AND COMPUTING↗