Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗

AutoReP: Automatic ReLU Replacement for Fast Private Network Inference

The proliferation of the Machine-Learning-As-A-Service (MLaaS) market has brought to light a number of clients’ data privacy and security concerns. One promising solution is private inference (PI) techniques using cryptographic primitives. These techniques often come with high computation and communication overhead associated with the non-linear operator such as ReLU. Several approaches have been developed in reducing the number of ReLU operations, however, they either require a heuristic threshold selection or introduce significant accuracy drop. This work presents AutoReP, a gradient-based framework for non-linear operators reduction that aims to mitigate these concerns from a systematic perspective. AutoReP automates the process of discrete selection of ReLU and polynomial functions on neurons to accelerate PI applications. We also introduce distribution-aware polynomial approximation (DaPa) to accurately approximate ReLUs under given distribution, preserving model expressivity. Our experimental results demonstrate significant accuracy improvements of 6.12% (94.31%, 12.9K ReLU budget, CIFAR-10), 8.39% (74.92%, 12.9K ReLU budget, CIFAR-100), and 9.45% (63.69%, 55K ReLU budget, Tiny-ImageNet) over current state-of-the-art methods, e.g., SNL. Morever, AutoReP is applied to EfficientNet-B2 on ImageNet dataset, and achieved 75.55% accuracy with 176.1 × ReLU budget reduction.

Peng, Hongwu↗

Modular Semantic Tagging of Medline Abstracts and its Use in Inferring Regulatory Networks

We describe MedstractPlus, a resource for mining relations from the Medline bibliographic database that is currently under construction. It was built on the remains of Medstract, a previously created resource that included a biorelation server and an acronym database. MedstractPlus uses simple and scalable natural language processing modules to structure text, is designed with reusability and extendibility in mind, and adheres to the philosophy of the Linguistic Annotation Framework. We show how MedstractPlus has been used to provide seeds for a novel approach to inferring transcriptional regulatory networks from gene expression data.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluating In-Clique and Topological Parallelism Strategies for Junction Tree-Based Bayesian Inference Algorithm on the Cray XMT

Long viewed as a strong statistical inference technique, Bayesian networks have emerged to be an important class of applications for high-performance computing. We have applied an architecture-conscious approach to parallelizing the Lauritzen-Spiegelhalter Junction Tree algorithm for exact inferencing in Bayesian networks. In optimizing the Junction Tree algorithm, we have implemented both in-clique and topological parallelism strategies to best leverage the fine-grained synchronization and massive-scale multithreading of the Cray XMT architecture. Two topological techniques were developed to parallelize the evidence propagation process through the Bayesian network. One technique involves performing intelligent scheduling of junction tree nodes based on its topology and relative size. The second technique involves decomposing the junction tree into a much finer tree-like representation to offer much more opportunities for parallelism. We evaluate these optimizations on five different Bayesian networks and report our findings and observations. Another important contribution of this paper is to demonstrate the application of massive-scale multithreading for load balancing and use of implicit parallelism-based compiler optimizations in designing scalable inferencing algorithms.

99 GENERAL AND MISCELLANEOUS↗

Topology Inference of Unknown Networks Based on Robust Virtual Coordinate Systems

Learning and exploring the connectivity of unknown networks represent an important problem in practical applications of communication networks and social-media networks. Modeling large-scale networks as connected graphs is highly desirable to extract their connectivity information among nodes to visualize network topology, disseminate data, and improve routing efficiency. This paper investigates a simple measurement model in which a small subset of source nodes collect hop distance information from networked nodes in order to generate a virtual coordinate system (VCS) for networks of unknown topology. We establish the VCS to define logical distance among nodes based on principal component analysis and to determine connectivity relationship and effective routing methods. More importantly, we present a robust analytical algorithm to derive the VCS against practical issues of missing and corrupted measurements. We also develop a connectivity inference method which classifies nodes into layers based on the hop distances and derives partial information on network connectivity.

97 MATHEMATICS AND COMPUTING↗

Inferring the Dynamics of the State Evolution During Quantum Annealing

To solve an optimization problem using a commercial quantum annealer, one has to represent the problem of interest as an Ising or a quadratic unconstrained binary optimization (QUBO) problem and submit its coefficients to the annealer, which then returns a user-specified number of low-energy solutions. It would be useful to know what happens in the quantum processor during the anneal process so that one could design better algorithms or suggest improvements to the hardware. However, existing quantum annealers are not able to directly extract such information from the processor. Hence, in this work we propose to use advanced features of D-Wave 2000Q to indirectly infer information about the dynamics of the state evolution during the anneal process. Specifically, D-Wave 2000Q allows the user to customize the anneal schedule, that is, the schedule with which the anneal fraction is changed from the start to the end of the anneal. Furthermore, using this feature, we design a set of modified anneal schedules whose outputs can be used to generate information about the states of the system at user-defined time points during a standard anneal. With this process, called "slicing", we obtain approximate distributions of lowest-energy anneal solutions as the anneal time evolves. We use our technique to obtain a variety of insights into the annealer, such as the state evolution during annealing, when individual bits in an evolving solution flip during the anneal process and when they stabilize, and we introduce a technique to estimate the freeze-out point of both the system as well as of individual qubits.

42 ENGINEERING↗

Distributed Inference with Sparse and Quantized Communication

Here, we consider the problem of distributed inference where agents in a network observe a stream of private signals generated by an unknown state, and aim to uniquely identify this state from a finite set of hypotheses. We focus on scenarios where communication between agents is costly, and takes place over channels with finite bandwidth. To reduce the frequency of communication, we develop a novel event-triggered distributed learning rule that is based on the principle of diffusing low beliefs on each false hypothesis. Building on this principle, we design a trigger condition under which an agent broadcasts only those components of its belief vector that have adequate innovation, to only those neighbors that require such information. We prove that our rule guarantees convergence to the true state exponentially fast almost surely despite sparse communication, and that it has the potential to significantly reduce information flow from uninformative agents to informative agents. Next, to deal with finite-precision communication channels, we propose a distributed learning rule that leverages the idea of adaptive quantization. We show that by sequentially refining the range of the quantizers, every agent can learn the truth exponentially fast almost surely, while using just 1 bit to encode its belief on each hypothesis. For both our proposed algorithms, we rigorously characterize the trade-offs between communication-efficiency and the learning rate.

42 ENGINEERING↗

Inferring Reliability Model Parameters from Expert Opinion

Here, we propose a method for constructing bathtub models of reliability from opinion. The method is intended for reliability studies early in the design and prototyping of a new system, before data concerning reliability has become available. A stylized bathtub curve is presented for soliciting best engineering judgement from technical experts. By pooling these stylized curves, we produce data that can be used to infer parameters for a piece-wise Weibull model of reliability. A numerical example demonstrates the practicality of the method while also highlighting potential pitfalls when working with subjective data.

42 ENGINEERING↗

Inference for Nonparanormal Partial Correlation via Regularized Rank-Based Nodewise Regression

Abstract Partial correlation is a common tool in studying conditional dependence for Gaussian distributed data. However, partial correlation being zero may not be equivalent to conditional independence under non-Gaussian distributions. In this paper, we propose a statistical inference procedure for partial correlations under the high-dimensional nonparanormal (NPN) model where the observed data are normally distributed after certain monotone transformations. The NPN partial correlation is the partial correlation of the normal transformed data under the NPN model, which is a more general measure of conditional dependence. We estimate the NPN partial correlations by regularized nodewise regression based on the empirical ranks of the original data. A multiple testing procedure is proposed to identify the nonzero NPN partial correlations. The proposed method can be carried out by a simple coordinate descent algorithm for lasso optimization. It is easy-to-implement and computationally more efficient compared to the existing methods for estimating NPN graphical models. Theoretical results are developed to show the asymptotic normality of the proposed estimator and to justify the proposed multiple testing procedure. Numerical simulations and a case study on brain imaging data demonstrate the utility of the proposed procedure and evaluate its performance compared to the existing methods. Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database.

97 MATHEMATICS AND COMPUTING↗

Proteomics-inferred genome typing (PIGT) demonstrates inter-populationrecombination as a strategy for environmental adaptation

Analyses of ecological and evolutionary processes that shape microbial consortia are facilitated by comprehensive studies of ecosystems with low species richness. In the current study we evaluated the role of recombination in altering the fitness of chemoautotrophic bacteria in their natural environment. Proteomics-inferred genome typing (PIGT) was used to determine the genomic make-up of Leptospirillum group II populations in 27 biofilms sampled from six locations in the Richmond Mine acid mine drainage system (Iron Mountain, CA) over a four-year period. We observed six distinct genotypes that are recombinants comprised of segments from two parental genotypes. Community genomic analyses revealed additional low abundance recombinant variants. The dominance of some genotypes despite a larger available genome pool, and patterns of spatiotemporal distribution within the ecosystem, indicate selection for distinct recombinants. Genes involved in motility, signal transduction and transport were overrepresented in the tens to hundreds of kilobase recombinant blocks, whereas core metabolic functions were significantly underrepresented. Our findings demonstrate the power of PIGT and reveal that recombination is a mechanism for fine-scale adaptation in this system.

54 ENVIRONMENTAL SCIENCES↗

Inference of photosynthetic capacity parameters from chlorophyll a fluorescence is affected by redox state of PSII reaction centers

Solar-induced chlorophyll fluorescence (SIF) has been used to infer photosynthetic capacity parameters (e.g., the maximum carboxylation rate V cmax , and the maximum electron transport rate J max ). However, the precise mechanism and practical utility of such approach under dynamic environments remain unclear. In this study, we used the balance between the light and carbon reactions to derive theoretical equations relating chlorophyll a fluorescence (ChlF) emission and photosynthetic capacity parameters, and formulated testable hypotheses regarding the dynamic relationships between the true total ChlF emitted from PSII (SIF PSII ) and V cmax and J max . We employed concurrent measurements of gas exchanges and ChlF parameters for 15 species from six biomes to test the formulated hypotheses across species, temperatures, and limitation state of carboxylation. Our results revealed that SIF PSII alone is incapable of informing the variations in V cmax and J max across species, even when SIF PSII is determined under the same environmental conditions. In contrast, the product of SIF PSII and the fraction of open PSII reactions q L , which indicates the redox state of PSII, is a strong predictor of both V cmax and J max , although their precise relationships vary somewhat with environmental conditions. Our findings suggest the redox state of PSII strongly influences the relationship between SIF PSII and V cmax and J max .

59 BASIC BIOLOGICAL SCIENCES↗

Advancements in Constitutive Model Calibration: Leveraging the Power of Full‐Field DIC Measurements and In Situ Load Path Selection for Reliable Parameter Inference

Accurate material characterization and model calibration are essential for computationally supported high-consequence engineering decisions. Historically, characterization and calibration methods (1) use simplified test specimen geometries and global data, (2) cannot guarantee that sufficient characterization data are collected for a specific model of interest, (3) use deterministic methods that provide best-fit parameter values with no uncertainty quantification, and (4) are sequential, inflexible, and time-consuming. This work brings together several recent advancements into an improved workflow called interlaced characterization and calibration (ICC) that advances the state-of-the-art in constitutive model calibration. The ICC paradigm (1) employs tools to efficiently use full-field data to calibrate high-fidelity material models, (2) aligns the data needed with the data collected by adopting an optimal experimental design protocol, (3) quantifies parameter uncertainty through Bayesian inference and (4) incorporates these advancements into a quasi real-time feedback loop. The ICC framework is demonstrated here on the calibration of a material model using simulated full-field data for an aluminium cruciform specimen being deformed biaxially. The cruciform is actively driven through the myopically preferred load path using Bayesian optimal experimental design, which selects load steps that yield the maximum expected information gain (EIG). Principal component analysis (PCA) is performed on the model predictions of full-field displacements, and fast surrogate models are built to approximate the input-output relationships of the expensive finite element model. Furthermore, the tools developed and demonstrated here show that high-fidelity constitutive models can be efficiently and reliably calibrated with quantified uncertainty, thus supporting credible decision-making and potentially increasing the agility of solid mechanics modelling by enabling utilization of computational simulations at earlier stages of the design cycle.

Bayesian optimal experimental design↗

Exceptionally stable preindustrial sea level inferred from the western Mediterranean Sea

An accurate record of preindustrial (pre-1900 CE) sea level is necessary to contextualize modern global mean sea level (GMSL) rise with respect to natural variability. Precisely dated phreatic overgrowths on speleothems (POS) provide detailed rates of Late Holocene sea-level rise in Mallorca. Statistical analysis indicates that sea level rose locally by 0.12 to 0.31 m (95% confidence) from 3.26 to 2.84 thousand years (ka) ago (2σ) and remained within 0.08 m (95% confidence) of preindustrial levels from 2.84 ka to 1900 CE. This sea-level history is consistent with glacial isostatic adjustment models adopting relatively weak upper mantle viscosities of ~10 20 Pa s. There is virtual certainty (>0.999 probability) that the average GMSL rise since 1900 CE has exceeded even the high average rate of sea-level rise between 3.26 and 2.84 ka inferred from the POS record. We conclude that modern GMSL rise is anomalous relative to any natural variability in ice volumes over the past 4000 years.

54 ENVIRONMENTAL SCIENCES↗

AIVT: Inference of turbulent thermal convection from measured 3D velocity data by physics-informed Kolmogorov-Arnold networks

We propose the artificial intelligence velocimetry-thermometry (AIVT) method to reconstruct a continuous and differentiable representation of the temperature and velocity in turbulent convection from measured three-dimensional (3D) velocity data. AIVT is based on physics-informed Kolmogorov-Arnold networks and trained by optimizing a loss function that minimizes residuals of the velocity data, boundary conditions, and governing equations. We apply AIVT to a set of simultaneously measured 3D temperature and velocity data of Rayleigh-Bénard convection, obtained by combining particle image thermometry and Lagrangian particle tracking. This enables us to directly compare machine learning results to true volumetric, simultaneous temperature and velocity measurements. We demonstrate that AIVT can reconstruct and infer continuous, instantaneous velocity and temperature fields and their gradients from sparse experimental data at a high resolution, providing an additional approach for understanding thermal turbulence.

Science & Technology - Other Topics↗

Neural simulation-based inference of the Higgs trilinear self-coupling via off-shell Higgs production

One of the forthcoming major challenges in particle physics is the experimental determination of the Higgs trilinear self-coupling. While efforts have largely focused on on-shell double- and single-Higgs production in proton-proton collisions, off-shell Higgs production has also been proposed as a valuable complementary probe. In this article, we design a hybrid neural simulation-based inference (NSBI) approach to construct a likelihood of the Higgs signal incorporating modifications from the Standard Model effective field theory (SMEFT), relevant background processes, and quantum interference effects. It leverages the training efficiency of matrix-element-enhanced techniques, which are vital for robust SMEFT applications, while also incorporating the practical advantages of classification-based methods for effective background estimates. We demonstrate that our NSBI approach achieves sensitivity close to the theoretical optimum and provide expected constraints for the high-luminosity upgrade of the Large Hadron Collider. While we primarily concentrate on the Higgs trilinear self-coupling, we also consider constraints on other SMEFT operators that affect off-shell Higgs production.

Ghosh, Aishik [Univ. of California, Irvine, CA (Un↗

Improving inference with matrix elements and machine learning

Particle physics processes bring together a high-energy amplitude described by quantum field theory and nonperturbative effects and detector interactions described by complex computer simulations. We review some recently developed multivariate inference techniques that leverage this structure and combine matrix-element information with machine learning. Automated by the MadMiner package, the new techniques have been applied to multiple problems in particle physics, allowing for stronger limits than traditional analysis methods and showing their potential to improve the sensitivity of the LHC legacy measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

O3BNN: An Out-Of-Order Architecture for High-Performance Binarized Neural Network Inference with Fine-Grained Pruning

In this work, we demonstrate that the highly-condensed BNN model can be shrunk significantly further by dynamically pruning irregular redundant edges. Based on two new observations on BNN-specific properties, our out-of-order (OoO) architecture – O3BNN, can curtail remaining edge evaluation in cases where the binary output of a neuron can be determined early. Similar to Instruction-Level-Parallelism (ILP), these fine-grained, irregular, runtime pruning opportunities are traditionally presumed to be di cult to exploit. We evaluate our design on an FPGA platform using three well-known networks, including VggNet-16, AlexNet for ImageNet, and a VGG-like network for Cifar-10. Results show that our out-of-order approach can prune 27%, 16%, and 42% of the operations for the three networks respectively, without any accuracy loss, leading to at least 1.7×, 1.5×, and 2.1× speedups over state-of-the-art implementations on FPGA/GPU/CPU BNN implementations. Our approach is inference runtime pruning, so no retrain or fine-tuning is needed. We demonstrate our design on an FPGA platform. However, this is only for showcasing the method; the approach does not rely on any FPGA-specific features and can thus be adopted by other devices as well.

Geng, Tong↗

Dataset: Inferring Thermal Comfort using Body Shape Information Utilizing Depth Sensors

Thermal comfort is very important for well-being and productivity of building occupants. It has been shown that body shape is a useful feature to determine thermal comfort of individuals [2]. It is because, the heat dissipation rate of individuals depends on the body surface area. As a result, a tall and skinny person can tolerate higher room temperature than a rounded body shape person [5]. In order to test this hypothesis, we performed a year-long experiment in 2017, where we recruited 77 participants and put each of them in a thermally controlled conference room in CMU for 3 hours and recorded their subjective responses regarding thermal comfort at different temperature ranging from 60°F to 80°F. In addition, we collected depth data of individuals using a vertically mounted Microsoft Kinect for XBOX One at the entrance of the conference room to capture their body shape. We also collected biometric features (e.g., Galvanic Skin Response (GSR), skin temperature) using a Microsoft Health Band worn by the subjects. The resulting dataset provides rich information regarding how different features can be used to infer thermal comfort of the individuals.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗