Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “randomized algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Further adoption of conservation tillage can increase maize yields in the western US Corn Belt

Conservation tillage can reduce soil erosion, increase soil health, and decrease labor and fuel input costs. Despite these benefits, potential yield impacts remain an important concern for farmers considering adoption. Previous research suggests that conservation tillage is likely to have the largest yield benefits in more arid conditions, but a lack of field-level analyses across climatic, management and soil conditions limits confidence in such predictions. Satellite imagery provides the opportunity to monitor agricultural lands at sub-field resolution across large spatial scales and wide environmental gradients. Here we investigate the maize yield impacts of conservation tillage in the semi-arid western US Corn Belt, using sub-field resolution datasets on tillage practices and crop yields derived from satellite data spanning four states (Nebraska, Kansas, South Dakota, and North Dakota) between 2008 and 2020. On these datasets, we estimate heterogenous yield outcomes for several thousand maize fields across gradients in climate, soil quality and irrigation status by using a causal forests analysis, an adaptation of the random forests machine-learning algorithm for causal inference on observational data. We find that long-term adoption of conservation tillage increased rainfed maize yields by an average of 9.9% in the region. Impacts on irrigated yields were small and not statistically significant. These results, along with an analysis of variables related to greater than average yield benefits, indicate that improved water infiltration and retention are the primary reasons for conservation tillage benefits. Despite yield benefits, many fields estimated to see increased yields under long term low till have not adopted the practice. Therefore, we identify specific counties likely to benefit most from increased levels of adoption. Our results strengthen the understanding of the impacts of conservation agriculture on crop yields and help define environments and counties most likely to benefit from conservation tillage.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

EcoPLOT: dynamic analysis of biogeochemical data

Motivation: We have created EcoPLOT (parameterized linkage of omics-driven technologies), a web-app for the dynamic, interactive analysis of biogeochemical datasets that combines state-of-the-art analysis tools to statistically and graphically explore environmental, geochemical and microbiome datasets. Using the iterative random forest, a machine learning algorithm, EcoPLOT allows for the de novo discovery of drivers which exhibit significant impact on plant, microbial or soil dynamics. Availability and implementation: EcoPLOT is built entirely within the R language. It can be accessed through any system where R is installed, including Windows, Mac and most Linux systems. EcoPLOT is free to use and can be accessed at https://github.com/cdsanchez18/EcoPLOT.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Methods for the robust computation of the long-period seismic spectrum of broad-band arrays

SUMMARY We describe array methods to search for low signal-to-noise ratio (SNR) signals in long-period seismic data using Fourier analysis. This is motivated by published results that find evidence of solar free oscillations in the Earth's seismic hum. Previous work used data from only one station. In this paper, we describe methods for computing spectra from array data. Arrays reduce noise level through averaging and provide redundancy that we use to distinguish coherent signal from a random background. We describe two algorithms for calculating a robust spectrum from seismic arrays, an algorithm that automatically removes impulsive transient signals from data, a jackknife method for estimating the variance of the spectrum, and a method for assessing the significance of an entire spectral band. We show examples of their application to data recorded by the Homestake Mine 3-D array in Lead, SD and the Piñon Flats PY array. These are two of the quietest small aperture arrays ever deployed in North America. The underground Homestake data has exceptionally low noise, and the borehole sensors of the PY array also have very low noise, making these arrays well suited to finding very weak signals. We find that our methods remove transient signals effectively from the data so that even low-SNR signals in the seismic background can be found and tested. Additionally, we find that the jackknife variance estimate is comparable to the noise floor, and we present initial evidence for solar g-modes in our data through the T2 test, a multivariate generalization of Student's t-test.

Caton, Ross C.↗
DOE OSTI2019

Application of convolutional neural networks for stellar spectral classification

ABSTRACT Due to the ever-expanding volume of observed spectroscopic data from surveys such as SDSS and LAMOST, it has become important to apply artificial intelligence (AI) techniques for analysing stellar spectra to solve spectral classification and regression problems like the determination of stellar atmospheric parameters Teff, $\rm {\log g}$, and [Fe/H]. We propose an automated approach for the classification of stellar spectra in the optical region using convolutional neural networks (CNNs). Traditional machine learning (ML) methods with ‘shallow’ architecture (usually up to two hidden layers) have been trained for these purposes in the past. However, deep learning methods with a larger number of hidden layers allow the use of finer details in the spectrum which results in improved accuracy and better generalization. Studying finer spectral signatures also enables us to determine accurate differential stellar parameters and find rare objects. We examine various machine and deep learning algorithms like artificial neural networks, Random Forest, and CNN to classify stellar spectra using the Jacoby Atlas, ELODIE, and MILES spectral libraries as training samples. We test the performance of the trained networks on the Indo-U.S. Library of Coudé Feed Stellar Spectra (CFLIB). We show that using CNNs, we are able to lower the error up to 1.23 spectral subclasses as compared to that of two subclasses achieved in the past studies with ML approach. We further apply the trained model to classify stellar spectra retrieved from the SDSS data base with SNR > 20.

Sharma, Kaushal↗
DOE OSTI2023

Stochastic quantum Krylov protocol with double-factorized Hamiltonians

Here we propose a class of randomized quantum Krylov diagonalization (rQKD) algorithms capable of solving the eigenstate estimation problem with modest quantum resource requirements. Compared to previous real-time evolution quantum Krylov subspace methods, our approach expresses the time evolution operator e –i$\widehat{H}$$\tau$ as a linear combination of unitaries and subsequently uses a stochastic sampling procedure to reduce circuit depth requirements. While our methodology applies to any Hamiltonian with fast-forwardable subcomponents, we focus on its application to the explicitly double-factorized electronic-structure Hamiltonian. To demonstrate the potential of the proposed rQKD algorithm on near-term quantum devices, we provide numerical benchmarks for a variety of molecular systems with circuit-based state-vector simulators including the effects of sampling noise, achieving ground-state energy errors of less than 1 kcal mol -1 with circuit depths orders of magnitude shallower than those required for low-rank deterministic Trotter-Suzuki decompositions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Composite Qdrift-product formulas for quantum and classical simulations in real and imaginary time

Recent study has shown that it can be advantageous to implement a composite channel that partitions the Hamiltonian H for a given simulation problem into subsets A and B such that H = A + B , where the terms in A are simulated with a Trotter-Suzuki channel and the B terms are randomly sampled via the Qdrift algorithm. Here we extend Qdrift and composite product formulas to imaginary time, formulating candidate classical algorithms for quantum Monte Carlo calculations. We upper bound the induced Schatten- 1 → 1 norm on both imaginary-time Qdrift and composite channels. Another recent result demonstrated that simulations of lattice Hamiltonians containing geometrically local interactions can be improved using a Lieb-Robinson argument to decompose H into subsets that contain only terms supported on that subset of the lattice. Here, we provide a quantum algorithm by unifying this result with the composite approach into “local composite channels” and we upper bound the diamond distance. We provide exact numerical simulations of algorithmic cost by counting the number of gates of the form e − i H j t and e − H j β to meet a certain error tolerance ε . In doing so, we optimize the partitioning into sets A and B using gradient boosted tree models from machine learning. These numerical studies are important given that product formulas have been historically known to outperform analytic upper bounds. We show constant factor advantages for a variety of interesting Hamiltonians, the maximum of which is a ≈ 20 -fold speedup that occurs in the simulation of Jellium. Published by the American Physical Society 2024

Pocrnic, Matthew (ORCID:0000000203089376)↗
DOE OSTI2022

Machine learning models for rat multigeneration reproductive toxicity prediction

Reproductive toxicity is one of the prominent endpoints in the risk assessment of environmental and industrial chemicals. Due to the complexity of the reproductive system, traditional reproductive toxicity testing in animals, especially guideline multigeneration reproductive toxicity studies, take a long time and are expensive. Therefore, machine learning, as a promising alternative approach, should be considered when evaluating the reproductive toxicity of chemicals. We curated rat multigeneration reproductive toxicity testing data of 275 chemicals from ToxRefDB (Toxicity Reference Database) and developed predictive models using seven machine learning algorithms (decision tree, decision forest, random forest, k-nearest neighbors, support vector machine, linear discriminant analysis, and logistic regression). A consensus model was built based on the seven individual models. An external validation set was curated from the COSMOS database and the literature. The performances of individual and consensus models were evaluated using 500 iterations of 5-fold cross-validations and the external validation data set. The balanced accuracy of the models ranged from 58% to 65% in the 5-fold cross-validations and 45%–61% in the external validations. Prediction confidence analysis was conducted to provide additional information for more appropriate applications of the developed models. The impact of our findings is in increasing confidence in machine learning models. We demonstrate the importance of using consensus models for harnessing the benefits of multiple machine learning models (i.e., using redundant systems to check validity of outcomes). While we continue to build upon the models to better characterize weak toxicants, there is current utility in saving resources by being able to screen out strong reproductive toxicants before investing in vivo testing. The modeling approach (machine learning models) is offered for assessing the rat multigeneration reproductive toxicity of chemicals. Our results suggest that machine learning may be a promising alternative approach to evaluate the potential reproductive toxicity of chemicals.

consensus model↗
DOE OSTI2021

Detecting Arsenic Contamination Using Satellite Imagery and Machine Learning

Arsenic, a potent carcinogen and neurotoxin, affects over 200 million people globally. Current detection methods are laborious, expensive, and unscalable, being difficult to implement in developing regions and during crises such as COVID-19. This study attempts to determine if a relationship exists between soil’s hyperspectral data and arsenic concentration using NASA’s Hyperion satellite. It is the first arsenic study to use satellite-based hyperspectral data and apply a classification approach. Four regression machine learning models are tested to determine this correlation in soil with bare land cover. Raw data are converted to reflectance, problematic atmospheric influences are removed, characteristic wavelengths are selected, and four noise reduction algorithms are tested. The combination of data augmentation, Genetic Algorithm, Second Derivative Transformation, and Random Forest regression (R 2 =0.840 and normalized root mean squared error (re-scaled to [0,1]) = 0.122) shows strong correlation, performing better than past models despite using noisier satellite data (versus lab-processed samples). Three binary classification machine learning models are then applied to identify high-risk shrub-covered regions in ten U.S. states, achieving strong accuracy (=0.693) and F1-score (=0.728). Overall, these results suggest that such a methodology is practical and can provide a sustainable alternative to arsenic contamination detection.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗
DOE OSTI2013

Quantum Statistical Testing of a QRNG Algorithm

We present the algorithmic design of a quantum random number generator, the subsequent synthesis of a physical design and its verification using quantum statistical testing. We also describe how quantum statistical testing can be used to diagnose channel noise in QKD protocols.

quantum information↗
DOE OSTI2015

Single-Pol Synthetic Aperture Radar Terrain Classification using Multiclass Confidence for One-Class Classifiers

Except in the most extreme conditions, Synthetic aperture radar (SAR) is a remote sensing technology that can operate day or night. A SAR can provide surveillance over a long time period by making multiple passes over a wide area. For object-based intelligence it is convenient to segment and classify the SAR images into objects that identify various terrains and man-made structures that we call “static features.” In this paper we introduce a novel SAR image product that captures how different regions decorrelate at different rates. Using superpixels and their first two moments we develop a series of one-class classification algorithms using a goodness-of-fit metric. P-value fusion is used to combine the results from different classes. We also show how to combine multiple one-class classifiers to get a confidence about a classification. This can be used by downstream algorithms such as a conditional random field to enforce spatial constraints.

Koch, Mark William↗
DOE OSTI2016

A computationally efficient parallel Levenberg-Marquardt algorithm for highly parameterized inverse model analyses

Inverse modeling seeks model parameters given a set of observations. However, for practical problems because the number of measurements is often large and the model parameters are also numerous, conventional methods for inverse modeling can be computationally expensive. We have developed a new, computationally-efficient parallel Levenberg-Marquardt method for solving inverse modeling problems with a highly parameterized model space. Levenberg-Marquardt methods require the solution of a linear system of equations which can be prohibitively expensive to compute for moderate to large-scale problems. Our novel method projects the original linear problem down to a Krylov subspace, such that the dimensionality of the problem can be significantly reduced. Furthermore, we store the Krylov subspace computed when using the first damping parameter and recycle the subspace for the subsequent damping parameters. The efficiency of our new inverse modeling algorithm is significantly improved using these computational techniques. We apply this new inverse modeling method to invert for random transmissivity fields in 2D and a random hydraulic conductivity field in 3D. Our algorithm is fast enough to solve for the distributed model parameters (transmissivity) in the model domain. The algorithm is coded in Julia and implemented in the MADS computational framework (http://mads.lanl.gov). By comparing with Levenberg-Marquardt methods using standard linear inversion techniques such as QR or SVD methods, our Levenberg-Marquardt method yields a speed-up ratio on the order of ~10 1 to ~10 2 in a multi-core computational environment. Furthermore, our new inverse modeling method is a powerful tool for characterizing subsurface heterogeneity for moderate- to large-scale problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Sparse Approximate Multifrontal Factorization with Butterfly Compression for High-Frequency Wave Equations

In this work, we present a fast and approximate multifrontal solver for large-scale sparse linear systems arising from finite-difference, finite-volume or finite-element discretization of high-frequency wave equations. The proposed solver leverages the butterfly algorithm and its hierarchical matrix extension for compressing and factorizing large frontal matrices via graph-distance guided entry evaluation or randomized matrix-vector multiplication-based schemes. Complexity analysis and numerical experiments demonstrate $\mathcal{O}(N\log^2 N)$ computation and $\mathcal{O}(N)$ memory complexity when applied to an $N\times N$ sparse system arising from 3D high-frequency Helmholtz and Maxwell problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Evaluating Optimization Strategies for Engine Simulations Using Machine Learning Emulators

This work evaluates different optimization algorithms for computational fluid dynamics (CFD) simulations of engine combustion. Due to the computational expense of CFD simulations, emulators built with machine learning algorithms were used as surrogates for the optimizers. Two types of emulators were used: a Gaussian process (GP) and a weighted variety of machine learning methods called SuperLearner (SL). The emulators were trained using a dataset of 2048 CFD simulations that were run concurrently on a supercomputer. The design of experiments (DOE) for the CFD runs was obtained by perturbing nine input parameters using a Monte-Carlo method. The CFD simulations were of a heavy duty engine running with a low octane gasoline-like fuel at a partially premixed compression ignition mode. Ten optimization algorithms were tested, including types typically used in research applications. Each optimizer was allowed 800 function evaluations and was randomly tested 100 times. The optimizers were evaluated for the median, minimum, and maximum merits obtained in the 100 attempts. Some optimizers required more sequential evaluations, thereby resulting in longer wall clock times to reach an optimum. The best performing optimization methods were particle swarm optimization (PSO), differential evolution (DE), GENOUD (an evolutionary algorithm), and micro-genetic algorithm (GA). These methods found a high median optimum as well as a reasonable minimum optimum of the 100 trials. Moreover, all of these methods were able to operate with less than 100 successive iterations, which reduced the wall clock time required in practice. Two methods were found to be effective but required a much larger number of successive iterations: the DIRECT and MALSCHAINS algorithms. A random search method that completed in a single iteration performed poorly in finding optimum designs but was included to illustrate the limitation of highly concurrent search methods. The last three methods, Nelder–Mead, bound optimization by quadratic approximation (BOBYQA), and constrained optimization by linear approximation (COBYLA), did not perform as well.

Probst, Daniel M.↗
DOE OSTI2014

Goal-oriented sensitivity analysis for lattice kinetic Monte Carlo simulations

In this paper we propose a new class of coupling methods for the sensitivity analysis of high dimensional stochastic systems and in particular for lattice Kinetic Monte Carlo (KMC). Sensitivity analysis for stochastic systems is typically based on approximating continuous derivatives with respect to model parameters by the mean value of samples from a finite difference scheme. Instead of using independent samples the proposed algorithm reduces the variance of the estimator by developing a strongly correlated-“coupled”- stochastic process for both the perturbed and unperturbed stochastic processes, defined in a common state space. The novelty of our construction is that the new coupled process depends on the targeted observables, e.g., coverage, Hamiltonian, spatial correlations, surface roughness, etc., hence we refer to the proposed method as goal-oriented sensitivity analysis. In particular, the rates of the coupled Continuous Time Markov Chain are obtained as solutions to a goal-oriented optimization problem, depending on the observable of interest, by considering the minimization functional of the corresponding variance. We show that this functional can be used as a diagnostic tool for the design and evaluation of different classes of couplings. Furthermore, the resulting KMC sensitivity algorithm has an easy implementation that is based on the Bortz–Kalos–Lebowitz algorithm's philosophy, where events are divided in classes depending on level sets of the observable of interest. Finally, we demonstrate in several examples including adsorption, desorption, and diffusion Kinetic Monte Carlo that for the same confidence interval and observable, the proposed goal-oriented algorithm can be two orders of magnitude faster than existing coupling algorithms for spatial KMC such as the Common Random Number approach. We also provide a complete implementation of the proposed sensitivity analysis algorithms, including various spatial KMC examples, in a supplementary MATLAB source code.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

Topology Estimation using Graphical Models in Multi-Phase Power Distribution Grids

Power distribution grids are structurally operated radially, such that energized lines form a collection of trees with a substation at the root of each tree. The operational topology may change from time to time, however tracking these changes, even though important for the distribution grid operation and control, is hindered by limited real-time monitoring. This work develops a learning framework to reconstruct the radial operational structure of the distribution grid from synchronized voltage measurements. To detect operational lines our learning algorithm uses conditional independence tests for continuous random variables that is applicable to a wide class of probability distributions and Gaussian injections in particular. Furthermore, our algorithm applies to the practical case of unbalanced three-phase power flow. The performance is validated on AC power flow simulations over three phase IEEE distribution grid test cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

Checking Questionable Entry of Personally Identifiable Information Encrypted by One-Way Hash Transformation

Background: As one of the several effective solutions for personal privacy protection, a global unique identifier (GUID) is linked with hash codes that are generated from combinations of personally identifiable information (PII) by a one-way hash algorithm. On the GUID server, no PII is permitted to be stored, and only GUID and hash codes are allowed. The quality of PII entry is critical to the GUID system. Objective: The goal of our study was to explore a method of checking questionable entry of PII in this context without using or sending any portion of PII while registering a subject. Methods: According to the principle of GUID system, all possible combination patterns of PII fields were analyzed and used to generate hash codes, which were stored on the GUID server. Based on the matching rules of the GUID system, an error-checking algorithm was developed using set theory to check PII entry errors. We selected 200,000 simulated individuals with randomly-planted errors to evaluate the proposed algorithm. These errors were placed in the required PII fields or optional PII fields. The performance of the proposed algorithm was also tested in the registering system of study subjects. Results: There are 127,700 error-planted subjects, of which 114,464 (89.64%) can still be identified as the previous one and remaining 13,236 (10.36%, 13,236/127,700) are discriminated as new subjects. As expected, 100% of nonidentified subjects had errors within the required PII fields. The possibility that a subject is identified is related to the count and the type of incorrect PII field. For all identified subjects, their errors can be found by the proposed algorithm. The scope of questionable PII fields is also associated with the count and the type of the incorrect PII field. The best situation is to precisely find the exact incorrect PII fields, and the worst situation is to shrink the questionable scope only to a set of 13 PII fields. In the application, the proposed algorithm can give a hint of questionable PII entry and perform as an effective tool. Conclusions: The GUID system has high error tolerance and may correctly identify and associate a subject even with few PII field errors. Correct data entry, especially required PII fields, is critical to avoiding false splits. In the context of one-way hash transformation, the questionable input of PII may be identified by applying set theory operators based on the hash codes. The count and the type of incorrect PII fields play an important role in identifying a subject and locating questionable PII fields.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Reconstructions from randomly generated longitudinal electron bunch profiles with Gaussian envelopes using the Gerchberg–Saxton algorithm

Knowledge of longitudinal electron bunch profiles is vital to optimize the performance of plasma wakefield accelerators and x-ray free electron laser linacs. Because of their importance to these novel applications, noninvasive frequency domain techniques are often employed to reconstruct longitudinal bunch profiles from coherent synchrotron, transition, or undulator radiation measurements. In this paper, we detail several common reconstruction techniques involving the Kramers–Kronig phase relationship and Gerchberg–Saxton algorithm. Additionally, through statistical analysis, we draw general conclusions about the accuracy of these reconstruction techniques and the most suitable candidate for reconstructing well-isolated longitudinal bunch profiles from spectroscopic data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2014

How to efficiently select an arbitrary Clifford group element

We give an algorithm which produces a unique element of the Clifford group on n qubits (C{sub n}) from an integer 0≤i<|C{sub n}| (the number of elements in the group). The algorithm involves O(n{sup 3}) operations and provides, in addition to a canonical mapping from the integers to group elements g, a factorization of g into a sequence of at most 4n symplectic transvections. The algorithm can be used to efficiently select random elements of C{sub n} which are often useful in quantum information theory and quantum computation. We also give an algorithm for the inverse map, indexing a group element in time O(n{sup 3})

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗