Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “consensus algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Metrics and Methods for Radiation Detection Algorithm Characterization for Nuclear/Radiological Source Search

This report presents a series of recommendations for data to train and evaluate radiation detection algorithms and performance metrics to evaluate these algorithms. These recommendations were formed through a community consensus approach through the Detection Radiation Algorithms Group (DRAG), a multi-institution collaboration spanning eight Department of Energy laboratories and John Hopkins Applied Physics Laboratory. This report includes recommendations on background data variability, and metrics to quantify variability, sources and shielding configurations to include in data collection campaigns and detector response variability. In addition, this report describes several anomaly detection and identification algorithms and recommends metrics to report their performance. Finally, this report ends with a discussion on machine learning algorithms.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Equitable Transactive Market Design to Coordinate Networked Microgrids with Mixed Ownership

This paper presents an inter-microgrid peer-topeer (P2P) transactive market mechanism to support the coordination of multiple microgrids in a mixed-ownership environment, while enabling prosumers to actively participate in the market to get their own benefits. An equitable P2P transactive energy market is designed, in which the energy burden to customers within microgrids is fairly distributed by leveraging a peer-to-peer communication network among MG owners and the distribution system operator (DSO). To reach the market settlement, a consensus-based distributed optimization algorithm is introduced to enable each MG owner and the DSO to distributedly determine the cleared price which is compensated by a household-income-based discount factor to encourage the energy-burden equity among customers in networked microgrids service territory. Numerical results on the modified 123 node test feeder including 6 microgrids are used to demonstrate the operation of the introduced equitable transactive energy market.

Vu, Thanh Long↗
DOE OSTI2023

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. Furthermore, we observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

FY21 Progress Report: SRNL Analysis of ICCWR LCM and WAMS data for Corrosion and Cracking

The development of algorithms for machine learning and data analysis for the 3013 Surveillance Program is a collaborative effort by the Savannah River National Laboratory (SRNL) and the University of South Carolina (USC). For corrosion detection, Laser Confocal Microscope (LCM) or Wide Area 3D Measurement System (WAMS) data is extracted from large binary files, with software written to convert the data to physical attributes (e.g., height, color and grayscale values; all as functions of a location in a plane projection). A user-friendly Matlab Graphical User Interface (GUI) that reads data from either LCM or WAMS files was developed to integrate input data with software developed for processing and evaluation. The GUI can selectively download binary data, interrogate data attributes, label data, flag significant features, execute Machine Learning (ML) algorithms, output parameters for trained ML algorithms, report ML model accuracy with respect to labeled data, and generate graphical representations for various analyses. Features can be called out by user-specified thresholds, manual labeling or machine learning algorithms when they have been completed. The ability to rapidly label data is important because of the volume of data required for training machine learning algorithms. The GUI has the flexibility to allow addition of improved ML algorithms, methods for data visualization, and statistical computations. Statistical analyses via the GUI include areas of pits within a defined range of pit depths, correlations between Red-Green-Blue (RGB) or grayscale intensity and relative surface height, covariances between values associated with features, and feature histograms. The development of supervised machine learning algorithms, however, has been hindered by a lack of training data. The machine learning algorithms for crack identification are being refined but require improvements to the true positive rate for crack detection. This shortcoming is an artifact of the limited training data currently available, perhaps more so than the structure of the neural networks. At present, the best results are had from a consensus over an ensemble of randomly generated Deep Neural Network (DNN) or Convolutional Neural Network (CNN) algorithms. Although the consensus accuracy method has yielded optimum true positive and true negative rates in excess of 80%, additional validation testing is necessary. In addition to the suite of LCM data that was initially used, and which represents the majority of the work presented in this report, WAMS image data was also reviewed at a preliminary level. The review included a comparison between image resolution and dynamic range for each method. WAMS (ZON file) image data was found to have a pixel pitch of 3.69μm compared to 1 μm for the LCM (vk4 file) data, which implies a lower resolution for the WAMS images. Conversely, the ratio of dynamic range of the WAMS data to the LCM data was approximately 41:20 for height data, suggesting that information from WAMS should more accurately determine the depth of pits. At present, the significance of the greater dynamic range of the WAMS data relative to the LCM data has not yet been evaluated.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

CONSTAX2: improved taxonomic classification of environmental DNA markers

Abstract Summary CONSTAX—the CONSensus TAXonomy classifier—was developed for accurate and reproducible taxonomic annotation of fungal rDNA amplicon sequences and is based upon a consensus approach of RDP, SINTAX and UTAX algorithms. CONSTAX2 extends these features to classify prokaryotes as well as eukaryotes and incorporates BLAST-based classifiers to reduce classification errors. Additionally, CONSTAX2 implements a conda-installable command-line tool with improved classification metrics, faster training, multithreading support, capacity to incorporate external taxonomic databases and new isolate matching and high-level taxonomy tools, replete with documentation and example tutorials. Availability and implementation CONSTAX2 is available at https://github.com/liberjul/CONSTAXv2, and is packaged for Linux and MacOS from Bioconda with use under the MIT License. A tutorial and documentation are available at https://constax.readthedocs.io/en/latest/. Data and scripts associated with the manuscript are available at https://github.com/liberjul/CONSTAXv2_ms_code. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Distributed ADMM Using Private Blockchain for Power Flow Optimization in Distribution Network With Coupled and Mixed-Integer Constraints

The optimization problem for scheduling distributed energy resources (DERs) and battery energy storage systems (BESS) integrated with the power grid is important to minimize energy consumption from conventional sources in response to demand. Conventionally this optimization problem is solved in a centralized manner, limiting the size of the problem that can be solved and creating a high communication overhead because all the data is transferred to the central controller. These limitations are addressed by the proposed distributed consensus-based alternating direction method of multiplier (DC-ADMM) optimization algorithm, which decomposes the optimization problem into subproblems with private cost function and constraints. The distribution feeder is partitioned into low coupling subnetworks/regions, which solves the private subproblem locally and exchanges information with the neighboring regions to reach consensus. The relaxation strategy is employed for mixed-integer and coupled constraints introduced in the optimal power flow (OPF) problem by stationary and transportable BESS because DC-ADMM convergence is only guaranteed for strict convex problems. The information exchange and synchronization between subnetworks/regions are vital for distributed optimization. In this work, both of these aspects are addressed by the blockchain. The smart contract deployed on the blockchain network acts as a mediator for secure data exchange and synchronization in distributed computation. The blockchain-based distributed optimization problem’s effectiveness is tested for a 0.5-MW laboratory microgrid for one hour ahead and day-ahead for the IEEE 123-bus and EPRI J1 test feeders, and results are compared with a centralized solution.

25 ENERGY STORAGE↗
DOE OSTI2020

Consensus and Disagreement in Atmospheric River Detection: ARTMIP Global Catalogues

Atmospheric rivers (ARs) constitute an important mechanism for water vapor transport, but research on their characteristics and impacts has relied on a diverse assortment of detection methodologies, complicating comparisons. The AR Tracking Method Intercomparison Project (ARTMIP) provides a platform for comparing such methodologies, but analysis of ARTMIP catalogues has heretofore focused primarily on specific regions. Here we investigate ARs as detected by an ensemble of algorithms with global coverage. We find that the frequency of occurrence of the majority–consensus ARs produces a robust distribution, featuring five hot spots over the extratropical oceans, against which we compare individual algorithm results. We further explore the underlying similarities and differences via two case studies of AR evolution. The dominant source of disagreement between detection methodologies globally consists of detections (or lack thereof) of weak features, and the algorithms otherwise tend to agree remarkably well on the footprints of ARs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Super Resolving Unrolled Neural Networks for Remote Sensing

In remote sensing systems, the capabilities of the system are constrained by the complex interactions between size, weight, and power (SWAP) of potential designs. In electro-optical (EO) systems, examples of these critical parameters include the system’s sensitivity and resolution. Those parameters can be increased by ever larger optical apertures and focal planes but at the cost of more SWAP. Multi-image super resolution (MISR) techniques allow resolution to be enhanced via computation rather than more sophisticated optical hardware. These algorithms combine multiple images together into a single, higher resolution image, trading temporal resolution and computation for spatial resolution. Fielded MISR techniques, such as Drizzle, can require several hundred images to create a single super resolved image, implying reduced temporal resolution, increased data acquisition load, and limiting mission applications. Iterative techniques, such as model-based image reconstruction and compressive sensing, have been shown to create super resolved images using fewer images than Drizzle. They do this by posing an optimization problem that balances accuracy between a highly accurate physical model and an image model. In the case of super resolution, the physical model is defined by the relation between low resolution input images and the desired high resolution output image. The image model encodes some assumptions about the super resolved image. These assumptions are meant to suppress reconstruction artifacts that arise due to deterministic physical model error, stochastic measurement noise, and potential undersampling. In practice, the performance of iterative methods are limited by imaging models compatible with optimization. Deep learning-based methods can effectively learn image models of arbitrary complexity, but lack the theoretical explainability and robustness of iterative techniques. Consensus equilibrium (CE) generalizes the iterative techniques beyond optimization, enabling blackbox algorithms such as traditional and neural image denoisers to be used as the image model. CE-based approaches retain much of the explainability and robustness of iterative techniques while allowing the expressiveness of machine learning image models to be used. Additionally, by unrolling iterations of CE with an embedded image denoiser, the image denoiser can be further trained and specialized to the specific application with potentially higher quality reconstructions. Under this project, we demonstrated the feasibility of training an unrolled neural network based upon CE. While we didn’t train one, we showed that the CE process is differentiable and its gradient can be tractably computed. We also explored the usage of a variants of CE akin to generative neural works. Most importantly, we applied the CE framework to a number of problems including non-blind deconvolution, upsampling, single-image super resolution, MISR, event-based sensing, and saturated deconvolution. Our MISR prototype creates high quality reconstructions with an order of magnitude fewer images than previous approaches and, critically, produces these reconstructions fast enough for practical usage.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Blockchain Enabled Intelligence of Federated Systems (BELIEFS): An attack-tolerant trustable distributed intelligence paradigm

In this article, a Blockchain Enabled Intelligence of Federated Systems (BELIEFS) is proposed to conduct cooperative control for the multi-regional large-scale power system with a multi-agents system (MAS). By establishing a two levels blockchain, each regional AI agent can simultaneously manage intra-regional controllers and cooperate with other AI agents. Under the consensus mechanism, the agents, which respectively conducted distributed deep reinforcement learning (DDRL) algorithm in multi-regions, can have the tolerant capability of malicious attacks in their training process. The demonstration of the proposed approach is within a multi-regional large-scale interconnected power system. Under the mode of "centralized dispatching and hierarchical management", this article aims to definite a mathematical model to deal with the control problem of the power systems. With the comparison experiments, the effectiveness and efficiency of our proposed method in the training process are verified. In addition, malicious attacks are set on the main chain and shard chains to verify the attack-tolerant capability. We expect that such approach and results can suggest a new paradigm of attack-tolerant trustable distributed AI deployment.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Secure Learning Control Strategy via Dynamic Camouflaging for Unknown Dynamical Systems under Attacks

This paper presents a secure reinforcement learning (RL) based control method for unknown linear time-invariant cyber-physical systems (CPSs) that are subjected to compositional attacks such as eavesdropping and covert attack. We consider the attack scenario where the attacker learns about the dynamic model during the exploration phase of the learning conducted by the designer to learn a linear quadratic regulator (LQR), and thereafter, use such information to conduct a covert attack on the dynamic system, which we refer to as doubly learning-based control and attack (DLCA) framework. We propose a dynamic camouflaging based attack-resilient reinforcement learning (ARRL) algorithm which can learn the desired optimal controller for the dynamic system, and at the same time, can inject sufficient misinformation in the estimation of system dynamics by the attacker. The algorithm is accompanied by theoretical guarantees and extensive numerical experiments on a consensus multi-agent system and on a benchmark power grid model.

Mukherjee, Sayak↗
DOE OSTI2023

Collaboration and Negotiation

Collaboration and Negotiation is a critical high-level function of an Autonomous Intelligent Cyber-Defense Agent (AICA) that enables communication among agents, central cyber C2, and human operators. Maintaining the Confidentiality, Integrity, and Availability (CIA) triad while achieving mission goals requires stealthy AICA agents to exercise: 1) minimal communication as needed for avoiding detection, 2) verification of information received with possibly limited resources, and 3) active learning during operations to address dynamic conditions. Moreover, negotiations to jointly identify and execute a Course of Action (COA) solution will require building consensus under distributed and/or decentralized multiagent settings with information uncertainties. This chapter presents algorithmic approaches for enabling the collaboration and negotiation function. Strengths and limitations of potential techniques are identified, and a representative example is illustrated. Recommendations for future development are also discussed.

Chatterjee, Samrat↗
DOE OSTI2022

A Hierarchical Framework for CO2 Storage Capacity in Deep Saline Aquifer Formations

Carbon dioxide (CO 2 ) storage in deep saline aquifers is a vital option for CO 2 mitigation at a large scale. Determining storage capacity is one of the crucial steps toward large-scale deployment of CO 2 storage. Results of capacity assessments tend toward a consensus that sufficient resources are available in saline aquifers in many parts of the world. However, current CO 2 capacity assessments involve significant inconsistencies and uncertainties caused by various technical assumptions, storage mechanisms considered, algorithms, and data types and resolutions. Furthermore, other constraint factors (such as techno-economic features, site suitability, risk, regulation, social-economic situation, and policies) significantly affect the storage capacity assessment results. Consequently, a consensus capacity classification system and assessment method should be capable of classifying the capacity type or even more related uncertainties. We present a hierarchical framework of CO 2 capacity to define the capacity types based on the various factors, algorithms, and datasets. Finally, a review of onshore CO 2 aquifer storage capacity assessments in China is presented as examples to illustrate the feasibility of the proposed hierarchical framework.

58 GEOSCIENCES↗
DOE OSTI2025

Adversarial Binaries: AI-guided Instrumentation Methods for Malware Detection Evasion

Adversarial binaries are executable files that have been altered without loss of function by an AI agent in order to deceive malware detection systems. Progress in this emergent vein of research has been constrained by the complex and rigid structure of executable files. Although prior work has demonstrated that these binaries deceive a variety of malware classification models which rely on disparate feature sets, a consensus as to the best approach has not been reached, either in terms of the optimization algorithms or the instrumentation methods. Furthermore, although inconsistencies in the data sets, target classifiers, and functionality verification methods make head-to-head comparisons difficult, here we extract lessons learned and make recommendations for future research.

malware obfuscation↗
DOE OSTI2020

Evaluation of probabilistic photometric redshift estimation approaches for The Rubin Observatory Legacy Survey of Space and Time (LSST)

Many scientific investigations of photometric galaxy surveys require redshift estimates, whose uncertainty properties are best encapsulated by photometric redshift (photo-z) posterior probability density functions (PDFs). A plethora of photo-z PDF estimation methodologies abound, producing discrepant results with no consensus on a preferred approach. We present the results of a comprehensive experiment comparing twelve photo-z algorithms applied to mock data produced forLarge Synoptic Survey Telescope The Rubin Observatory Legacy Survey of Space and Time (lsst) Dark Energy Science Collaboration (desc). By supplying perfect prior information, in the form of the complete template library and a representative training set as inputs to each code, we demonstrate the impact of the assumptions underlying each technique on the output photo-z PDFs. In the absence of a notion of true, unbiased photo-z PDFs, we evaluate and interpret multiple metrics of the ensemble properties of the derived photo-z PDFs as well as traditional reductions to photo-z point estimates. Additionally, we report systematic biases and overall over/under-breadth of the photo-z PDFs of many popular codes, which may indicate avenues for improvement in the algorithms or implementations. Furthermore, we raise attention to the limitations of established metrics for assessing photo-z PDF accuracy; though we identify the conditional density estimate (CDE) loss as a promising metric of photo-z PDF performance in the case where true redshifts are available but true photo-z PDFs are not, we emphasize the need for science-specific performance metrics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

From Points to Planes: A Workflow for Converting Three‐Dimensional Point Cloud Data Into Discrete Fracture Network Flow and Transport Models

We present the Point cLoud Algorithm for NEtwork Extraction of Discrete Fracture Networks (PLANE-DFN), a point cloud–based algorithm for automatic fracture network extraction designed to support discrete fracture network (DFN) modeling workflows. PLANE-DFN segments three-dimensional fracture planes from raw point cloud data using RANdom SAmple Consensus coupled with statistical outlier removal and density-based clustering to isolate individual fracture features. Each candidate plane is constrained against site-specific structural constraints based on strike and dip. After segmentation, each fracture is converted into a 2-D convex polygon suitable for meshing and simulation. The PLANE-DFN algorithm is validated by comparing geometric and flow and transport data against data from dfnWorks simulations with ensembles of plane-fit networks. We find that the flow and transport in plane-fit networks are comparable to dfnWorks-generated networks when realistic network geometry is maintained. The PLANE-DFN algorithm provides an automated and streamlined workflow to transform point clouds of data into DFN network geometry.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Machine learning models for rat multigeneration reproductive toxicity prediction

Reproductive toxicity is one of the prominent endpoints in the risk assessment of environmental and industrial chemicals. Due to the complexity of the reproductive system, traditional reproductive toxicity testing in animals, especially guideline multigeneration reproductive toxicity studies, take a long time and are expensive. Therefore, machine learning, as a promising alternative approach, should be considered when evaluating the reproductive toxicity of chemicals. We curated rat multigeneration reproductive toxicity testing data of 275 chemicals from ToxRefDB (Toxicity Reference Database) and developed predictive models using seven machine learning algorithms (decision tree, decision forest, random forest, k-nearest neighbors, support vector machine, linear discriminant analysis, and logistic regression). A consensus model was built based on the seven individual models. An external validation set was curated from the COSMOS database and the literature. The performances of individual and consensus models were evaluated using 500 iterations of 5-fold cross-validations and the external validation data set. The balanced accuracy of the models ranged from 58% to 65% in the 5-fold cross-validations and 45%–61% in the external validations. Prediction confidence analysis was conducted to provide additional information for more appropriate applications of the developed models. The impact of our findings is in increasing confidence in machine learning models. We demonstrate the importance of using consensus models for harnessing the benefits of multiple machine learning models (i.e., using redundant systems to check validity of outcomes). While we continue to build upon the models to better characterize weak toxicants, there is current utility in saving resources by being able to screen out strong reproductive toxicants before investing in vivo testing. The modeling approach (machine learning models) is offered for assessing the rat multigeneration reproductive toxicity of chemicals. Our results suggest that machine learning may be a promising alternative approach to evaluate the potential reproductive toxicity of chemicals.

consensus model↗
DOE OSTI2021

A Consensus Equilibrium Approach for 3-D Land Seismic Shots Recovery

Physical and budget constraints often result in inadequate sampling for accurate subsurface imaging. Preprocessing approaches, such as missing trace interpolation, are typically employed to enhance seismic data in such cases. The compressed sensing (CS) framework has been applied for modeling missing seismic data, which is estimated by sparsity-based computational algorithms. While existing work mainly focuses on recovering missing traces resulting from receiver subsampling, source subsampling has greater economical advantages, as sources are more expensive than receivers. Moreover, stronger image models different from sparsity have not been explored for source recovery. This work presents a consensus equilibrium (CE) approach to recover missing seismic shots, which enables to incorporate various regularization operators modeling different data priors. Here, simulation results from a real 3-D land seismic dataset demonstrate that the CE approach provides more accurate estimations of the linear and hyperbolic events in the recovered shots, compared with pure sparsity-based reconstructions.

58 GEOSCIENCES↗