Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “unsupervised learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Ca-dimers, solvent layering, and dominant electrochemically active species in Ca(BH4)2 in THF

Abstract Divalent ions (Mg, Ca, and Zn) are being considered as competitive, safe, and earth-abundant alternatives to Li-ion electrochemistry, but present challenges for stable cycling due to undesirable interfacial phenomena. We explore the formation of electroactive species in the electrolyte Ca(BH 4 ) 2 ∣THF using molecular dynamics coupled with a continuum model of bulk and interfacial speciation. Free-energy analysis and unsupervised learning indicate a majority population of neutral Ca dimers and monomers with diverse molecular conformations and an order of magnitude lower concentration of the primary electroactive charged species – the monocation, $${\rm{CaBH}}_{4}^{+}$$ CaBH 4 + – produced via disproportionation of neutral complexes. Dense layering of THF molecules within ~1 nm of the electrode surface strongly modulates local electrolyte species populations. A dramatic increase in monocation population in this interfacial zone is induced at negative bias. We see no evidence for electrochemical activity of fully-solvated Ca 2+ . The consequences for performance are discussed in light of this molecular-scale insight.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Hydrogen-induced tunable remanent polarization in a perovskite nickelate

Materials with field-tunable polarization are of broad interest to condensed matter sciences and solid-state device technologies. Here, using hydrogen (H) donor doping, we modify the room temperature metallic phase of a perovskite nickelate NdNiO 3 into an insulating phase with both metastable dipolar polarization and space-charge polarization. We then demonstrate transient negative differential capacitance in thin film capacitors. The space-charge polarization caused by long-range movement and trapping of protons dominates when the electric field exceeds the threshold value. First-principles calculations suggest the polarization originates from the polar structure created by H doping. We find that polarization decays within ~1 second which is an interesting temporal regime for neuromorphic computing hardware design, and we implement the transient characteristics in a neural network to demonstrate unsupervised learning. These discoveries open new avenues for designing ferroelectric materials and electrets using light-ion doping.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Superconducting neural networks with disordered Josephson junction array synaptic networks and leaky integrate-and-fire loop neurons

Fully coupled randomly disordered recurrent superconducting networks with additional open-ended channels for inputs and outputs are considered the basis to introduce a new architecture to neuromorphic computing in this work. Various building blocks of such a network are designed around disordered array synaptic networks using superconducting devices and circuits as an example, while emphasizing that a similar architectural approach may be compatible with several other materials and devices. A multiply coupled (interconnected) disordered array of superconducting loops containing Josephson junctions [equivalent to superconducting quantum interference devices (SQUIDs)] forms the aforementioned collective synaptic network that forms a fully recurrent network together with compatible neuron-like elements and feedback loops, enabling unsupervised learning. This approach aims to take advantage of superior power efficiency, propagation speed, and synchronizability of a small world or a random network over an ordered/regular network. Additionally, it offers a significant factor of increase in scalability. Here, a compatible leaky integrate-and-fire neuron made of superconducting loops with Josephson junctions is presented, along with circuit components for feedback loops as needed to complete the recurrent network. Several of these individual disordered array neural networks can further be coupled together in a similarly disordered way to form a hierarchical architecture of recurrent neural networks that is often suggested as similar to a biological brain.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Constrained non-negative matrix factorization enabling real-time insights of in situ and high-throughput experiments

Non-negative matrix factorization (NMF) is an appealing class of methods for performing unsupervised learning on streaming spectral data, particularly in time-sensitive applications such as in situ characterization of materials. These methods seek to decompose a dataset into a small number of components and weights that can compactly represent the underlying signal while effectively reconstructing the observations with minimal error. However, canonical NMF methods have no underlying requirement that the reconstruction uses components or weights that are representative of the true physical processes. In this work, we demonstrate how constraining a subset of the NMF weights or components as rigid priors, provided as known or assumed values, can provide significant improvement in revealing true underlying phenomena. We present a PyTorch-based method for efficiently applying constrained NMF and demonstrate its application to several synthetic examples. Our implementation allows an expert researcher-in-the-loop to provide and dynamically adjust the constraints during a live experiment involving streaming spectral data. Such interactive priors allow researchers to specify known or identified independent components, as well as functional expectations about the mixing or transitions between the components. We further demonstrate the application of this method to measured synchrotron x-ray total scattering data from in situ beamline experiments. In such a context, constrained NMF can result in a more interpretive and scientifically relevant decomposition than canonical NMF or other decomposition techniques. As a result, the details of the method are provided, along with general guidance for employing constrained NMF in the extraction of critical information and insights during time-sensitive experimental applications.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Attention-based explainability for structure–property relationships

Machine learning methods are emerging as a universal paradigm for constructing correlative structure–property relationships in materials science based on multimodal characterization. However, this necessitates the development of methods for the physical interpretability of the resulting correlative models. Here, we demonstrate the potential of attention-based neural networks for revealing structure–property relationships and the underlying physical mechanisms, using the ferroelectric properties of PbTiO3 thin films as a case study. Through the analysis of attention scores, we disentangle the influence of distinct domain patterns on the polarization switching process. The attention-based Transformer model is explored both as a direct interpretability tool and as a surrogate for explaining representations learned via unsupervised machine learning, enabling the identification of physically grounded correlations. We compare attention-derived interpretability scores with classical SHapley Additive exPlanations analysis and show that, in contrast to applications in natural language processing, attention mechanisms in materials science exhibit high efficiency in highlighting meaningful structural features.

Slautin, Boris [Independent Researcher]↗
DOE OSTI2020

A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy datasets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored dataset. Unfortunately, and this quantity is rarely known a priori. The existing methods for determining latent dimensionality, such as Automatic Relevance Determination (ARD), are mostly heuristic and utilize different characteristics to estimate the number of hidden features. However, all of them require human presence to make a final determination of K. Here we utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial dataset, and estimates which K produces stable groups of latent features that reconstruct the initial dataset well. We then train a Multi-Layter Perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristics of the NMF solution, obtained from NMFk. In order to train the MLP classifier, a training set of 58,660 matrices with predetermined latent features were factorized with NMFk. The MLP classifier in conjunction with NMFk maintains a greater than 95% success rate when applied to a held out test set. Additionally, when applied to two well-known benchmark datasets, the swimmer and MIT face data, NMFk/MLP correctly recovers the established number of hidden features. Finally, we compare the accuracy of our method to the ARD, AIC and Stability-based methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Improving qubit readout with hidden Markov models

We demonstrate the application of pattern recognition algorithms via hidden Markov models (HMM) for qubit readout. This scheme provides a state-path trajectory approach capable of detecting qubit-state transitions and makes for a robust classification scheme with higher starting-state assignment fidelity than when compared to a multivariate Gaussian or a support vector machine scheme. Therefore, the method also eliminates the qubit-dependent readout time optimization requirement in current schemes. Using a HMM state discriminator we estimate fidelities reaching the ideal limit. Unsupervised learning gives access to transition matrix, priors, and IQ distributions, providing a toolbox for studying qubit-state dynamics during strong projective readout.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Symbolic pregression: Discovering physical laws from distorted video

In this work, we present a method for unsupervised learning of equations of motion for objects in raw and optionally distorted unlabeled synthetic video (or, more generally, for discovering and modeling predictable features in time-series data). We first train an autoencoder that maps each video frame into a low-dimensional latent space where the laws of motion are as simple as possible, by minimizing a combination of nonlinearity, acceleration, and prediction error. Differential equations describing the motion are then discovered using Pareto-optimal symbolic regression. We find that our pre-regression (“pregression”) step is able to rediscover Cartesian coordinates of unlabeled moving objects even when the video is distorted by a generalized lens. Using intuition from multidimensional knot theory, we find that the pregression step is facilitated by first adding extra latent space dimensions to avoid topological problems during training and then removing these extra dimensions via principal component analysis. An inertial frame is autodiscovered by minimizing the combined equation complexity for multiple experiments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Topic modeling to discern irregular order patterns in unlabeled electronic health records

This paper presents an application of topic modeling on event sequences of Electronic Health Record (EHR) orders. Metaphorically, we approach clinical order event sequences of unlabeled data as if they are documents where words are the events that occurred in the history of an order. We demonstrate the approach leveraging Consult orders data. The details of the data preprocessing and the data structures are described along with the data sources. Latent Dirichlet Allocation (LDA) is leveraged to fit against the limited datasets prepared. Another open source tool – LDAvis is used for exploratory analysis of the LDA results. The preliminary results revealed some order patterns that are qualitatively evaluated as potential irregular transitions. The goal of this analysis is to provide unsupervised learning application to domain experts in the absence of labeled data where they can investigate captured patterns and identify irregular transitions of orders. Ultimately, such efforts will guide formalization of hazard detection algorithms that monitor EHR data to identify health information technology related hazards.

Ozmen, Ozgur↗
DOE OSTI2016

Detecting Peer-to-Peer Botnets in SCADA Systems

Supervisory Control and Data Acquisition (SCADA) systems monitor and control critical infrastructure such as the smart grid. As SCADA systems become increasingly interconnected and adopt more and more cyber-enabled components, the risks of cyber attacks become a major concern. Due to their decentralized organization, peer-to-peer (P2P) botnets are resilient to many existing takedown measures and can be exploited as an effective way to launch cyber attacks on SCADA systems. However, little work has been done to detect P2P botnets in SCADA systems, which carry traffic flows with characteristics significantly different from the Internet. In this paper, we design a P2P-botnet detection method for SCADA systems, leveraging built-in traffic monitoring capabilities of SCADA networking devices. The proposed method consists of two stages. In the first stage, we design a simple feature test to filter out non-P2P hosts, which significantly reduces the data volume for P2P-botnet identification. In the second stage, we jointly consider flow-based and connectivity-based features that effectively set apart bots from benign hosts. We propose to use unsupervised learning for P2P-botnet identification, which not only identifies known P2P botnets but also captures newly emerged ones. Our simulation results show that the proposed system achieves high detection rates with very few false positives. Furthermore, our evaluation shows that the proposed method can detect hosts running P2P SCADA applications that are infected by P2P bots.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

Mixture Model Framework for Traumatic Brain Injury Prognosis Using Heterogeneous Clinical and Outcome Data

Prognoses of Traumatic Brain Injury (TBI) outcomes are neither easily nor accurately determined from clinical indicators. This is due in part to the heterogeneity of damage inflicted to the brain, ultimately resulting in diverse and complex outcomes. Using a data-driven approach on many distinct data elements may be necessary to describe this large set of outcomes and thereby robustly depict the nuanced differences among TBI patients’ recovery. In this work, we develop a method for modeling large heterogeneous data types relevant to TBI. Our approach is geared toward the probabilistic representation of mixed continuous and discrete variables with missing values. The model is trained on a dataset encompassing a variety of data types, including demographics, blood-based biomarkers, and imaging findings. In addition, it includes a set of clinical outcome assessments at 3, 6, and 12 months post-injury. The model is used to stratify patients into distinct groups in an unsupervised learning setting. We use the model to infer outcomes using input data, and show that the collection of input data reduces uncertainty of outcomes over a baseline approach. In addition, we quantify the performance of a likelihood scoring technique that can be used to self-evaluate the extrapolation risk of prognosis on unseen patients.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Nondestructive Damage Detection of Concrete With Alkali-Silica Reactions Using Coda Wave and Anomaly Detection

An anomaly detection model for early damage detection for concrete structures undergoing alkali-silica reaction (ASR) is presented. It is difficult to detect ASR initiation and early damage without a reference expansion measurement. Coda waves, or the multiply scattered portion of ultrasonic waves, have been found to be indicative of small changes in complex material such as concrete. The relationship between concrete damage and relative velocity change and decorrelation of coda waves has been studied, but a generalized model which detects when damage occurs in a concrete structure is still lacking. The presented method uses features extracted from coda waves to detect early damage in concrete structures. The model uses unsupervised learning and only requires data from undamaged structures for training. During the training process, the reconstruction error of the training data is minimized. When the data collected from damaged concrete structures is used as an input of the model, it returns high reconstruction errors that indicate the occurrence of damage in the structures. The performance of the model is validated using experimental studies and has been shown to generalize across two different ASR specimens.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2016

Spatial intratumoral heterogeneity of proliferation in immunohistochemical images of solid tumors

Purpose: The interactions of neoplastic cells with each other and the microenvironment are complex. To understand intratumoral heterogeneity, subtle differences should be quantified. Main factors contributing to heterogeneity include the gradient ischemic level within neoplasms, action of microenvironment, mechanisms of intercellular transfer of genetic information, and differential mechanisms of modifications of genetic material/proteins. This may reflect on the expression of biomarkers in the context of prognosis/stratification. Hence, a rigorous approach for assessing the spatial intratumoral heterogeneity of histological biomarker expression with accuracy and reproducibility is required, since patterns in immunohistochemical images can be challenging to identify and describe. Methods: A quantitative method that is useful for characterizing complex irregular structures is lacunarity; it is a multiscale technique that exhaustively samples the image, while the decay of its index as a function of window size follows characteristic patterns for different spatial arrangements. In histological images, lacunarity provides a useful measure for the spatial organization of a biomarker when a sampling scheme is employed and relevant features are computed. The proposed approach quantifies the segmented proliferative cells and not the textural content of the histological slide, thus providing a more realistic measure of heterogeneity within the sample space of the tumor region. The aim is to investigate in whole sections of primary pancreatic neuroendocrine neoplasms (pNENs), using whole-slide imaging and image analysis, the spatial intratumoral heterogeneity of Ki-67 immunostains. Unsupervised learning is employed to verify that the approach can partition the tissue sections according to distributional heterogeneity. Results: The architectural complexity of histological images has shown that single measurements are often insufficient. Inhomogeneity of distribution depends not only on percentage content of proliferation phase but also on how the phase fills the space. Lacunarity curves demonstrate variations in the sampled image sections. Since the spatial distribution of proliferation in each case is different, the width of the curves changes too. Image sections that have smaller numerical variations in the computed features correspond to neoplasms with spatially homogeneous proliferation, while larger variations correspond to cases where proliferation shows various degrees of clumping. Grade 1 (uniform/nonuniform: 74%/26%) and grade 3 (uniform: 100%) pNENs demonstrate a more homogeneous proliferation with grade 1 neoplasms being more variant, while grade 2 tumor regions render a more diverse landscape (50%/50%). Hence, some cases show an increased degree of spatial heterogeneity comparing to others with similar grade. Whether this is a sign of different tumor biology and an association with a more benign/malignant clinical course needs to be investigated further. The extent and range of spatial heterogeneity has the potential to be evaluated as a prognostic marker. Conclusions: The association with tumor grade as well as the rationale that the methodology reflects true tumor architecture supports the technical soundness of the method. This reflects a general approach which is relevant to other solid tumors and biomarkers. Drawing upon the merits of computational biomedicine, the approach uncovers salient features for use in future studies of clinical relevance.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

Disentangling the Lexicons of Disaster Response in Twitter

Abstract: People around the world use social media platforms such as Twitter heavily to express their opinion about various as- pects of daily life. In the same way social media changes communication in daily life, it also is transforming the way individuals communicate during disasters and emergencies. Emergency officials have come to rely on social media to communicate alerts and updates. How do users communi- cate risk on social media? We used a novel information- theoretic unsupervised learning tool, CorEx, to extract and classify highly relevant words used by the public on Twit- ter during known emergencies, such as fires, explosions, and hurricanes. By utilizing the resulting classification strategy, authorities can use the derived language to craft more rel- evant risk communication to maximize the effectiveness of short-message broadcasts such as the Wireless Emergency Alerts (WEA) service.

clustering↗
DOE OSTI2024

DISSIDE: Dynamic In Silico Sample Identification for Discrete Evaluation

DISSIDE uses novel unsupervised learning to select samples that best represent the strongest a priori discrete pattern in a given data set. It further removes major outliers and "noisy" samples that do not fit discrete patterns well or represent outliers in groups below a defined n value. It then estimates the fit and strength of the a priori discrete pattern using both unconstrained and constrained methods for raw and cleaned data

LeBrun, Erick↗
DOE OSTI2023

Subseasonal Representation and Predictability of North American Weather Regimes Using Cluster Analysis

Abstract This study focuses on assessing the representation and predictability of North American weather regimes, which are persistent large-scale atmospheric patterns, in a set of initialized subseasonal reforecasts created using the Community Earth System Model, version 2 (CESM2). The k -means clustering was used to extract four key North American (10°–70°N, 150°–40°W) weather regimes within ERA5 reanalysis, which were used to interpret CESM2 subseasonal forecast performance. Results show that CESM2 can recreate the climatology of the four main North American weather regimes with skill but exhibits biases during later lead times with overoccurrence of the West Coast high regime and underoccurrence of the Greenland high and Alaskan ridge regimes. Overall, the West Coast high and Pacific trough regimes exhibited higher predictability within CESM2, partly related to El Niño. Despite biases, several reforecasts were skillful and exhibited high predictability during later lead times, which could be partly attributed to skillful representation of the atmosphere from the tropics to extratropics upstream of North America. The high predictability at the subseasonal time scale of these case-study examples was manifested as an “ensemble realignment,” in which most ensemble members agreed on a prediction despite ensemble trajectory dispersion during earlier lead times. Weather regimes were also shown to project distinct temperature and precipitation anomalies across North America that largely agree with observational products. This study further demonstrates that unsupervised learning methods can be used to uncover sources and limits of subseasonal predictability, along with systematic biases present in numerical prediction systems. Significance Statement North American weather regimes are large-scale atmospheric patterns that can persist for several days. Their skillful subseasonal (2 weeks or greater) prediction can provide valuable lead time to prepare for temperature and precipitation anomalies that can stress energy and water resources. The purpose of this study was to assess the climatological representation and subseasonal predictability of four key North American weather regimes using a research subseasonal prediction system and clustering analysis. We found that the Pacific trough and West Coast high regimes exhibited higher predictability than other regimes and that skillful representation of conditions across the tropics and extratropics can increase predictability during later lead times. Future work will quantify causal pathways associated with high predictability.

58 GEOSCIENCES↗
DOE OSTI2023

Seasonal and Geographical Variations in Fundamental Weather Patterns during Extreme Precipitation as Identified from Omega Equation Forcing

Abstract We investigate the large-scale weather patterns during extreme precipitation (PEx) events over the conterminous United States (CONUS) by applying a version of the quasigeostrophic (QG) omega equation. This work aims to develop a climatology of the weather patterns most related to PEx events during current climate. Extreme events are examined for each of seven regions defined by consistent annual cycles of precipitation and spanning the CONUS. For the CONUS we train several self-organizing maps (SOM) on a pressure–time series of vertical velocity from each of the advective forcing terms in the QG omega equation for each extreme event. The unsupervised learning of the SOM allows us to identify the most descriptive set of nine patterns in vertical velocity associated with precipitation extremes. This method finds multiple frontal- and cyclone-driven patterns while grouping primarily convective events into one pattern. Frontal events include a synoptic pattern consistent with West Coast atmospheric river events as well as pattern groups linked to developing and to mature (“occluded”) frontal cyclones. The primary patterns found during PEx events vary seasonally and geographically. Frontal cyclone patterns are most common during PEx events during summer in the part of the Great Plains and during winter for the Northeast, Southeast, Pacific Northwest, and Southwest. Convection is the most common pattern during summer in all regions. Except in the Southeast, the annual cycles of monthly number of PEx events and average precipitation match well, partially validating our choice of regions to aggregate PEx events.

Meteorology & Atmospheric Sciences↗