Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “constrained learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Seismicity-constrained fault detection and characterization with a multitask machine learning model

Geological fault detection and characterization are crucial for understanding subsurface dynamics across scales. While methods for fault delineation based on either seismicity location analysis or seismic image reflector discontinuity are well-established, a systematic approach that integrates both data types remains absent. We develop a novel machine learning model that unifies seismic reflector images and seismicity location information to automatically identify geological faults and characterize their geometrical properties. The model encodes a seismic image and a seismicity location image separately, and fuses the encoded features with a spatial-channel attention fusion module to improve the learning of important features in both inputs. We design an automated strategy to generate high-quality synthetic training data and labels. To improve the realism of the seismicity location image, we include random seismicity noise and missing seismicity location associated with some of the faults. We validate the model’s efficacy and accuracy using synthetic data examples and two field data examples. Moreover, we show that fine-tuning the trained model with a small, domain-specific dataset enhances its fidelity for field data applications. The results demonstrate that integrating seismicity location and seismic images into a unified framework allows the end-to-end neural network to achieve higher fidelity and accuracy in delineating subsurface faults and their geometrical properties compared with image-only fault detection methods. Our approach offers an adaptive data-driven tool for geological fault characterization and seismic hazard mitigation, bridging the gap between seismicity location and image-based fault detection methods.

58 GEOSCIENCES↗
DOE OSTI2023

Constraining the Multiscale Structure of Geophysical Fields in Machine Learning: The Case of Precipitation

The use of deep-learning algorithms for estimating the value of geophysical variables from remotely sensed information is rapidly expanding. The typical objective function minimized in such algorithms is the mean square error (MSE), which is known to lead to smooth estimates with compressed dynamical range as compared to the true distribution of the variable of interest. Here, we introduce and evaluate alternative objective functions, focusing on the retrieval of precipitation rates from satellite passive microwave radiometric measurements using a deep convolutional neural network. For this testbed application, the results show that explicitly imposing the preservation of the statistical distribution and spatial wavelet power spectrum of the target variable allows to accurately reproduce extreme values and sharp gradients across multiple scales in this study.

58 GEOSCIENCES↗
DOE OSTI2024

Learning protocols for the fast and efficient control of active matter

Exact analytic calculation shows that optimal control protocols for passive molecular systems often involve rapid variations and discontinuities. However, similar analytic baselines are not generally available for active-matter systems, because it is more difficult to treat active systems exactly. Here we use machine learning to derive efficient control protocols for active-matter systems, and find that they are characterized by sharp features similar to those seen in passive systems. We show that it is possible to learn protocols that effect fast and efficient state-to-state transformations in simulation models of active particles by encoding the protocol in the form of a neural network. We use evolutionary methods to identify protocols that take active particles from one steady state to another, as quickly as possible or with as little energy expended as possible. Our results show that protocols identified by a flexible neural-network ansatz, which allows the optimization of multiple control parameters and the emergence of sharp features, are more efficient than protocols derived recently by constrained analytical methods. Our learning scheme is straightforward to use in experiment, suggesting a way of designing protocols for the efficient manipulation of active matter in the laboratory.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

Turn-key constrained parameter space exploration for particle accelerators using Bayesian active learning

Abstract Particle accelerators are invaluable discovery engines in the chemical, biological and physical sciences. Characterization of the accelerated beam response to accelerator input parameters is often the first step when conducting accelerator-based experiments. Currently used techniques for characterization, such as grid-like parameter sampling scans, become impractical when extended to higher dimensional input spaces, when complicated measurement constraints are present, or prior information known about the beam response is scarce. Here in this work, we describe an adaptation of the popular Bayesian optimization algorithm, which enables a turn-key exploration of input parameter spaces. Our algorithm replaces the need for parameter scans while minimizing prior information needed about the measurement’s behavior and associated measurement constraints. We experimentally demonstrate that our algorithm autonomously conducts an adaptive, multi-parameter exploration of input parameter space, potentially orders of magnitude faster than conventional grid-like parameter scans, while making highly constrained, single-shot beam phase-space measurements and accounts for costs associated with changing input parameters. In addition to applications in accelerator-based scientific experiments, this algorithm addresses challenges shared by many scientific disciplines, and is thus applicable to autonomously conducting experiments over a broad range of research topics.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While traditional ensemble classifiers yield improved task performance, such methods come with a steep cost in model complexity. Through knowledge distillation, it is possible to retain some performance improvements from an ensemble, while minimizing costs to model complexity. In this work, we implement a weakly supervised ensemble using clustering-constrained-attention multiple-instance learners (CLAM), which uses attention and instance-level clustering to identify task salient regions and feature extraction in whole slides. By applying logit-based and attention-based knowledge distillation, we show it is possible to retain some performance improvements resulting from the ensemble at zero cost to model complexity.

Alamudun, Folami↗
DOE OSTI2024

Physics-informed neural networks for heterogeneous poroelastic media

This study presents a novel physics-informed neural network (PINN) framework for modeling poroelasticity in heterogeneous media with material interfaces. The approach introduces a composite neural network (CoNN) where separate neural networks predict displacement and pressure variables for each material. While sharing identical activation functions, these networks are independently trained for all other parameters. To address challenges posed by heterogeneous material interfaces, the CoNN is integrated with the Interface-PINNs (I-PINNs) framework (Sarma et al., Comput. Methods Appl. Mech. Eng. 429: 117135, 2024), allowing different activation functions across material interfaces. Further, this ensures accurate approximation of discontinuous solution fields and gradients. Performance and accuracy of this combined architecture were evaluated against the conventional PINNs approach, a single neural network (SNN) architecture, and the eXtended PINNs (XPINNs) framework through two one-dimensional benchmark examples with discontinuous material properties. The results show that the proposed CoNN with I-PINNs architecture achieves an RMSE that is two orders of magnitude better than the conventional PINNs approach and is at least 40 times faster than the SNN framework. Compared to XPINNs, the proposed method achieves an RMSE at least one order of magnitude better and is 40% faster.

42 ENGINEERING↗
DOE OSTI2020

Deciphering the archaeological record: Cosmological imprints of nonminimal dark sectors

Many proposals for physics beyond the Standard Model give rise to a dark sector containing many degrees of freedom. In this work, we explore the cosmological implications of the nontrivial dynamics which may arise within such dark sectors, focusing on decay processes which take place entirely among the dark constituents. First, we demonstrate that such decays can leave dramatic imprints on the resulting dark-matter phase-space distribution. In particular, this distribution need not be thermal—it can even be multimodal, exhibiting a nontrivial pattern of peaks and troughs as a function of momentum. We then proceed to show how these features can induce modifications to the matter power spectrum. Finally, we assess the extent to which one can approach the archaeological “inverse” problem of deciphering the properties of an underlying dark sector from the matter power spectrum. Indeed, one of the main results of this paper is a remarkably simple conjectured analytic expression which permits the reconstruction of many of the important features of the dark-matter phase-space distribution directly from the matter power spectrum. Here, our results provide an interesting toolbox of methods for learning about, and potentially constraining, the features of nonminimal dark sectors and their dynamics in the early universe.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Dynamic Decarbonization through Autonomous Physics-Centric Deep Learning and Optimization of Building Operations (Abstract only)

This project directly addresses the primary goal of Area of Interest 2 in the CRADA call: to advance optimization-based integrated energy management systems in commercial and residential buildings. Pacific Northwest National Laboratory (PNNL) and its industry partner PassiveLogic aim to accomplish this by reaching three key objectives. First, to ensure a broad impact in the building controls industry, PNNL will extend its open-source library for predictive control synthesis by augmenting its capabilities with data-driven self-learning of building models and auto-calibration of predictive controllers. The effort will focus on building use cases selected in collaboration with PassiveLogic. The team will specifically address the development of methods for data-driven adaptation of building models, investigation of model architectures that best address specific building types, and automated synthesis of differentiable predictive controllers that optimize diverse objectives. Second, PNNL will collaborate with PassiveLogic to integrate the aforementioned methods with PasiveLogic’s advanced controls platform. The collaborative integration effort will inform the developments under the first objective by providing specific data on the attainable performance of model learning on resource-constrained edge computing platforms. This software integration effort will increase the technical maturity of the developed libraries by exploring the use of software integration tools and methods. Third, PNNL and PassiveLogic will work to improve the technology readiness of the developed predictive controllers by testing their performance in relevant test environments, such as high-fidelity simulation, hardware in the loop, and actual test buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1998

Hybrid Motion Planning with Multiple Destinations

In our initial proposal, we laid plans for developing a hybrid motion planning system that combines the concepts of visibility-based motion planning, artificial potential field based motion planning, evolutionary constrained optimization, and reinforcement learning. Our goal was, and still is, to produce a hybrid motion planning system that outperforms the best traditional motion planning systems on problems with dynamic environments. The proposed hybrid system will be in two parts the first is a global motion planning system and the second is a local motion planning system. The global system will take global information about the environment, such as the placement of the obstacles and goals, and produce feasible paths through those obstacles. We envision a system that combines the evolutionary-based optimization and visibility-based motion planning to achieve this end.

Clouse, Jeffery↗
NASA NTRSDate not supplied

An Inverse Chance-constrained Approach to the Calibration of Robust Models

This paper proposes a strategy to calibrate computational models according to uncertain input-output data. To this end, uncertainty in the data is first quantified by creating adversarial data sets. Samples drawn from such sets are then mapped from the input-output space to the parameter space using an inverse mapping. This mapping minimizes the collective output spread of an ensemble of point predictions while satisfying a set of individual data-matching requirements. The distribution of the resulting parameter points, which often exhibits strong parameter dependencies, is then modeled using sliced-normals. The chance-constrained formulation used to learn this distribution enables the analyst to trade-off a greater likelihood for most of the data against a lower likelihood for some of the data thereby relaxing the conservatism of the calibrated model. This formulation not only neglects the worst-performing quantiles of each adversarial distribution but also eliminates the potentially serious effects that outliers might have on the resulting model. This calibration approach not only has a considerably lower computational cost than the standard forward approach but it also allows for the identification of suitable distribution classes, which in turn yield better calibrated models.

Calibration↗
DOE OSTI2024

Coherency-Aware Learning Control of Inverter-Dominated Grids: A Distributed Risk-Constrained Approach

Here, this letter investigates the importance of integrating the coherency knowledge for designing controllers to dampen sustained oscillations in wide-area power networks with significant penetration of inverter-interfaced resources. Coherency is a fundamental property of power systems, where time-scale separation in frequency dynamics leads to clustered behavior among generators of different groups. Large-scale penetration of inverter-driven low inertia resources replacing conventional synchronous generators (SGs) can lead to perturbation in the coherent partitioning; hence, integrating such information is of utmost importance for oscillation control designs. We present the coherency-aware design of a distributed output feedback-based reinforcement learning method that additionally incorporates risk constraints to capture the uncertainties related to net-load fluctuations. The use of domain-aware coherency information has produced improved training and oscillation performance than the coherency-agnostic control design, hence proving to be effective in controller design. Finally, we validated the proposed method with numerical experiments on the benchmark IEEE 68-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2007

Salience Assignment for Multiple-Instance Regression

We present a Multiple-Instance Learning (MIL) algorithm for determining the salience of each item in each bag with respect to the bag's real-valued label. We use an alternating-projections constrained optimization approach to simultaneously learn a regression model and estimate all salience values. We evaluate this algorithm on a significant real-world problem, crop yield modeling, and demonstrate that it provides more extensive, intuitive, and stable salience models than Primary-Instance Regression, which selects a single relevant item from each bag.

regression↗
DOE OSTI2024

TNet: A Model-Constrained Tikhonov Network Approach for Inverse Problems

Deep learning (DL), in particular deep neural networks, by default is purely data-driven and in general does not require physics. This is the strength of DL but also one of its key limitations when applied to science and engineering problems in which underlying physical properties—such as stability, conservation, and positivity—and accuracy are required. DL methods in their original forms are often not capable of respecting the underlying mathematical models or achieving desired accuracy even in big-data regimes. On the other hand, many data-driven science and engineering problems, such as inverse problems, typically have limited experimental or observational data, and DL would overfit the data in this case. Leveraging information encoded in the underlying mathematical models, we argue, not only compensates for missing information in low data regimes but also provides opportunities to equip DL methods with the underlying physics, hence promoting better generalization. This paper develops a model-constrained DL approach and its variant TNet—a Tikhonov neural network—which are capable of learning not only information hidden in the training data but also in the underlying mathematical models to solve inverse problems governed by partial differential equations in low data regimes. We provide the constructions and some theoretical results for the proposed approaches for both linear and nonlinear inverse problems. Since TNet is designed to learn inverse solutions with Tikhonov regularization, it is interpretable: in fact it recovers Tikhonov solutions for linear cases while potentially approximating Tikhonov solutions for nonlinear inverse problems. We also prove that data randomization can enhance not only the smoothness of the networks but also their generalizations. Comprehensive numerical results confirm the theoretical findings and show that with even as little as 1 training data sample for one-dimensional (1D) deconvolution, 5 for an inverse 2D heat conductivity problem, 100 for inverse initial conditions for a time-dependent 2D Burgers’s equation, and 50 for inverse initial conditions for 2D Navier–Stokes equations, TNet solutions can be as accurate as Tikhonov solutions while being several orders of magnitude faster. Furthermore, this is possible owing to the model-constrained term, replications, and randomization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Quantum-inspired weight-constrained neural network: Reducing variable numbers by 100× compared to standard neural networks

Although quantum machine learning has shown great promise, the practical application of quantum computers remains constrained in the noisy intermediate-scale quantum era. To take advantage of quantum machine learning, we investigate the underlying mathematical principles of these quantum models and find that the quantum neural network with amplitude encoding is equivalent to a weight-constrained neural network. Motivated by this discovery, we develop a classical weight-constrained neural network. We find that this approach can reduce the number of variables in a classical neural network by a factor of 135 while preserving its accuracy. In addition, we develop a dropout method to enhance the robustness of quantum machine learning models, which are highly susceptible to adversarial attacks. This technique can also be applied to improve the adversarial robustness of the classical weight-constrained neural network, which is essential for industry applications, such as self-driving vehicles. Our work offers an approach to reduce the complexity of large classical neural networks, addressing a critical challenge in machine learning.

quantum algorithms & computation↗
DOE OSTI2025

The Effects of Compounded Model Size Reductions on Adversarial Robustness

Recent advances in Edge AI and Tiny Machine Learning (TinyML) have enabled the deployment of machine learning models on resource-constrained environments. However, deploying these models on edge devices, such as micro-controllers, requires significant model footprint reduction through a variety of techniques such as quantization, pruning, and clustering. While these optimization methods offer considerable advantages, they potentially introduce AI-related security vulnerabilities, particularly concerning model robustness with respect to adversarial AI attacks. Prior research has extensively examined the impact of quantization on adversarial robustness; however, the effects of alternative reduction techniques and their combinations remain understudied. This paper investigates the impact of model size reduction techniques on adversarial robustness, when applied individually and combined. We utilized Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks to generate adversarial perturbations for both training and testing data, and then evaluated the models' accuracy under adversarial training conditions. Our findings revealed that reduction techniques generally diminished robustness; although, combining techniques was not found to make robustness any worse than when applied individually. Moreover, specific techniques can potentially enhance resistance to small size perturbations. This research provides insights into the trade-offs between model size reduction and security, establishing a foundation for future investigations into improving adversarial training techniques and methodologies for maintaining robustness while preserving memory footprint benefits.

Austria, Phillipe [ORNL] (ORCID:0000000236223973)↗
DOE OSTI2023

Extending Parsimonious Bayesian Inference

Parsimonious Bayesian inference is a theoretical framework for efficient data assimilation that seeks to balance increased consistency between predictions and training data against corresponding increases in model complexity. Within this framework, over-training is understood as optimization that encodes excessive information within model parameters while only achieving small improvements between predictions and training data. This project aims to develop practical methods of limiting excess model information during optimization. One key observation is that practical heuristics for parsimonious learning in high-dimensions must balance expressivity, i.e. the ability of the model to capture diverse predictions with only a few non-zero parameters, against discoverability, i.e. the ability to train the model with gradient-based optimization and drive parameters to low information states. As such, we developed logical activation functions that are able to adaptively approximate arbitrary truth tables that define Boolean logic operations within a probabilistic framework. These functions have demonstrated the ability to learn exclusive disjunction (XOR) and conditioned disjunction (if [condition] then [result_if_true] else [result_if_false]) within a single layer of a neural network. To efficiently exploit these activation functions to drive parsimonious learning required several other advances within the domain of variational inference. The most efficient form of complexity suppression is structured sparsification, driving most model parameters to zero while achieving the structural coherence among nonzeros needed for bandwidth reduction. Such models are not only far more efficient at suppressing information-theoretic complexity, they also reduce the other forms of complexity (computations, communication, storage, and the number of dependencies needed to evaluate predictions). Aiming to support enhanced sparsification, this project examined new approaches to high-dimensional variational inference that allow us to calibrate and control parameter uncertainty during optimization. By identifying which parameters can sustain sparsifying perturbations with little impact on prediction quality, we can develop better pruning strategies by framing them as approximate Bayesian inference. These advances also open paths to mitigate concerns with deploying advanced learning methods in resource-constrained environments, such as running models on power-limited or communication-limited devices.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1990

Planning and scheduling research at NASA Ames Research Center

Planning and scheduling is the area of artificial intelligence research that focuses on the determination of a series of operations to achieve some set of (possibly) interacting goals and the placement of those operations in a timeline that allows them to be accomplished given available resources. Work in this area at the NASA Ames Research Center ranging from basic research in constrain-based reasoning and machine learning, to the development of efficient scheduling tools, to the application of such tools to complex agency problems is described.

Friedland, Peter↗