Parameterized MDPs and Reinforcement Learning Problems—A Maximum Entropy Principle-Based Framework
Not provided.
Automation & Control Systems↗
SEARCH · Engineering Papers
Results for “control framework”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
When physics-informed data analytics outperforms black-box machine learning: A case study in thickness control for additive manufacturing
Aerosol jet printing (AJP) has emerged as a promising noncontact additive manufacturing method for high-resolution printing for a wide range of material systems. A key challenge limiting the broader adoption of AJP in the material science community is the lack of methods to precisely control thickness. Herein, we develop a model-based design of experiment (MBDoE) framework that integrates physics-informed models, nonlinear regression, and information criteria to postulate, select and calibrate the best model to describe and optimize the AJP manufacturing process. Starting with already available data from system commissioning (e.g., prior single variable sensitivity analysis), four candidate physics-informed models are postulated and trained. MBDoE identifies a single additional optimal experiment to validate these predictive models with quantified uncertainties, which are then used to determine the best experimental conditions to control printed film thickness. As a comparative benchmark, the analysis is repeated using the same dataset with nonparametric Gaussian process regression (GPR) model that does not incorporate physical information. Using MBDoE principles, we find that only five experiments are necessary to calibrate the nonlinear physics-informed parametric model, and with said limited data, this model outperforms the black-box machine learning GPR model. This key result underscores an emerging trend in the data science community: incorporating physical information into predictive models often drastically reduces the data requirements. Leveraging MBDoE further increased the data efficiency. By design, the proposed data science framework is general in nature and can be easily extended to other experimental and additive manufacturing systems beyond AJP.
Aerosol jet printing↗
Multi-objective automatic discovery of optimized metamaterials for varying velocity impact protection
Mechanical metamaterials have demonstrated exceptional impact performance while remaining lightweight. Impact resistance has traditionally been investigated using quasi-static simulations, often with the assumption that performance will translate to high-velocity impact scenarios. However, critical crash protection parameters—such as peak stress and absorbed energy—are highly sensitive to impact velocity, leading to inconsistent performance under dynamic loading. To address this, we introduce a strain-rate-aware, active deep learning framework that enables multi-objective optimization of impact protection metrics across a wide range of impact velocities. Our framework captures the strain-rate sensitivity of architected lattices by learning to control spatial gradation in cellular metamaterials, resulting in over 200 % enhancement in impact protection relative to state-of-the-art designs such as Voronoi and re-entrant lattices. We demonstrate its practical utility by designing next-generation lattice structures for automotive bumper systems that satisfy multiple, velocity-specific safety criteria—capabilities beyond those of conventional designs. More than just a predictive tool, this framework marks the first step towards enabling adaptable impact-resistant structures across dynamic regimes.
Deep learning↗
Model Predictive Control of Discrete-Continuous Energy Systems via Generalized Disjunctive Programming
Generalized Disjunctive Programming (GDP) provides an alternative framework to model optimization problems with both discrete and continuous variables. The key idea behind GDP involves the use of logical disjunctions to represent discrete decisions in the continuous space, and logical propositions to denote algebraic constraints in the discrete space. Compared to traditional mixed-integer programming (MIP), the inherent logic structure in GDP yields tighter relaxations that are exploited by global branch and bound algorithms to improve solution quality. In this paper, we present a general GDP model for optimal control of hybrid systems that exhibit both discrete and continuous dynamics. Specifically, we use GDP to formulate a model predictive control (MPC) model for piecewise-affine systems with implicit switching logic. As an example, the GDP-based MPC approach is used as a supervisory control to improve energy efficiency in residential buildings with binary on/off, relay-based thermostats. A simulation study is used to demonstrate the validity of the proposed approach, and the improved solution quality compared to existing MIPbased control approaches.
Bhattacharya, Arnab↗
Joint ptycho-tomography with deep generative priors
Abstract Joint ptycho-tomography is a powerful computational imaging framework to recover the refractive properties of a 3D object while relaxing the requirements for probe overlap that is common in conventional phase retrieval. We use an augmented Lagrangian scheme for formulating the constrained optimization problem and employ an alternating direction method of multipliers (ADMM) for the joint solution. ADMM allows the problem to be split into smaller and computationally more efficient subproblems: ptychographic phase retrieval, tomographic reconstruction, and regularization of the solution. We extend our ADMM framework with plug-and-play (PnP) denoisers by replacing the regularization subproblem with a general denoising operator based on machine learning. While the PnP framework enables integrating such learned priors as denoising operators, tuning of the denoiser prior remains challenging. To overcome this challenge, we propose a denoiser parameter to control the effect of the denoiser and to accelerate the solution. In our simulations, we demonstrate that our proposed framework with parameter tuning and learned priors generates high-quality reconstructions under limited and noisy measurement data.
97 MATHEMATICS AND COMPUTING↗
Simulation-based assessment of ASHRAE Guideline 36, considering energy performance, indoor air quality, and control stability
This study assesses American Society of Heating, Refrigerating and Air-Conditioning Engineers Guideline 36 (G36) with a typical medium office building. Specifically, this study employed a Modelica model of a variable air volume (VAV) system that serves this building, which includes components for representing indoor virus transmission and filtration. It then implemented the G36 control sequences for both water-side and air-side equipment in Python. After that, this study conducted the assessment by co-simulating G36 and the Modelica model using the Building Operations Testing Framework. Unlike existing works, this work has three unique features: (1) It considers the interactions between control sequences for water-side and air-side equipment of the studied VAV system. (2) It assesses the performance of G36 from the perspective of IAQ. (3) It examines the short-term behaviors of the studied building under G36 to understand the control stability. This assessment confirms significant energy savings from G36, largely because of the interaction between supply air temperature and hot water controls. It also reveals a trade-off between the ability to slow the spread of virus and the energy performance via demand controlled ventilation. Lastly, it emphasizes the necessity of tuning local feedback control when implementing G36.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Network Reconfiguration for Enhanced Operational Resilience Using Reinforcement Learning
This paper proposes a reinforcement learning-based approach for distribution network reconfiguration(DNR) to enhance the resilience of the electric power supply. Resilience enhancements usually require solving large-scale stochastic optimization problems that are computationally expensive and sometimes infeasible. The exceptional performance of reinforcement learning techniques has encouraged their adoption in various power system control studies, specifically resilience-based real-time applications. In this paper, a single agent framework is developed using an Actor-Critic algorithm (ACA) to determine statuses of tie-switches in a distribution feeder impacted by an extreme weather event. The proposed approach provides a fast-acting control algorithm that reconfigures the feeder topology to reduce or even avoid load shedding. The problem is formulated as a discrete Markov decision process in such a way that a system state captures the system topology and its operational characteristics. An action is made to open or close a specific set of tie-switches after which a reward is calculated to evaluate the practicality and advantage of that action. The iterative Markov process is used to train the proposed ACA under diverse failure scenarios and is demonstrated on the 33-node distribution feeder system. Results show the capability of the proposed ACA to determine proper switching action of tie-switches with accuracy exceeding 93%.
actor critic↗
A deep learning framework for layer-wise porosity prediction in metal powder bed fusion using thermal signatures
Abstract Part quality manufactured by the laser powder bed fusion process is significantly affected by porosity. Existing works of process–property relationships for porosity prediction require many experiments or computationally expensive simulations without considering environmental variations. While efforts that adopt real-time monitoring sensors can only detect porosity after its occurrence rather than predicting it ahead of time. In this study, a novel porosity detection-prediction framework is proposed based on deep learning that predicts porosity in the next layer based on thermal signatures of the previous layers. The proposed framework is validated in terms of its ability to accurately predict lack of fusion porosity using computerized tomography (CT) scans, which achieves a F1-score of 0.75. The framework presented in this work can be effectively applied to quality control in additive manufacturing. As a function of the predicted porosity positions, laser process parameters in the next layer can be adjusted to avoid more part porosity in the future or the existing porosity could be filled. If the predicted part porosity is not acceptable regardless of laser parameters, the building process can be stopped to minimize the loss.
42 ENGINEERING↗
RLGBS: Reinforcement Learning-Guided Beam Search for process optimization in a paper machine dryer section
Paper drying is responsible for over two-thirds of energy consumption in the U.S. pulp and paper industry, presenting significant potential for energy savings through optimization of process parameters. Current approaches often assume fixed operating conditions, neglecting dynamic ambient and process variations that limit achievable savings and real-world applicability. To this end, we develop a physics-based simulation environment for a paper machine dryer section and propose a reinforcement learning (RL) framework to minimize overall energy consumption by optimizing drying process parameters under diverse operating conditions. To mitigate overdrying and numerical instabilities caused by suboptimal local RL actions, we introduce Reinforcement Learning-Guided Beam Search (RLGBS), which explores multiple action sequences in parallel using beam search. Instead of making step-by-step decisions, RLGBS prioritizes solutions based on cumulative probability, reducing the impact of individual suboptimal actions. Experiments demonstrate that RLGBS achieves consistent energy savings under unseen operating conditions not encountered during training, outperforming conventional RL methods. While validated in drying optimization, this framework is broadly applicable to other RL-based industrial process control problems.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
The Role of Cation Coordination in the Electrical and Optical Properties of Amorphous Transparent Conducting Oxides
Amorphous oxide semiconductor materials have demonstrated numerous advantages without compromise of electrical properties as compared to their crystalline counterparts, yet understanding of the fundamental principles allowing this has remained elusive. To study the origins of enhanced optoelectronic properties, we apply high-throughput, combinatorial sputtering, structural and spectral mapping, and computationally intensive ab initio molecular dynamics simulations with density functional theory to a ternary, post-transition metal oxide system, namely, zinc tin oxide. The deposited thin films exhibit a high figure of merit, achieving carrier densities in the range of 1019 to 1020 cm–3 and carrier mobilities up to 35 cm2/Vs. These results highlight the role of local distortions and cation coordination in determining the microscopic origins of carrier generation and transport. In particular, we identify the strong likelihood of Sn undercoordination in both Zn-poor and Zn-rich phases leading to the high carrier concentrations observed. This not only diverges from the still widespread historical indictment of oxygen vacancies controlling carrier population in crystalline oxides but also provides a comprehensive framework to describe the unique structure–property relationships using specific structural and electronic descriptors in disordered phase materials.
CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS,M↗
Dimensional Reduction Guides Electronic Structure Evolution in the A n Cu 4–n SnS 4 Semiconductor Series
The search for new functional materials with tunable properties remains a central challenge in chemistry, particularly for applications in energy and electronics. In this work, we present a framework for predictive crystal design in alkali metal chalcogenides that enables controlled dimensional reduction of a parent covalent motif, yielding a broad range of electronic structures, which systematically evolve from one parent to the other. We present 11 new members of the A n Cu 4–n SnS 4 family (A = alkali metal; n = 0–4), which reduce the three-dimensional (3D) covalent network of Cu 4 SnS 4 into various 3D, 2D, 1D, and 0D [Cu 4–n SnS 4 ] n− motifs through the substitution of Cu with alkali metals of various radii. The end members of the family set the range in achievable band gaps at 0.99 eV for fully covalent Cu 4 SnS 4 (n = 0) and 3.38 eV for K 4 SnS 4 (n = 4) with 0D [SnS 4 ] n− tetrahedra. As the dimensionality of [Cu 4–n SnS 4 ] n− systematically reduces within A n Cu 4–n SnS 4 (n = 1–3), a stepwise increase in band gap energy occurs through a gradual decrease in the energy of the valence band maximum and an increase in the conduction band minimum, with an increase in the effective masses of charge carriers. Furthermore, irrespective of the alkali metal, the thermal stability decreases with decreasing [Cu 4–n SnS 4 ] n− dimensionality within the quaternary members. Most importantly, we demonstrate that predictable crystal structure and property evolution for a given composition space is possible by deriving a general formula based on substituting the covalent metals of a parent structure with alkali metals.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Room temperature doping of Ln 3+ in perovskite nanoparticles: a halide exchange mediated cation exchange approach
This study presents a halide exchange mediated cation exchange strategy for a room temperature doping of trivalent lanthanide cations (Ln 3+ ) in cesium lead halide (CsPbX 3 ) nanoparticles (NPs). Post-synthetic addition of LnCl 3 [Ln = Nd, Sm, Eu, Tb, Dy, and Yb] to a solution of CsPbBr 3 NPs generates the corresponding lanthanide doped NPs which display host sensitized Ln 3+ emission. Structural and spectroscopic characterizations indicate a successful halide exchange and substitutional displacement of Pb 2+ by Ln 3+ . The effect of halide identity in controlling the Ln 3+ sensitization was also evaluated. Here, a photophysical framework is presented that can be used to predict the Ln 3+ sensitization in perovskite NPs semiempirically, thereby removing the constraints of trial and error in designing a perovskite NP-Ln 3+ host–guest combination.
77 NANOSCIENCE AND NANOTECHNOLOGY↗
The concept of spin ice graphs and a field theory for their charges
Originally detected in rare earth pyrochlores, spin ice physics is now being artificially extended to a variety of geometries that control collective behavior and exotic properties, making graph theory their proper framework. We relate spin ice notions, such as ice rule, ice manifold, Coulomb phases, charges, and monopoles, to graph-theoretical notions, such as balance, in/out-degrees, and Euler paths. We then propose a field-theoretical treatment in which topological charges and monopoles are the degrees of freedom, while the binary spins are subsumed in an entropic interaction among charges. We show that for a spin ice on a graph in a Gaussian approximation, the kernel of the entropic interaction is the inverse of the graph Laplacian, and we compute screening functions from the graph spectra as Green operators for the screened Poisson problem on a graph. We then apply the treatment to star graphs, tournaments, cycles, and regular spin ice in different dimensions. Our aim is twofold: to set spin ice physics in a proper graph setting, where only topological rather than geometrical notions hold, and to invite graph theorists to contribute their powerful tools to the field of spin ice.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Virtual Time III, Part 1: Unified Virtual Time Synchronization for Parallel Discrete Event Simulation
Algorithms for synchronization of parallel discrete event simulation have historically been divided between conservative methods that require lookahead but not rollback, and optimistic methods that require rollback but not lookahead. In this paper we present a new approach in the form of a framework called Unified Virtual Time (UVT) that unifies the two approaches, combining the advantages of both within a single synchronization theory. Whenever timely lookahead information is available, a logical process (LP) executes conservatively using an irreversible event handler. When lookahead information is not available the LP does not block, as it would in a classical conservative execution, but instead executes optimistically using a reversible event handler. The switch from conservative to optimistic synchronization and back is decided on an event-by-event basis by the simulator, transparently to the model code. UVT treats conservative synchronization algorithms as optional accelerators for an underlying optimistic synchronization algorithm, enabling the speed of conservative execution whenever it is applicable, but otherwise falling back on the generality of optimistic execution. We describe UVT in a novel way, based on fundamental invariants, monotonicity requirements, and synchronization rules. UVT permits zero-delay messages and pays careful attention to tie-handling using superposition. We prove that under fairly general conditions a UVT simulation always makes progress in virtual time. This is Part 1 of a trio of papers describing the UVT framework for PDES, mixing conservative and optimistic synchronization and integrating throttling control.
97 MATHEMATICS AND COMPUTING↗
HP-MDR: High-performance and Portable Data Refactoring and Progressive Retrieval with Advanced GPUs
Scientific applications produce vast amounts of data, posing grand challenges in the underlying data management and analytic tasks. Progressive compression is a promising way to address this problem, as it allows for on-demand data retrieval with significantly reduced data movement cost. However, most existing progressive methods are designed for CPUs, leaving a gap for them to unleash the power of today’s heterogeneous computing systems with GPUs.In this work, we propose HP-MDR, a high-performance and portable data refactoring and progressive retrieval framework for GPUs. Our contributions are four-fold: (1) We carefully optimize the bitplane encoding and lossless encoding, two key stages in progressive methods, to achieve high performance on GPUs; (2) We propose pipeline optimization and incorporate it with data refactoring and progressive retrieval workflows to further enhance the performance for large data process; (3) We leverage our framework to enable high-performance data retrieval with guaranteed error control for common Quantities of Interest; (4) We evaluate HP-MDR and compare it with state of the arts using five real-world datasets. Experimental results demonstrate that HP-MDR delivers an average 13.68 × and 6.31 × throughput in data refactoring and progressive retrieval tasks, respectively. It also leads to 11.22 × throughput for recomposing required data representations under Quantity-of-Interest error control and 6.04 × performance for the corresponding end-to-end data retrieval, when compared with state-of-the-art solutions.
Li, Yanliang [University of Oregon]↗
Industrial Control Systems Network Protocol Parsers
Industrial Control Systems protocol parsers plugins for the Zeek network security monitoring framework. Currently we have four fully developed protocol parsers but we plan on adding more in the future. The protocol parsers we currently have developed are for BACnet, DNP3, Ethernet/IP, and Modbus.
Rasmussen, BrettD↗
End-to-End Differentiable Modeling and Management of the Environment
Focal Area: (2) Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization. We emphasize the importance of leveraging optimization techniques from AI/machine learning (ML) to solve challenging problems in Earth system modeling. Science Challenge: Automatic differentiation has had a transformative effect on ML by allowing the calculation of gradients of arbitrary functions in an incredibly large class of models. We can potentially realize similar improvements in parameter estimation and control for Earth system models (ESMs) by reimplementing them in computational frameworks from ML. Practitioners working with large (>10 7 parameters) models in ML and AI can obtain good predictive performance in a range of spatiotemporal tasks by making use of optimization via stochastic gradient descent and incorporating prior knowledge at multiple levels. We propose writing ESMs in open-source computational frameworks such as Torch, Tensorflow, and JAX to greatly expand the scope of environmental forecasting and management challenges, which can be addressed by leveraging automatic differentiation and gradient descent-like algorithms. We do not call for a wholesale replacement of physical models with data-driven surrogates, but rather advocate for interleaving physical and empirical equations in a manner that is most faithful to the extent of our scientific knowledge and observational data. Central to this topic is the merging of differentiable physical simulations with differentiable optimization layers, which are now both beginning to come to the forefront.
54 ENVIRONMENTAL SCIENCES↗
Underground Hydrogen Storage: Transforming Subsurface Science into Sustainable Energy Solutions
As the global economy moves toward net-zero carbon emissions, large-scale energy storage becomes essential to tackle the seasonal nature of renewable sources. Underground hydrogen storage (UHS) offers a feasible solution by allowing surplus renewable energy to be transformed into hydrogen and stored in deep geological formations such as aquifers, salt caverns, or depleted reservoirs, making it available for use on demand. This study thoroughly evaluates UHS concepts, procedures, and challenges. This paper analyzes the most recent breakthroughs in UHS technology and identifies special conditions needed for its successful application, including site selection guidelines, technical and geological factors, and the significance of storage characteristics. The integrity of wells and caprock, which is important for safe and efficient storage, can be affected by the operating dynamics of the hydrogen cycle, notably the fluctuations in pressure and stress within storage formations. To evaluate its potential for broader adoption, we also examined economic elements such as cost-effectiveness and the technical practicality of large-scale storage. We also reviewed current UHS efforts and identified key knowledge gaps, primarily in the areas of hydrogen–rock interactions, geochemistry, gas migration control, microbial activities, and geomechanical stability. Resolving these technological challenges, regulatory frameworks, and environmental sustainability are essential to UHS’s long-term and extensive integration into the energy industry. This article provides a roadmap for UHS research and development, emphasizing the need for further research to fully realize the technology’s promise as a pillar of the hydrogen economy.
25 ENERGY STORAGE↗