Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Agents”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,279 records · Page 71
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Temperature measurements in heavily-sooting ethylene/air flames using synchrotron x-ray fluorescence of krypton

High-fidelity temperature field measurements have been made for several heavily-sooting ethylene/air flames that have historically been challenging environments for conventional optical diagnostics. These challenges have largely been overcome here, in this study, by conducting x-ray fluorescence (XRF) measurements of a Kr fluorescent agent in the hard x-ray regime (15 keV). The current methodology presents a more economical diagnostic than a previously reported implementation of the Kr-XRF method, by limiting seeding of the expensive fluorescent agent to only the fuel stream flows. Detailed reacting flow simulations have been used to interpret experimental signals by tracking the mole fraction of the fluorescent agent in the flow field. Simulated Kr densities are in excellent agreement with measurements throughout the flow field. Temperature measurements of the flow field also agree well with simulations and recent literature studies. However, uncertainties in the measurements become increasingly large downstream of the burner surface as the krypton fraction drops due to mixing of the fuel and co-flow streams. Additionally, we demonstrate that soot particles in the heavily sooting flames studied do not impede the Kr-XRF measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Comprehensive assessment of deep reinforcement learning approaches for economic dispatch in nuclear-driven microgrids

As the electrical grid integrates more variable renewable energy sources such as wind and solar, the demand for distributed and flexible systems to address this increased variability becomes critical. Nuclear-driven microgrids provide a promising solution by offering stable generation to complement intermittent renewables, ensuring grid reliability and operating efficiency. This paper proposes a recurrent deep reinforcement learning framework for optimal economic dispatch in a nuclear-powered microgrid integrating renewable energy sources, small modular reactors, battery storage systems, and balance-of-plant dynamics. A three-agent control architecture is developed, where demand and renewable energy agents act as forecasters, and a reinforcement learning-based dispatch agent performs real-time energy allocation. A nonlinear programming formulation is first used to generate an optimal baseline for benchmarking. The proposed dispatch controller, based on Proximal Policy Optimization enhanced with Long Short-Term Memory networks, exploits temporal correlations in system dynamics by taking advantage of the time series used as inputs to improve policy robustness under uncertainty. Comparative analysis against established deep reinforcement learning methods, including Proximal Policy Optimization with a feedforward architecture, Soft Actor-Critic, and Twin Delayed Deep Deterministic Policy Gradient, demonstrates superior performance. Numerical results indicate that the proposed controller achieves a 0.39% cost reduction relative to the nonlinear programming benchmark and outperforms other learning-based methods by generating additional revenue of up to 0.35%. All reinforcement learning controllers compute dispatch actions in less than 0.3 s, resulting in a computational speedup of more than three orders of magnitude over the nonlinear programming baseline. The findings of this paper highlight their applicability for real-time operation and control in nuclear-integrated microgrids under volatile operating conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

How good are learning-based control v.s. model-based control for load shifting? Investigations on a single zone building energy system

Both model predictive control (MPC) and deep reinforcement learning control (DRL) have been presented as a way to approximate the true optimality of a dynamic programming problem, and these two have shown significant operational cost saving potentials for building energy systems. Furthermore, there is still a lack of in-depth quantitative studies on their approximation levels to the true optimality, especially in the building energy domain. To fill in the gap, this paper provides a numerical framework that enables the evaluation of the optimality levels of different controllers for building energy systems. This framework is then used to comprehensively compare the optimal control performance of both MPC and DRL controllers with given computation budgets for a single zone fan coil unit system. Note the optimality is estimated based on a user-specific selection of trade-off weights among energy costs, thermal comfort and control slew rates. Compared with the best optimality we can find through expensive optimization simulations, the best DRL agent can maximally approximate the optimality by 96.54%, which outperforms the best MPC whose optimality level is 90.11%. However, due to the stochasticity, the DRL agent is only expected to approximate the optimality by 90.42%, which is almost equivalent to the best MPC. Except for Proximal Policy Optimization (PPO), all DRL agents can have a better approximation to the optimality than the best MPC, and are expected to have better approximation than the MPC with a prediction horizon of 32 steps (15 min per step). In terms of reducing energy cost and thermal discomfort, MPC can outperform the rule-based control (RBC) by 18.47%–25.44%. DRL can be expected to outperform RBC by 18.95%–25.65% ,and the best DRL control policy can outperform RBC by 20.29%–29.72%. Although the comparison of the optimality level is performed in a perfect setting, e.g., MPC assumes perfect models, and DRL assumes a perfect offline training process and online deployment process, this can shed insight on their capabilities of approximating to the original dynamic programming problem.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Locating fast-varying line disturbances with the frequency mismatch

In an attempt to provide an efficient method for line disturbance identification in complex networks of diffusively coupled agents, we recently proposed to leverage the frequency mismatch. The frequency mismatch filters out the intricate combination of interactions induced by the network structure and quantifies to what extent the trajectory of each agent is affected by the disturbance. In this previous work, we provided some analytical evidence of its efficiency when the perturbation is assumed to be slow. Here we claim that the frequency mismatch performs actually well for most disturbance regimes. This is shown through a series of simulations and is backed up by an analytical argument. Therefore, we argue that the frequency mismatch is an efficient and elegant tool for line disturbance location in complex networks of diffusively coupled agents.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Fabrication and durability characterization of superhydrophobic and lubricant-infused surfaces

Hypothesis: Practical applications of non-wetting surfaces require good mechanical durability in the wet environments for which they are intended to be used. Durability of non-wetting surfaces is influenced by the surface features, interaction with the functionalization agent, and the lubricant properties that can be tuned independently to identify optimal combination. Experiments: In this study, superhydrophobic and lubricant-infused surfaces are fabricated on copper tubes using chemical etching and electrodeposition texturing techniques, six different functionalizing agents, and five different infused lubricants. Through 180 fabrication combinations and 102 durability tests, each parameter is systematically studied for contributions to initial non-wetting behavior and its durability in heated, wet environment, under high-energy water jet impingement, and under accelerated flow conditions. Findings: Among the adsorbing and curing functionalization agents investigated, n-Hexadecyl mercaptan that belongs to the sulfhydryl group and Sylgard-184, respectively, showed high durability in heated water immersion and under jet impingement tests. For lubricant-infused surfaces, lubricants with high surface tension demonstrated high durability in heated water immersion test, whereas durability in hydrodynamic conditions is closely correlated to lubricant viscosity. Results showed that a lubricant-infused surface will maintain its non-wetting properties in dropwise condensation conditions for approximately 1.5 years.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Enhancement of anticancer drug sensitivity in multidrug resistance cells overexpressing ATP-binding cassette (ABC) transporter ABCC10 by CP55, a synthetic derivative of 5-cyano-6-phenylpyrimidin

Highlights: • Multidrug resistance (MDR) in cancer is one of the major obstacles in chemotherapy. • ABCC10 mediates resistance to wide spectrum of chemotherapeutic agents, but only limited number of studies are available. • Overexpression of ABCC10 has been linked to various types of cancers including colorectal, gastric and lung cancer. • CP55 is a potent ABCC10 inhibitor which enhances the anticancer drug sensitivity in ABCC10-overexpression MDR cells. ATP-binding cassette (ABC) transporter C10 (ABCC10), also named multidrug resistance protein 7 (MRP7), is a member of ABC transporter superfamily and has been revealed to transport a wide range of chemotherapeutic agents including taxanes, epothilone B, Vinca alkaloids, and anthracyclines. In our previous study, a 5-cyano-6-phenylpyrimidin derivative CP55 was synthesized and found significantly reversal effect of multidrug resistance (MDR) mediated by ABCB1. In this study, we found CP55 also efficiently reversed MDR mediated by ABCC10. Our in vitro study showed that co-treatment with CP55 significantly increased the efficacy of ABCC10-substrate anticancer drugs in MDR cells overexpressing ABCC10. Furthermore, we showed that treatment with CP55 increased the intracellular accumulation of [{sup 3}H]-labeled anticancer drugs and in-turn decreasing drug efflux by inhibiting the transport activity, without altering ABCC10 protein ex-pression level or cellular localization. Potential CP55-ABCC10 interactions were predicted via docking analysis using human ABCC10 homology model and obtained high docking score. Therefore, CP55 represents a promising therapeutic agent in the combinational treatment of chemo-resistant cancer related to ABCC10.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

177 Lu and 227 Th Labeled 3,4,3-(LI-1,2-HOPO): characterization, optimized synthesis, biological distribution and dosimetry

Objectives: Interest in targeted radionuclide therapy has greatly increased namely due to the FDA approval of radiotherapeutic drugs such as Lutathera The identification of chelators that can bind diag- nostic and therapeutic radionuclides would allow facile development of agents tailored for personalized medicine. Hydroxypyridinonate (HOPO) derivatives have been demonstrated as effective decorpo- ration agents for actinides owing to their hard oxygen donor atoms and strong complexation. Utilizing these molecules as chelators for potential nuclear medicine agents should allow for radiometals to be effectively sequestered by the ligand. The aim of this work was to evaluate the in vivo stability of 3-4-3-(LI-1,2-HOPO) (HOPO for short) with 177 Lu and 227 Th as potential beta- and alpha-emitting radiothera- peutic complexes, respectively.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2025

Rapid Crystallization of Zeolites with Controllable Defects: Disentangling Fluoride Concentration and pH Using NH 4 F

Zeolite synthesis is typically conducted either under basic conditions or in neutral fluoride media using hydrofluoric acid (HF). While basic (OH − ) conditions generally result in faster zeolite crystallization, they can also increase the likelihood of framework defects and crystal intergrowths. In contrast, synthesis in neutral fluoride media tends to produce fewer defects because fluoride balances positive charges from structure-directing agents. However, this method often requires significantly longer crystallization times and involves the handling of dangerous HF. In the present study, we pursue the best of both synthesis conditions, rapid syntheses with controllable defect concentrations, by disentangling of mineralizing agent and chargebalancing agent using ammonium fluoride (NH 4 F) as an alternative to HF. We have investigated the use of NH 4 F in the syntheses of siliceous and aluminum-containing zeolite A (LTA, small pore), ZSM-5 (MFI, medium pore), and siliceous Beta(*BEA, large pore). The crystallization times of all four zeolites decreased substantially with an increasing NH 4 F concentration. Crystallization times were reduced from 24 to 4 h (Si-LTA), 96 to 36 h (Al-LTA), 240 to 6 h (ZSM-5), and 24 to 3 h (Si-*BEA). Additionally, increasing the NH 4 F concentration in the synthesis mixtures decreases the defect densities of siliceous zeolites. Raman spectroscopy, along with 29 Si MAS NMR, 19 F MAS NMR, 13 C MAS NMR, and fluorine elemental analysis of Si-LTA samples confirms that the reduction in charged defects (Si−O − ) is due to the higher incorporation of F − within the double four-membered ring (D4R) present in the LTA samples. We show that the accelerated crystallization is due to the role of F − in enhancing the silica mineralization rate (formation of silicon hexafluoride species) and stabilizing D4Rs under basic conditions. As a result, combining basic and fluoride-mediated synthesis could therefore be advantageous for faster zeolite production and improved control over structural properties for a wide variety of zeolite structures.

Anions↗
DOE OSTI2025

Probing the Redox Reactivity of Alkyl Bound Astatine: A Study on the Formation and Cleavage of a Stable At–C Bond

The formation of a stable alkyl At–C bond occurs during the shipment of 211 At on a 3-octanone-impregnated column and the reactivity of 211 At stripped from columns has been studied. The 211 At could not be recovered from the 3-octanone organic phase using nitric acid or sodium hydroxide, even up to 10 and 15.7 M, respectively. Several reducing and oxidizing agents, including hydrazine, hydroxylamine, ascorbic acid, ceric ammonium nitrate, potassium permanganate, sodium hypochlorite, and calcium hypochlorite were used to promote the recovery of 211 At. The most effective reducing agent was hydroxylamine, where ~70% of the 211 At was recovered, while among oxidizing agents ceric ammonium nitrate, potassium permanganate, and sodium hypochlorite all showed near quantitative recovery of 211 At. These results indicate an At–C bond is being formed during the shipment of the column and a redox reaction is required for bond cleavage to occur. Furthermore, DFT calculations have been used to propose several products of an AtO + -3-octanone reaction, with 4-astato-5-hydroxy-octa-3-one being the most probable.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Characterization of Lanthanum Monazite Surface Chemistry and Crystal Morphology through Density Functional Theory and Experimental Approaches

Monazite is a rare earth element (REE)-containing mineral that consists of (REE)PO 4 formal units and is one of the most important sources of these critical materials. The concentration of REEs from mined monazite ore often involves froth flotation, which is a beneficiation process that enhances the efficiency of downstream processing. The effectiveness of froth flotation is largely governed by the ability of collector agents to selectively bind to monazite particles. Thus, a molecular-level understanding of monazite interfacial chemistry is integral to the design of effective collector agents. To address this need, we performed density functional theory (DFT) calculations and a variety of experimental techniques to characterize La-monazite and elucidate its crystal morphology. Interestingly, we find minimal differences in the predicted morphologies of La-monazite for hydrous and anhydrous environments, which are largely dominated by low-index facets (e.g., {110}, {100}, and {010}). Indexing of synthesized La-monazite crystals via X-ray diffraction also uncovers {110} and {100} as the predominant facets. The average surface energies of 0% and 100% water coverage La-monazite crystals were predicted to be 0.87 and 0.76 J/m 2 , respectively, while calorimetry suggests values of 1.30 and 1.15 J/m 2 , respectively. The apparent discrepancies between the theoretical and experimental values are expected and attributed to defects present in physical crystals, in contrast to the perfect mineral surfaces in simulations. The difference in surface energy between the 0% and 100% water coverage morphologies predicted by theory is consistent with the value measured via calorimetry. DFT reveals a wide range of adsorption energies for water across the studied facets, but in all cases, water is predicted to strongly bind to monazite surfaces with an average adsorption energy of -92.7 kJ/mol for a La-monazite single crystal. Finally, this study provides the groundwork necessary for the rational design of froth flotation collector agents by granting molecular-level insight into the predominant facets of monazite.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Partitioning of Seven Different Classes of Antibiotics into LPS Monolayers Supports Three Different Permeation Mechanisms through the Outer Bacterial Membrane

The outer membrane (OM) of Gram-negative (G-) bacteria presents a barrier for x classes of antibacterial agents. Lipopolysaccharide (LPS), present in the outer leaflet of the OM, is stabilized by divalent cations and is considered to be the major impediment for antibacterial agent permeation. However, the actual affinities of major antibiotic classes toward LPS have not yet been determined. In the present work, we use Langmuir monolayers formed from E. coli Re and Rd types of LPS to record pressure-area isotherms in the presence of antimicrobial agents. Our observations suggest three general types of interactions. Firstly, some antimicrobials demonstrated no measurable interactions with LPS. This lack of interaction in the case of cefsulodin, a third-generation cephalosporin antibiotic, correlates with its low efficacy against G- bacteria. Ampicillin and ciprofloxacin also show no interactions with LPS, but in contrast to cefsulodin, both exhibit good efficacy against G- bacteria, indicating permeation through common porins. Secondly, we observe substantial intercalation of the more hydrophobic antibiotics novobiocin, rifampicin, azithromycin, and telithromycin into relaxed LPS monolayers. These largely re-partition back to the subphase with monolayer compression. We find that the hydrophobic area, charge, and dipole all show correlations with both the mole fraction of antibiotic retained in the monolayer at the monolayer-bilayer equivalence pressure and the efficacies of these antibiotics against G- bacteria. Thirdly, amine-rich gentamicin and the cationic antimicrobial peptides polymyxin B and colistin show no hydrophobic insertion but are instead strongly driven into the polar LPS layer by electrostatic interactions in a pressure-independent manner. Their intercalation stably increases the area per molecule (by up to 20%), which indicates massive formation of defects in the LPS layer. These defects support a self-promoted permeation mechanism of these antibiotics through the OM, which explains the high efficacy and specificity of these antimicrobials against G- bacteria.

lateral pressure↗
DOE OSTI2024

Carbon-13 Radiofrequency Amplification by Stimulated Emission of Radiation of the Hyperpolarized Ketone and Hemiketal Forms of Allyl [1- 13 C]Pyruvate

13 C hyperpolarized pyruvate is an emerging MRI contrast agent for sensing molecular events in cancer and other diseases with aberrant metabolic pathways. This metabolic contrast agent can be produced via several hyperpolarization techniques. Despite remarkable success in research settings, widespread clinical adoption faces substantial roadblocks because the current sensing technology utilized to sense this contrast agent requires the excitation of 13 C nuclear spins that also need to be synchronized with MRI field gradient pulses. Here, we demonstrate sensing of hyperpolarized allyl [1- 13 C]pyruvate via the stimulated emission of radiation that mitigates the requirements currently blocking broader adoption. Specifically, 13 C Radiofrequency Amplification by Stimulated Emission of Radiation ( 13 C RASER) was obtained after pairwise addition of parahydrogen to a pyruvate precursor, detected in a commercial inductive detector with a quality factor (Q) of 32 for sample concentrations as low as 0.125 M with 13 C polarization of 4%. Moreover, parahydrogen-induced polarization allowed for the preparation of a mixture of ketone and hemiketal forms of hyperpolarized allyl [1- 13 C]pyruvate, which are separated by 10 ppm in 13 C NMR spectra. This is a good model system to study the simultaneous 13 C RASER signals of multiple 13 C species. This system models the metabolic production of hyperpolarized [1- 13 C]lactate from hyperpolarized [1- 13 C]pyruvate, which has a similar chemical shift difference. Our results show that 13 C RASER signals can be obtained from both species simultaneously when the emission threshold is exceeded for both species. On the other hand, when the emission threshold is exceeded only for one of the hyperpolarized species, 13 C stimulated emission is confined to this species only, therefore enabling the background-free detection of individual hyperpolarized 13 C signals. Furthermore, the reported results pave the way to novel sensing approaches of 13 C hyperpolarized pyruvate, potentially unlocking hyperpolarized 13 C MRI on virtually any MRI system–an attractive vision for the future molecular imaging and diagnostics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Force-Triggered, Biobased Sealants for Prefabricated Building Components: Toward Improved Efficiency and Performance

The prefabricated building construction industry has made extensive progress in expediting the manufacture of prefabricated components at off-site plants. However, the sealing of joints between these components, which is crucial to ensuring the weatherproofing of the assembly, still represents a labor-intensive, on-site effort that relies on the manual installation of tapes and caulks. Here, to reduce work at the jobsite and improve the airtightness and waterproofness of building envelopes, we developed a sealant that can be installed at the plant on prefab components and have the curing reaction triggered at the jobsite by using microencapsulation technology to separate the reactive agents. A series of force-triggered, high-strength, and fast-curing sealants derived from biobased feedstocks were developed, which consist of a biobased epoxy agent encapsulated in a polymer shell, embedded in a biobased amine curing agent. The shell of the microcapsules allows an effective separation of the reactive species in the one-part sealant, allowing shelf stability to an otherwise fast-curing system as well as improving the hydrophobicity of the whole system. When force activates and breaks the microcapsules, the highly reactive epoxy and amine mix and cure, exhibiting peel strength values of up to 143 ppi (pounds per inch). The hydrophobicity of the sealants allows them to retain up to 94% of the original peel strength after complete submersion in water for 24 h, showcasing the water resistivity of the sealant system. The open-air shelf stability of the sealant complex is demonstrated by the obtention of peel strength values of ∼16 ppi when triggering the curing reaction even after being exposed 8 months to open air and humidity. The successful on-demand triggering of curing reactions and the shelf stability provide efficacy of these force-triggered sealants for installation on prefabricated components, storage for months prior to delivery, and assembly at a jobsite. These force-triggered biobased sealants for prefabricated buildings can result in lower installation time and cost and better performance than tapes and caulks at the jobsite.

biobased↗
DOE OSTI2023

Explaining Forcing Efficacy With Pattern Effect and State Dependence

The magnitude of global surface temperature change in response to unit radiative forcing depends on the type and magnitude of forcing agent—a concept known as a “forcing efficacy.” However, the mechanisms behind the forcing efficacy are still unclear. In this study, we perform a set of simulations using CESM1 to calculate the efficacy of 10 different forcing agents defined in terms of fixed-SST effective radiative forcing, and then use a Green's function approach to show that each forcing efficacy can be largely understood in terms of the radiative feedbacks associated with the different surface temperature patterns induced by the forcing agents (a pattern effect). We also quantify how the state dependence of feedbacks on global mean surface temperature anomalies impacts forcing efficacies. The results show that the forcing efficacy can be well reconstructed with a combination of pattern effect and state dependence.

58 GEOSCIENCES↗
DOE OSTI2021

Adsorption of ethylenediamine on Cu surfaces: attributes of a successful capping molecule using first-principles calculations

The shape-controlled synthesis of Cu nanocrystals can benefit a wide range of applications, though challenges exist in achieving high and selective yields to a particular shape. Capping agents play a pivotal role in controlling shape, but their exact role remains ambiguous. In this study, the adsorption of ethylenediamine (EDA) on Cu(100) and Cu(111) was investigated with quantum density functional theory (DFT) to reveal the complex roles of EDA in promoting penta-twinned Cu nanowire growth. We find EDA has stronger binding on Cu(100) than on Cu(111), which agrees the general expectation that penta-twinned Cu nanowires express facets with stronger capping-molecule binding. Despite this stronger binding, ab initio thermodynamics reveals the surface energy of EDA-covered Cu(111) is lower than that EDA-covered Cu(100) at all solution-phase EDA chemical potentials, so there is no thermodynamic driving force for penta-twinned nanowires. We also investigated the capability of EDA to protect Cu surfaces from oxidation in water by quantifying energy barriers for a water molecule to diffuse through EDA layers on Cu(100) and Cu(111). The energy barrier on Cu(100) is significantly lower, which supports observations of faster oxidation of Cu(100) in electrochemical experiments. Thus, we elucidate another possible function of a capping agent – to enable selective oxidation of crystal facets. This finding adds to the general understanding of successful attributes of capping agents for shape-selective nanocrystal growth.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Opinion dynamics in financial markets via random networks

We investigate financial market dynamics by introducing a heterogeneous agent-based opinion formation model. In this work, we organize individuals in a financial market according to their trading strategy, namely, whether they are noise traders or fundamentalists. The opinion of a local majority compels the market exchanging behavior of noise traders, whereas the global behavior of the market influences the decisions of fundamentalist agents. We introduce a noise parameter, q , to represent the level of anxiety and perceived uncertainty regarding market behavior, enabling the possibility of adrift financial action. We place individuals as nodes in an Erdös-Rényi random graph, where the links represent their social interactions. At any given time, individuals assume one of two possible opinion states ±1 regarding buying or selling an asset. The model exhibits fundamental qualitative and quantitative real-world market features such as the distribution of logarithmic returns with fat tails, clustered volatility, and the long-term correlation of returns. We use Student’s t distributions to fit the histograms of logarithmic returns, showing a gradual shift from a leptokurtic to a mesokurtic regime depending on the fraction of fundamentalist agents. Furthermore, we compare our results with those concerning the distribution of the logarithmic returns of several real-world financial indices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Gaming the beamlines—employing reinforcement learning to maximize scientific outcomes at large-scale user facilities

Abstract Beamline experiments at central facilities are increasingly demanding of remote, high-throughput, and adaptive operation conditions. To accommodate such needs, new approaches must be developed that enable on-the-fly decision making for data intensive challenges. Reinforcement learning (RL) is a domain of AI that holds the potential to enable autonomous operations in a feedback loop between beamline experiments and trained agents. Here, we outline the advanced data acquisition and control software of the Bluesky suite, and demonstrate its functionality with a canonical RL problem: cartpole. We then extend these methods to efficient use of beamline resources by using RL to develop an optimal measurement strategy for samples with different scattering characteristics. The RL agents converge on the empirically optimal policy when under-constrained with time. When resource limited, the agents outperform a naive or sequential measurement strategy, often by a factor of 100%. We interface these methods directly with the data storage and provenance technologies at the National Synchrotron Light Source II, thus demonstrating the potential for RL to increase the scientific output of beamlines, and layout the framework for how to achieve this impact.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Explainable physics-based constraints on reinforcement learning for accelerator optimization

We present a reinforcement learning (RL) framework for optimizing particle accelerator experiments that builds explainable physics-based constraints on agent behavior. The goal is to increase transparency and trust by letting users verify that the agent’s decision-making process incorporates suitable physics. Our algorithm uses a learnable surrogate function for physical observables, such as energy, and uses them to fine-tune how actions are chosen. This surrogate can be represented by a neural network or by an interpretable sparse dictionary model. We test our algorithm on a range of particle accelerator optimization environments designed to emulate the Continuous Electron Beam Accelerator Facility at Jefferson Lab. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment. In addition, we find that the introduction of a physics-based surrogate enables our RL algorithms to reliably converge for difficult high-dimensional accelerator optimization environments.

explainability↗