SEARCH · Engineering Papers
Results for “deep reinforcement learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Deep RL for Fast Long-Horizon Operations Scheduling on NASA's Carruthers Geocorona Observatory Mission
Spacecraft operations scheduling is a highly constrained, long-horizon combinatorial optimization problem that traditionally relies on heuristics, constraint programming, or manual planning. We present a scalable deep reinforcement learning framework developed and deployed for NASA’s Carruthers Geocorona Observatory mission. Our framework introduces a macro-action abstraction known as activity blocks coupled with dynamic action-masking to navigate the intractably large search space and strictly enforce complex power, thermal, and instrument constraints. The resulting architecture generates globally feasible schedules with overwhelming probability, establishes operational trust, and executes a full training cycle in under six hours, circumventing the need for policy robustness by enabling rapid, on-demand retraining. Further, resulting schedules outperform baseline heuristics in scheduled science quality. The deep reinforcement learning framework was deployed as the default operational scheduler for the Carruthers Geocorona Observatory mission from the outset of the mission, demonstrating that deep reinforcement learning can be trusted for real spacecraft operations under complex, evolving constraints.
User-Centric Communication With Aerial Network for 6G: A Reinforcement Learning Approach
Meeting the diverse needs of user verticals requires innovative cellular architectures that can offer additional degrees of freedom to provide on-demand services. The terrestrial user-centric radio access network (UC-RAN) stands out as an excellent choice for this purpose. However, a drawback of UC-RAN is its tendency to prioritize high-priority verticals, often resulting in a subpar quality of experience for low-priority verticals. This issue is particularly exacerbated in hotspot areas. Here, to address this problem, we introduce an aerial network integrated with terrestrial UC-RAN to provide coverage to users which are not served by the terrestrial network. Furthermore, we analyze the impact of key configuration and optimization parameters (COPs), such as location, transmit power, altitude, and beamwidth of aerial base stations (ABSs) on system key performance indicators (KPIs), such as coverage, latency satisfaction, average spectral efficiency, and energy efficiency. We formulate a robust multiobjective function to maximize these KPIs without biasing toward any specific KPI(s). Finally, we propose a deep reinforcement learning optimization framework based on the state-of-the-art soft actor-critic algorithm to control ABS COPs and optimize system KPIs. Experimental evaluations demonstrate that the proposed optimization framework can converge to near-optimal solutions derived from the pseudo brute force in a few thousand epochs.
Optimization of the FRIB beam dump: a hybrid genetic algorithm and reinforcement learning approach
The operational envelope of high-power-density systems, such as particle accelerators and advanced nuclear energy systems, is critically constrained by the need to manage extreme thermal loads. To address this, we present a novel hybrid optimization framework combining a genetic algorithm (GA) with a soft actor-critic (SAC) deep reinforcement learning agent. This framework was applied to a practical high-heat-flux problem: redesigning the beam dump at the Facility for Rare Isotope Beams (FRIB) for a power upgrade from 20 kW to 50 kW. The resulting design, validated by three-dimensional conjugate heat transfer simulations, suppresses hazardous hot spots and yields a markedly more uniform temperature distribution. This provides a robust operating margin, increasing the average power-handling capability by 72% relative to the current design, demonstrating the framework’s potential to solve complex thermal management challenges in both accelerator technology and advanced nuclear systems.
Micrometer: Micromechanics transformer for predicting full field mechanical responses of heterogeneous materials
Predicting mechanical responses of heterogeneous materials across scales remains a significant challenge. Traditional computational methods often struggle with complex and multiscale nature of these materials, limiting their effectiveness in real-world applications. Here, in this paper, we introduce Micrometer, a vision transformer based deep learning model designed to predict full field mechanical responses of heterogeneous materials, bridging the gap between computer vision and solid mechanics problems. We show that Micrometer, trained on a large-scale high-resolution dataset of 2D fiber-reinforced composites, can achieve state-of-the-art performance in predicting microscale strain fields across a wide range of material properties and loading conditions. Our model demonstrates accuracy and computational efficiency in applications such as computational homogenization and multiscale modeling, reducing computational time by up to two orders of magnitude compared to conventional numerical solvers while maintaining less than 1 % errors in predicting macroscale stress fields. Furthermore, we showcase Micrometer’s adaptability through transfer learning experiments on new materials with limited data, highlighting its potential to tackle diverse scenarios in computational solid mechanics. These results represent a significant step towards AI-driven innovation in materials science, addressing the limitations of traditional numerical methods and paving the way for more efficient simulations of heterogeneous materials across various industrial applications.
Reinforcement learning framework for the mechanical design of microelectronic components under multiphysics constraints
Explore the source record for details and available documents.
Infinite Virtual Testing Using Physics-informed Deep Learning
Explore the source record for details and available documents.
Spatio-temporal, multi-field deep learning of shock propagation in meso-structured media
Explore the source record for details and available documents.
Unveiling the lithium-ion transport mechanism in Li{sub 2}ZrCl{sub 6} Solid-State Electrolyte {ital via} deep learning-accelerated molecular dynamics simulations.
Lithium zirconium chlorides (LZCs) present a promising class of cost-effective solid electrolytes for next-generation all-solid-state batteries. The unique crystal structure of LZCs plays a crucial role in facilitating lithium-ion mobility, which further affects the electrochemical performance. To understand the underlying mechanism governing ion transport, we employed deep learning-accelerated molecular dynamics simulation on Li2ZrCl6 (trigonal alpha- and monoclinic beta-LZC), focusing specifically on the zirconium coordination environment. Our results reveal that disordered alpha-LZC exhibits the highest ionic conductivity, while beta-LZC demonstrates significantly lower conductivity, closely aligning with experimental findings. The study confirms that across all phases, lithium migration proceeds via the site-to-site hopping mechanism, where variations in site residence times critically impact the overall ionic conductivity. In alpha-LZCs, lithium ions prefer to anisotropically diffuse across interlayers as the result of a lower energy barrier, driven primarily by collective diffusion. In contrast, lithium ions in beta-LZC primarily isotropically diffuse within the intralayer, hindered by higher energy barriers and determined by individual diffusion. The variation in ZrCl6 2- octahedral unit softening, induced by the specific layered arrangement of zirconium atoms, emerges as a critical determinant of the energy barriers across the LZC phases. These atomic-scale insights into the transport processes provide valuable guidance for the rational design and optimization of LZCs-based electrolytes, accelerating their practical application in advanced energy storage technologies.
Physics-Guided Deep Learning for Complex System Health Management and Decision Making
The landscape of complex engineered systems is rapidly evolving, from smart manufacturing facilities to next-generation transportation vehicles. As these systems become increasingly sophisticated and interconnected, the need for advanced health management systems grows ever more critical. These systems must go beyond simple monitoring, actively predicting potential failures before they occur. This paradigm shift from fixed maintenance schedules to condition-based predictions is key to optimizing system performance, enhancing safety, and paving the way for autonomous decision-making across various industries. Whether in industrial processes, energy systems, or advanced transportation, the ability to anticipate and prevent failures is becoming a cornerstone of operational excellence. To accurately predict the future health of any complex system, knowledge of its current health state and future operational conditions is essential. Recent advancements in data-driven algorithms have generated growing interest in artificial intelligence for industrial applications. However, the limitations of pure data-driven methods, particularly in industries where data acquisition is costly and limited, have become apparent. This has led to a focus on blending physics with data-driven algorithms, mitigating the drawbacks of both approaches while emphasizing their respective advantages. This research proposes a novel framework for integrating physics-based performance models with deep learning algorithms for the prognostics of complex safety-critical systems. In this approach, physics-based models serve as a blueprint, capturing fundamental system behaviors, while deep learning algorithms, leveraging real-world sensor data, fill in gaps and identify subtle patterns indicative of potential problems. This hybrid methodology, utilizing techniques such as Physics-Informed Neural Networks (PINNs), offers a powerful solution for predicting system health. By fusing domain knowledge with data-driven insights, this approach promises more accurate, adaptable, and reliable models for health prediction. The resulting framework is versatile, applicable across various sectors including aerospace, manufacturing, and energy systems, ultimately contributing to safer, more efficient operations in our increasingly complex technological landscape.
Concerted Electron-Ion Transport by Polyacrylonitrile Elucidated with Reactive Deep Learning Potentials
Charge transport in polymers, such as polyacrylonitrile (PAN), is crucial for electronics and energy storage. For instance, PAN can transport cations e.g., Li + , by facilitating dynamic cation-nitrile coordination in batteries. However, little is known regarding the underlying role of complex reactive polymer configurations. Herein, we develop a deep-learning potential, trained on ab initio energies and forces of nonequilibrium reactive PAN configurations, to unravel the kinetics of PAN cyclization initiated by a nucleophile (OH – dissociated from LiOH) attacking the terminal nitrile carbon. We find, based on the reaction free-energetics, rates, and charge analysis, that the nucleophile attack producing the first ring is the rate-limiting step, which subsequently triggers Li + -coupled electron transfer along the PAN backbone, causing ∼10 4 times faster sequential ring-formation of the remaining nitriles. PAN’s extended configurations, where dipolar and H-bonding interactions are minimal, enable such rapid kinetics. By validating our computational findings with IR and NMR experiments, we establish a pathway for designing reactive polymers with enhanced charge transport for energy applications.
PERSIANN-Unet: A Global Deep Learning Framework for Near-Real-Time Precipitation Estimation Using Infrared Data
Access to high-quality, high-resolution, near-real-time precipitation data is essential for hydrological and meteorological research and disaster mitigation. Traditional tools such as rain gauges and radar networks, though effective, have limitations, including sparse coverage in remote areas and high operational costs. Satellite data, with its global coverage and high spatial and temporal resolutions, mitigates limitations in coverage. Satellite precipitation products like Hydro Estimator (HE), Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (IMERG), and Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) utilize both geosynchronous thermal infrared (IR) and passive microwave (PMW) data in their operation. PMW sensors offer detailed atmospheric profiles but suffer from higher latency, whereas IR sensors provide lower latency but only capture cloud-top information. Despite this constraint, IR data remains attractive for low-latency precipitation estimation. Recent advances in deep learning, particularly convolutional neural networks (CNNs), have further improved satellite precipitation retrievals. This study introduces PERSIANN-Unet (PUnet or PERSIANN V3), a quasi-global algorithm covering 60°N–60°S that combines IR data, monthly climatology, and the UNet architecture to produce half-hourly precipitation estimates at 0.04° resolution. The product is evaluated against HE, IMERG, and PDIR-Now for 2022–2023. Results show that PUnet closely matches its training target, IMERG V07 Final, at the global scale, and performance is further evaluated against Stage IV as a reference over CONUS. Training PUnet on IMERG (2016–2021) leverages a high-quality, integrated PMW IR-gauge precipitation product while developing an IR-based framework not reliant on PMW availability. By operating on a single global image, PUnet avoids tile partitioning and blending steps, reducing edge discontinuities, and produces more spatially consistent precipitation fields across hemispheres.
Anticipating Optical Availability in Hybrid RF/FSO Links Using RF Beacons and Deep Learning
Radiofrequency (RF) communications offer reliable but low data rates and energy-inefficient satellite links, while free-space optical (FSO) promises high bandwidth but struggles with disturbances imposed by atmospheric effects. A hybrid RF/FSO architecture aims to achieve optimal reliability along with high data rates for space communications. Accurate prediction of dynamic ground-to-satellite FSO link availability is critical for routing decisions in low-earth orbit constellations. In this paper, we propose a system leveraging ubiquitous RF links to proactively forecast FSO link degradation prior to signal drops below threshold levels. This enables pre-calculation of rerouting to maximally maintain high data rate FSO links throughout the duration of weather effects. We implement a supervised learning model to anticipate FSO attenuation based on the analysis of RF patterns. Through the simulation of a dense lower earth orbit (LEO) satellite constellation, we demonstrate the efficacy of our approach in a simulated satellite network, highlighting the balance between predictive accuracy and prediction duration. An emulated cloud attenuation model is proposed to provide insight into the temporal profiles of RF signals and their correlation to FSO channel dynamics. Our investigation sheds light on the trade-offs between prediction horizon and accuracy arising from RF beacon numbers and proximity.
Deep Learning and Photogrammetric Reconstruction for Automated Crack Detection and Dimensional Measurement in Mining Operations
Surface crack detection and dimensional measurement at active mining sites present significant safety and operational challenges. Manual inspection methods are labor-intensive, spatially incomplete, and expose personnel to hazardous environments, while existing automated approaches have been developed primarily for concrete civil infrastructure and have not been validated on the complex, variable surfaces characteristic of mining environments. This dissertation presents an automated pipeline that integrates deep learning semantic segmentation with Structure-from-Motion photogrammetry to detect surface cracks and measure their aperture, length, and vertical displacement from standard RGB imagery acquired during routine Uncrewed Aerial Vehicle (UAV) survey operations, without requiring additional sensor hardware or manual measurement. The pipeline combines a U-Net architecture with an EfficientNet-B0 encoder, pretrained on the SDNET2018 concrete crack dataset and fine-tuned on a mining-specific dataset spanning laboratory concrete specimens, coal refuse impoundment embankments, and post-blast limestone quarry benches. Photogrammetric reconstruction is performed using COLMAP Structure-from-Motion and Multi-View Stereo, with crack segmentation masks projected into the reconstructed point cloud to enable three-dimensional vertical displacement measurement through local plane fitting and bimodal surface detection. The pipeline was validated across 36 controlled laboratory specimens at three imaging distances and four vertical displacement levels, achieving aperture measurement RMSE of 0.047 cm and R² of 0.954, and vertical displacement RMSE of 0.140 cm and R² of 0.966, against independent caliper measurements. Field application at a coal refuse impoundment in southwestern Pennsylvania detected 71 crack components across the embankment crest, with a dominant longitudinal crack exhibiting aperture values reaching 28 cm and a 95th percentile vertical displacement of 35.53 cm, consistent in magnitude and spatial distribution with simultaneously acquired LiDAR-derived estimates. Application across four post-blast limestone quarry bench datasets in California successfully characterized blast-induced fracture networks at ground sampling distances ranging from 0.59 to 1.23 cm/pixel, with detected crack geometries physically consistent with observable surface conditions at each site. The results demonstrate that deep learning-based crack detection and photogrammetric measurement can be integrated into routine UAV inspection workflows at mining sites, providing repeatable, scalable, and quantitative crack characterization across surface types, crack scales, and displacement magnitudes not previously addressed in the literature. The pipeline requires no dedicated surveying equipment beyond the UAV platforms already deployed at mine sites for survey and monitoring purposes, supporting practical adoption within existing operational workflows.
Manufacturing Process Development of a Carbon Fiber Reinforced Polymer Composite Shaft for Electric Motors
Electric aircraft applications require electric motors with increased specific power and efficiency. Composite structural components in motors are a potential solution for reducing motor mass, reducing magnetic losses, and limiting undesired conduction paths for fault, electromagnetic interference, or common-mode currents. In this report, manufacturing trials for a high-speed carbon fiber reinforced polymer composite motor shaft are presented. Four prototype shafts were produced using a hybrid biaxial/triaxial fabric that was circumferentially wrapped onto an additively manufactured high-temperature washout mandrel. An additional traditional overbraid approach was also evaluated and shows promise for high-rate, high-performance parts using automated manufacturing. This paper discusses the shaft design, manufacturing methods explored, material selection, the manufacturing trials, and the lessons learned. The results of this manufacturing investigation show feasibility for manufacturing composite shafts for electric motors.
Feedback, physics, and forecasts: The emerging paradigm of machine learning-driven battery research
Machine learning (ML) is reshaping how we understand, predict, and optimize electrochemical systems. In batteries, ML accelerates discovery across chemistry, design, and operation by transforming massive experimental and simulated datasets into predictive, interpretable models. This review consolidates a decade of progress in ML-driven battery innovation, from early-cycle feature extraction to operando image analysis and physics-informed modeling. We categorize approaches by data domain and physical fidelity, emphasizing interpretable ML for diagnostics, reinforcement learning for control, and multi-objective optimization for lifetime extension strategies. Additionally, we demonstrate how integrated models accelerate discovery, reduce testing time, and guide sustainable design. Economic analyses furthermore illustrate how these advances can lower cost per cycle and improve circularity. Together, these developments chart a path toward self-optimizing, sustainable battery technologies.
Large Language Model for Validation, Optical Calibration, and Learning (VOCAL) Distributed Temperature Sensing Interface
Distributed temperature sensing (DTS) using fiber optic sensors (FOS) offers a promising method for temperature measurements in advanced reactors, such as sodium fast reactors and molten salt cooled reactors. To support the calibration and validation of DTS measurements, Argonne National Laboratory developed the Validation, Optical Calibration, and Learning (VOCAL) software package. This report describes the integration of a local large language model (LLM) with a retrieval-augmented generation (RAG) system into the VOCAL interface to serve as an interactive user assistant. The LLM framework enhances the VOCAL platform’s accessibility to users by explaining interface components, clarifying inputs and outputs, and answering user queries dynamically in real-time. The accuracy of the LLM assistant performance was evaluated with 20 queries regarding the interface and its parameters using experimental data from the Thermal Hydraulic Experimental Test Article (THETA) facility. Results demonstrate that the LLM achieved a 95% accuracy rate, with a BERTScore of 0.8816 and SBERT value of 0.7417. Furthermore, validation of the RAG system within the LLM framework showed optimal accuracy with k-values between 1 and 2 using the k-refinement convergence test. The prompt perturbation analysis demonstrated good initial consistency for the RAG system, exhibiting the highest accuracy under punctuation variations and the greatest sensitivity under query reordering. Notably, the model’s errors were limited to data retrieval failures rather than factual hallucinations, reinforcing its baseline reliability. The integration of LLM provides a highly accurate, userfriendly enhancement to the VOCAL platform without disrupting its core computational capabilities for FOS calibration and validation.
In-Situ Scanning Electron Microscope Experiments for Microscale Mechanical Testing and Validated Modeling of Fiber Reinforced Thermoplastics
A novel, in-situ, scanning electron microscope (SEM) mechanical testing capability for materials at the microscale which provides experimental validation to a machine learning (ML) toolset for full-field validation of physics-based micromechanics models is being developed by researchers at NASA Glenn Research Center. These are enabling technologies for the integration of multiscale digital twins for materials into system level models which will result in the improved performance, material discovery, reduced production cost and time, rapid characterization, and prognostic structural health monitoring (SHM) for materials and structures for extreme environments in support of NASA space exploration missions. In order to bridge the material structure-to-system gap for digital twins, physics-based models must be experimentally validated at multiple length scales. Seminal microscale experiments, conducted at the Air Force Research Laboratory (AFRL), were limited to transverse compression of single-layer, unidirectional thermoset polymer matrix composite (PMC) micropillar specimens [1]. The early phases of the current project followed those initial results and setup to reproduce the compression testing of PMC material on the custom-built piezoelectric actuated micromechanical testing rig built by MicroTesting Solutions LLC. In this work, samples of thermoplastic PMC material were first machined into 3 mm cubes, and then further machining and final milling was done using a Focused Ion Beam (FIB). The initial experiment was done on a pillar roughly 20 µm x 20 µm x 40 µm tall. Additional pillars were milled with final sizes ranging from 20 µm x 20 µm x 40 µm tall to 40 µm x 40 µm x 65 µm tall. A speckle pattern for in-situ full-field measurements using Digital Image Correlation (DIC) was applied with platinum, which was coated on the surface, and then the FIB was used to mill away some of the coating to produce an irregular pattern of Pt on the pillar surface. The samples were loaded into the custom testing rig and placed into the SEM and loaded under compression until failure. Images were collected in the SEM during testing. Post-processing of the images was conducted using DIC to obtain full-field displacement and strain measurements elucidating the role of the matrix as well as fiber-fiber interaction at the microscale within the composite subjected to compression loading well into the non-linear regime of the material. Moreover, the evolution of fiber-matrix debonding and matrix cracking is observed in-situ at the microscale. This data, along with images segmented with a newly developed ML toolset [2], was used to create and validate physics-based micromechanics models. An image of the failed micropillar is shown in Figure 1. The techniques developed in the initial compression experiment was tailored to the validation needs of the models and expanded to include different sized samples as well as possibly tension and fatigue.