Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Structural error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Nature and Consequences of Biological Reductionism for the Immunological Study of Infectious Diseases

Evolution has conserved “economic” systems that perform many functions, faster or better, with less. For example, three to five leukocyte types protect from thousands of pathogens. In order to achieve so much with so little, biological systems combine their limited elements, creating complex structures. Yet, the prevalent research paradigm is reductionist. Focusing on infectious diseases, reductionist and non-reductionist views are here described. Furthermore, the literature indicates that reductionism is associated with information loss and errors, while non-reductionist operations can extract more information from the same data. When designed to capture one-to-many/many-to-one interactions—including the use of arrows that connect pairs of consecutive observations—non-reductionist (spatial–temporal) constructs eliminate data variability from all dimensions, except along one line, while arrows describe the directionality of temporal changes that occur along the line. To validate the patterns detected by non-reductionist operations, reductionist procedures are needed. Integrated (non-reductionist and reductionist) methods can (i) distinguish data subsets that differ immunologically and statistically; (ii) differentiate false-negative from -positive errors; (iii) discriminate disease stages; (iv) capture in vivo, multilevel interactions that consider the patient, the microbe, and antibiotic-mediated responses; and (v) assess dynamics. Integrated methods provide repeatable and biologically interpretable information.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

ION BEAM POLARIZATION DYNAMICS IN THE 8 GEV BOOSTER OF THE JLEIC PROJECT AT JLAB

In the Jefferson Lab’s Electron-Ion Collider (JLEIC) project, an injector of polarized ions into the collider ring is a superconducting 8 GeV booster. Both figure-8 and racetrack booster versions were considered. Our analysis showed that the figure-8 ring configuration allows one to preserve the polarization of any ion species during beam acceleration using only small longitudinal field with an integral less than 0.5 Tm. In the racetrack booster, to pre-serve the polarization of ions with the exception of deu-terons, it suffices to use a solenoidal Siberian snake with a maximum field integral of 30 Tm. To preserve deuteron polarization, we propose to use arc magnets for the race-track booster structure with a field ramp rate of the order of 1 T/s. We calculate deuteron and proton beam polari-zations in both the figure-8 and racetrack boosters includ-ing alignment errors of their magnetic elements using the Zgoubi code.

Kondratenko, A. M.↗
DOE OSTI2014

Correlation of anomalous write error rates and ferromagnetic resonance spectrum in spin-transfer-torque-magnetic-random-access-memory devices containing in-plane free layers

In a small fraction of magnetic-tunnel-junction-based magnetic random-access memory devices with in-plane free layers, the write-error rates (WERs) are higher than expected on the basis of the macrospin or quasi-uniform magnetization reversal models. In devices with increased WERs, the product of effective resistance and area, tunneling magnetoresistance, and coercivity do not deviate from typical device properties. However, the field-swept, spin-torque, ferromagnetic resonance (FS-ST-FMR) spectra with an applied DC bias current deviate significantly for such devices. With a DC bias of 300 mV (producing 9.9 × 10{sup 6} A/cm{sup 2}) or greater, these anomalous devices show an increase in the fraction of the power present in FS-ST-FMR modes corresponding to higher-order excitations of the free-layer magnetization. As much as 70% of the power is contained in higher-order modes compared to ≈20% in typical devices. Additionally, a shift in the uniform-mode resonant field that is correlated with the magnitude of the WER anomaly is detected at DC biases greater than 300 mV. These differences in the anomalous devices indicate a change in the micromagnetic resonant mode structure at high applied bias.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

AP-Cloud: Adaptive Particle-in-Cloud method for optimal solutions to Vlasov–Poisson equation

We propose a new adaptive Particle-in-Cloud (AP-Cloud) method for obtaining optimal numerical solutions to the Vlasov–Poisson equation. Unlike the traditional particle-in-cell (PIC) method, which is commonly used for solving this problem, the AP-Cloud adaptively selects computational nodes or particles to deliver higher accuracy and efficiency when the particle distribution is highly non-uniform. Unlike other adaptive techniques for PIC, our method balances the errors in PDE discretization and Monte Carlo integration, and discretizes the differential operators using a generalized finite difference (GFD) method based on a weighted least square formulation. As a result, AP-Cloud is independent of the geometric shapes of computational domains and is free of artificial parameters. Efficient and robust implementation is achieved through an octree data structure with 2:1 balance. We analyze the accuracy and convergence order of AP-Cloud theoretically, and verify the method using an electrostatic problem of a particle beam with halo. Simulation results show that the AP-Cloud method is substantially more accurate and faster than the traditional PIC, and it is free of artificial forces that are typical for some adaptive PIC techniques.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

An unstructured-mesh finite-volume MPDATA for compressible atmospheric dynamics

An advancement of the unstructured-mesh finite-volume MPDATA (Multidimensional Positive Definite Advection Transport Algorithm) is presented that formulates the error-compensative pseudo-velocity of the scheme to rely only on face-normal advective fluxes to the dual cells, in contrast to the full vector employed in previous implementations. This is essentially achieved by expressing the temporal truncation error underlying the pseudo-velocity in a form consistent with the flux-divergence of the governing conservation law. The development is especially important for integrating fluid dynamics equations on non-rectilinear meshes whenever face-normal advective mass fluxes are employed for transport compatible with mass continuity—the latter being essential for flux-form schemes. In particular, the proposed formulation enables large-time-step semi-implicit finite-volume integration of the compressible Euler equations using MPDATA on arbitrary hybrid computational meshes. Furthermore, it facilitates multiple error-compensative iterations of the finite-volume MPDATA and improved overall accuracy. The advancement combines straightforwardly with earlier developments, such as the nonoscillatory option, the infinite-gauge variant, and moving curvilinear meshes. A comprehensive description of the scheme is provided for a hybrid horizontally-unstructured vertically-structured computational mesh for efficient global atmospheric flow modelling. The proposed finite-volume MPDATA is verified using selected 3D global atmospheric benchmark simulations, representative of hydrostatic and non-hydrostatic flow regimes. Besides the added capabilities, the scheme retains fully the efficacy of established finite-volume MPDATA formulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

Data Structures for Extreme Scale Computing

As computing problems of national importance grow, the government meets the increased demand by funding the development of ever larger systems. The overarching goal of the work supported in part by this grant is to increase efficiency of programming and performing computations on these large computing systems. In past work, we have demonstrated that some of these computations once thought to require expensive hardware designs and/or complex, special-purpose programming may be executed efficiently on low-cost commodity cluster computing systems using a general-purpose “latency-tolerant” programming framework. One important developed application of the ideas underlying this framework is graph database technology supporting social network pattern matching used by US intelligence agencies to more quickly identify potential terrorist threats. This database application has been spun out by the Pacific Northwest National Laboratory, a Department of Energy Laboratory, into a commercial start-up, Trovares Inc. We explore an alternative application of the same underlying ideas to a well-studied challenge arising in engineering: solving unstructured sparse linear equations. Solving these equations is key to predicting the behavior of large electronic circuits before they are fabricated. Predicting that behavior ahead of fabrication means that designs can optimized and errors corrected ahead of the expense of manufacture.

42 ENGINEERING↗
DOE OSTI2020

General Protocol for the Accurate Prediction of Molecular 13 C/ 1 H NMR Chemical Shifts via Machine Learning Augmented DFT

An accurate prediction of NMR chemical shifts at affordable computational cost is very important for different types of structural assignments in experimental studies. Density functional theory (DFT) and gauge-including atomic orbital (GIAO) are two of the most popular computational methods for NMR calculation, yet they often fail to resolve ambiguities in structural assignments. In this work, we present a new method that uses machine learning (ML) techniques (DFT + ML) that significantly increases the accuracy of 13 C/ 1 H NMR chemical shift prediction for a variety of organic molecules. The input of the generalizable DFT + ML model contains two critical parts: one is a vector providing insights into chemical environments, which can be evaluated without knowing the exact geometry of the molecule; the other one is the DFT-calculated isotropic shielding constant. The DFT + ML model was trained with a data set containing 476 13 C and 270 1 H experimental chemical shifts. For the DFT methods used here, the root mean square deviations (RMSDs) for the errors between predicted and experimental 13 C/ 1 H chemical shifts can be as small as 2.10/0.18 ppm, which is much lower than those from simple DFT (5.54/0.25 ppm), or DFT + linear regression (LR) (4.77/0.23 ppm) approaches. It also has a smaller maximum absolute error than two previously proposed NMR-predicting ML models. The robustness of the DFT + ML model is tested on two classes of organic molecules (TIC10 and hyacinthacines), where the correct isomers were unambiguously assigned to the experimental ones. Overall, the DFT + ML model shows promise for structural assignments in a variety of systems, including stereoisomers, that are often challenging to determine experimentally.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

Spin-orbit torques for current parallel and perpendicular to a domain wall

We report field- and current-induced domain wall (DW) depinning experiments in Ta\Co{sub 20}Fe{sub 60}B{sub 20}\MgO nanowires through a Hall cross geometry. While purely field-induced depinning shows no angular dependence on in-plane fields, the effect of the current depends crucially on the internal DW structure, which we manipulate by an external magnetic in-plane field. We show depinning measurements for a current sent parallel to the DW and compare its depinning efficiency with the conventional case of current flowing perpendicularly to the DW. We find that the maximum efficiency is similar for both current directions within the error bars, which is in line with a dominating damping-like spin-orbit torque (SOT) and indicates that no large additional torques arise for currents perpendicular to the DW. Finally, we find a varying dependence of the maximum depinning efficiency angle for different DWs and pinning levels. This emphasizes the importance of our full angular scans compared with previously used measurements for just two field directions (parallel and perpendicular to the DW) to determine the real torque strength and shows the sensitivity of the SOT to the precise DW structure and pinning sites.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2014

SU-E-T-477: An Efficient Dose Correction Algorithm Accounting for Tissue Heterogeneities in LDR Brachytherapy

Purpose: Seed brachytherapy is currently used for adjuvant radiotherapy of early stage prostate and breast cancer patients. The current standard for calculation of dose surrounding the brachytherapy seeds is based on American Association of Physicist in Medicine Task Group No. 43 (TG-43 formalism) which generates the dose in homogeneous water medium. Recently, AAPM Task Group No. 186 emphasized the importance of accounting for tissue heterogeneities. This can be done using Monte Carlo (MC) methods, but it requires knowing the source structure and tissue atomic composition accurately. In this work we describe an efficient analytical dose inhomogeneity correction algorithm implemented using MIM Symphony treatment planning platform to calculate dose distributions in heterogeneous media. Methods: An Inhomogeneity Correction Factor (ICF) is introduced as the ratio of absorbed dose in tissue to that in water medium. ICF is a function of tissue properties and independent of source structure. The ICF is extracted using CT images and the absorbed dose in tissue can then be calculated by multiplying the dose as calculated by the TG-43 formalism times ICF. To evaluate the methodology, we compared our results with Monte Carlo simulations as well as experiments in phantoms with known density and atomic compositions. Results: The dose distributions obtained through applying ICF to TG-43 protocol agreed very well with those of Monte Carlo simulations as well as experiments in all phantoms. In all cases, the mean relative error was reduced by at least 50% when ICF correction factor was applied to the TG-43 protocol. Conclusion: We have developed a new analytical dose calculation method which enables personalized dose calculations in heterogeneous media. The advantages over stochastic methods are computational efficiency and the ease of integration into clinical setting as detailed source structure and tissue segmentation are not needed. University of Toronto, Natural Sciences and Engineering Research Council of Canada.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Robust trajectory-constrained frequency control for microgrids considering model linearization error

Grid supportive modes integrated within inverter-based resources can improve the frequency response of renewable-rich microgrids. The synthesis of grid supportive modes to guarantee frequency trajectory constraints under a predefined disturbance set is challenging but essential. To tackle this challenge, a numerical optimal control (NOC)-based control synthesis methodology is proposed. Without loss of generality, a wind-diesel fed microgrid is studied, where we aim to design grid supportive functions in the wind turbine. In the control design, linearized models are used, and the linearization-induced errors are quantitatively analyzed by reachability and interval arithmetics and represented in the form of interval uncertainties. Then, the NOC problem can be formulated into a robust mixed-integer linear program. The control structure is strategically configured into two levels to realize online deployment. The proposed control is verified on the modified 33-node microgrid with a full-order three-phase nonlinear model in Simulink. In conclusion, the simulation results show the effectiveness of the proposed control paradigm and the necessity of considering linearization-induced uncertainty.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

Best practices in evaluation of the magnetocaloric effect from bulk magnetization measurements

Conventional magnetometry is irreplaceable in evaluating bulk magnetization of materials over broad temperature and field ranges. The technique is also effective in quantifying hysteresis that may be associated with magnetic and structural phase transitions that occur during the magnetizing/demagnetizing cycling, and the derived magnetic field-induced isothermal entropy change – one of the most important properties in the field of magnetocalorics. Both systematic and random errors present during the measurements of magnetization, however, may lead to erroneous conclusions. Using two well-known materials – elemental Gd and intermetallic Gd 5 Si 2 Ge 2 as examples, we consider best practices in performing reliable and rapid magnetization measurements for proper characterization of magnetocaloric properties.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Tracking seasonal variability in plant traits from spaceborne PRISMA and NEON AOP across forest types and ecoregions

Plant traits serve as critical indicators of how plants adapt to environmental changes and influence ecosystem functions. While airborne hyperspectral remote sensing effectively maps plant traits through detailed reflectance properties, it is limited by cost and scale, making large-scale and temporal studies challenging. The recently launched spaceborne hyperspectral imager, PRecursore IperSpettrale della Missione Applicativa (PRISMA), offers frequent, large scale and high-fidelity observations on a spatial resolution of 30 m and a revisit time of around 29 days, making it suitable for large-scale seasonal trait mapping. However, their potential remains largely unexplored. This study developed a multi-stage framework by leveraging the PRISMA spaceborne hyperspectral data and National Ecological Observatory Network (NEON) Airborne Observation Platform (AOP) hyperspectral data to investigate the seasonal dynamics of four key plant traits — chlorophyll content, carotenoid content, equivalent water thickness, and nitrogen content — across eleven NEON sites representing diverse forest types and ecoregions in the contiguous U.S. Our results demonstrated that PRISMA hyperspectral data can reliably track seasonal variability in plant traits, achieving overall R 2 values ranging from 0.78 to 0.88 and normalized root mean square error (NRMSE) values ranging from 5.4% to 8.4% for the four traits. Seasonal patterns revealed bell-shaped trajectories for chlorophyll and carotenoids, while equivalent water thickness decreased steadily across most sites, driven by structural changes during leaf maturation and senescence. Nitrogen content exhibited less pronounced seasonal variation but followed expected nutrient resorption patterns. Analysis of environmental drivers showed that seasonal variability is primarily controlled by solar radiation and day length in northern sites, vapor pressure in semi-arid regions, and temperature in mid-southeastern sites. Spatial variability, meanwhile, was primarily driven by soil properties, particularly during the peak growing season. However, the influence of soil variables slightly declines toward the end of the season at several sites, as climatic factors become more prominent. This study highlights the capability of PRISMA, and potentially other similar spaceborne hyperspectral data for large-scale, time-series plant trait mapping and provides valuable insights into the interactions between plant traits and environmental factors. In conclusion, these findings contribute to advancing our understanding of plant functional ecology and improving predictions of ecosystem responses to environmental changes.

Environmental drivers↗
DOE OSTI2014

In Situ Observation of Directed Nanoparticle Aggregation During the Synthesis of Ordered Nanoporous Metal in Soft Templates

The prevalent approach to developing new nanomaterials is a trial and error process of iteratively altering synthesis procedures and then characterizing the resulting nanostructures. This is fundamentally limited in that the growth processes that occur during synthesis can only be inferred from the final synthetic structure. Directly observing real-time nanomaterial growth provides unprecedented insight into the relationship between synthesis conditions and product evolution, and facilitates a mechanistic approach to nanomaterial development. Here we use in situ liquid stage scanning transmission electron microscopy to observe the growth of mesoporous palladium in a solvated block copolymer (BCP) template under various synthesis conditions, and ultimately determine a refined synthesis procedure that yields ordered pores. We find that at low organic solvent (tetrahydrofuran, THF) content, the BCP assembles into a rigid, cylindrical micelle array with a high degree of short-range order, but poor long-range order. Upon slowing the THF evaporation rate using a solvent-vapor anneal step, the long-range order is greatly improved. The electron beam induces nucleation of small particles in the aqueous phase around the micelles. The small particles then flocculate and grow into denser structures that surround the micelles, forming an ordered mesoporous structure. The microscope observations revealed that template disorder can be addressed prior to reaction, and is not invariably induced by the growth process itself, allowing us to more quickly optimize the synthetic method. This work was conducted in the William R. Wiley Environmental Molecular Sciences Laboratory (EMSL), a national scientific user facility sponsored by DOE’s Office of Biological and Environmental Research and located at Pacific Northwest National Laboratory. The Pacific Northwest National Laboratory is operated by Battelle for the U.S. Department of Energy under contract DE-AC05-76RL01830. This research was funded in part by: the Presidential Early Career Award for Scientist and Engineers for I.A., the University of California Academic Senate and the University of California Laboratory fee research grant, the Laboratory-Directed Research and Development program at Sandia National Laboratories, and the Chemical Imaging Initiative at Pacific Northwest National Laboratory. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000.

cylindrical micelle↗
DOE OSTI2014

The intrinsic quasar luminosity function: Accounting for accretion disk anisotropy

Quasar luminosity functions are a fundamental probe of the growth and evolution of supermassive black holes. Measuring the intrinsic luminosity function is difficult in practice, due to a multitude of observational and systematic effects. As sample sizes increase and measurement errors drop, characterizing the systematic effects is becoming more important. It is well known that the continuum emission from the accretion disk of quasars is anisotropic—in part due to its disk-like structure—but current luminosity function calculations effectively assume isotropy over the range of unobscured lines of sight. Here, we provide the first steps in characterizing the effect of random quasar orientations and simple models of anisotropy on observed luminosity functions. We find that the effect of orientation is not insignificant and exceeds other potential corrections such as those from gravitational lensing of foreground structures. We argue that current observational constraints may overestimate the intrinsic luminosity function by as much as a factor of ∼2 on the bright end. This has implications for models of quasars and their role in the universe, such as quasars' contribution to cosmological backgrounds.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

TU-CD-BRA-01: A Novel 3D Registration Method for Multiparametric Radiological Images

Purpose: Multiparametric and multimodality radiological imaging methods, such as, magnetic resonance imaging(MRI), computed tomography(CT), and positron emission tomography(PET), provide multiple types of tissue contrast and anatomical information for clinical diagnosis. However, these radiological modalities are acquired using very different technical parameters, e.g.,field of view(FOV), matrix size, and scan planes, which, can lead to challenges in registering the different data sets. Therefore, we developed a hybrid registration method based on 3D wavelet transformation and 3D interpolations that performs 3D resampling and rotation of the target radiological images without loss of information Methods: T1-weighted, T2-weighted, diffusion-weighted-imaging(DWI), dynamic-contrast-enhanced(DCE) MRI and PET/CT were used in the registration algorithm from breast and prostate data at 3T MRI and multimodality(PET/CT) cases. The hybrid registration scheme consists of several steps to reslice and match each modality using a combination of 3D wavelets, interpolations, and affine registration steps. First, orthogonal reslicing is performed to equalize FOV, matrix sizes and the number of slices using wavelet transformation. Second, angular resampling of the target data is performed to match the reference data. Finally, using optimized angles from resampling, 3D registration is performed using similarity transformation(scaling and translation) between the reference and resliced target volume is performed. After registration, the mean-square-error(MSE) and Dice Similarity(DS) between the reference and registered target volumes were calculated. Results: The 3D registration method registered synthetic and clinical data with significant improvement(p<0.05) of overlap between anatomical structures. After transforming and deforming the synthetic data, the MSE and Dice similarity were 0.12 and 0.99. The average improvement of the MSE in breast was 62%(0.27 to 0.10) and prostate was 63%(0.13 to 0.04;p<0.05). The Dice similarity was in breast 8%(0.91 to 0.99) and for prostate was 89%(0.01 to 0.90;p<0.05) Conclusion: Our 3D wavelet hybrid registration approach registered diverse breast and prostate data of different radiological images(MR/PET/CT) with a high accuracy.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Understanding Uncertainty in Microstructure Evolution and Constitutive Properties in Additive Process Modeling

Coupled process–microstructure–property modeling, and understanding the sources of uncertainty and their propagation toward error in part property prediction, are key steps toward full utilization of additive manufacturing (AM) for predictable quality part development. The OpenFOAM model for process conditions, the ExaCA model for as-solidified grain structure, and the ExaConstit model for constitutive mechanical properties are used as part of the ExaAM modeling framework to examine a few of the various sources of uncertainty in the modeling workflow. In addition to “random” uncertainty (due to random number generation in the orientations and locations of grains present), the heterogeneous nucleation density N 0 and the mean substrate grain spacing S 0 are varied to examine their impact of grain area development as a function of build height in the simulated microstructure. While mean grain area after 1 mm of build is found to be sensitive to N 0 and S 0 , particularly at small N 0 and large S 0 (despite some convergence toward similar values), the resulting grain shapes and overall textures develop in a reasonably similar manner. As a result of these similar textures, ExaConstit simulation using ExaCA representative volume elements (RVEs) from various permutations of N 0 , S 0 , and location within the build resulted in similar yield stress, stress–strain curve shape, and stress triaxiality distributions. It is concluded that for this particular material and scan pattern, 15 layers is sufficient for ExaCA texture and ExaConstit predicted properties to become relatively independent of additional layer simulation, provided that reasonable estimates for N 0 and S 0 are used. However, additional layers of ExaCA will need to be run to obtain mean grain areas independent of build height and baseplate structure.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Phase-curve Pollution of Exoplanet Transmission Spectra

The occurrence of a planet transiting in front of its host star offers the opportunity to observe the planet’s atmosphere filtering starlight. The fraction of occulted stellar flux is roughly proportional to the optically thick area of the planet, the extent of which depends on the opacity of the planet’s gaseous envelope at the observed wavelengths. Chemical species, haze, and clouds are now routinely detected in exoplanet atmospheres through rather small features in transmission spectra, i.e., collections of planet-to-star area ratios across multiple spectral bins and/or photometric bands. Technological advances have led to a shrinking of the error bars down to a few tens of parts per million (ppm) per spectral point for the brightest targets. The upcoming James Webb Space Telescope (JWST) is anticipated to deliver transmission spectra with precision down to 10 ppm. The increasing precision of measurements requires a reassessment of the approximations hitherto adopted in astrophysical models, including transit light-curve models. Recently, it has been shown that neglecting the planet’s thermal emission can introduce significant biases in the transit depth measured with the JWST/Mid-InfraRed Instrument, integrated between 5 and 12 μm. In this paper, we take a step forward by analyzing the effects of the approximation on transmission spectra over the 0.6–12 μm wavelength range covered by various JWST instruments. We present open-source software to predict the spectral bias, showing that, if not corrected, it may affect the inferred molecular abundances and thermal structure of some exoplanet atmospheres.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation

Whittle index policy is a heuristic to the intractable restless multi-armed bandits (RMAB) problem. Although it is provably asymptotically optimal, finding Whittle indices remains difficult. In this paper, we present Neural-Q-Whittle, a Whittle index based Q-learning algorithm for RMAB with neural network function approximation, which is an example of nonlinear two-timescale stochastic approximation with Q-function values updated on a faster timescale and Whittle indices on a slower timescale. Despite the empirical success of deep Q-learning, the non-asymptotic convergence rate of Neural-Q-Whittle, which couples neural networks with two-timescale Q-learning largely remains unclear. This paper provides a finite-time analysis of Neural-Q-Whittle, where data are generated from a Markov chain, and Q-function is approximated by a ReLU neural network. Our analysis leverages a Lyapunov drift approach to capture the evolution of two coupled parameters, and the nonlinearity in value function approximation further requires us to characterize the approximation error. Combing these provide Neural-Q-Whittle with convergence rate, where is the number of iterations.

reinforcement learning, structured learning, conve↗