Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gradient descent”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Closed-Form Approximation of the Total Variation Proximal Operator

Total variation (TV) is a widely used function for regularizing imaging inverse problems that is particularly appropriate for images whose underlying structure is piecewise constant. TV regularized optimization problems are typically solved using proximal methods, but the way in which they are applied is constrained by the absence of a closed-form expression for the proximal operator of the TV function. A closed-form approximation of the TV proximal operator has previously been proposed, but its accuracy was not theoretically explored in detail. Here, we address this gap by making several new theoretical contributions, proving that the approximation leads to a proximal operator of some convex function, it is equivalent to a gradient descent step on a smoothed version of TV, and that its error can be fully characterized and controlled with its scaling parameter. We experimentally validate our theoretical results on image denoising and sparse-view computed tomography (CT) image reconstruction.

97 MATHEMATICS AND COMPUTING↗

Meta-Learning Enhanced Physics-Informed Graph Attention Convolutional Network for Distribution Power System State Estimation

Promptly perceiving distribution system states is challenged by frequent topology changes and uncertain power injections. To address these issues, a Meta-learning enhanced physics-informed graph attention convolutional network (Meta-PIGACN) model is proposed to handle topological variability in distribution system state estimation (DSSE). Specifically, physics information is integrated into the graph convolutional network, enabling a physics-informed edge-weighting process that incorporates physical information to control the aggregation of neighboring nodes. Besides, the graph attention mechanism automatically adjusts the importance of different neighboring nodes, allowing the capture and preservation of inherent system features across varying topologies, thereby improving state estimation accuracy. Furthermore, meta-learning is proposed to acquire empirical knowledge across multiple topologies so that the model can rapidly adapt to new configurations through iterative gradient descent updates even in large-scale systems. In conclusion, the simulation results based on the 33/118/1746-node distribution systems show the high accuracy and efficiency of the proposed model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Noise-Resilient Quantum Machine Learning for Stability Assessment of Power Systems

Transient stability assessment (TSA) is a cornerstone for resilient operations of todays interconnected power grids. This paper is a confluence of quantum computing, data science and machine learning to potentially address the power system TSA issue. Here, we devise a quantum TSA (QTSA) method to enable scalable and efficient data-driven transient stability prediction for bulk power systems, which is the first attempt to tackle the TSA issue with quantum computing. Our contributions are three-fold: 1) A high expressibility, low-depth (HELD) quantum circuit is designed for accurate and noise-resilient TSA; 2) A quantum natural gradient descent algorithm is developed for efficient HELD circuit training; 3) A systematical analysis on QTSAs performance under various quantum factors is per-formed. QTSA underpins a foundation of quantum-enabled and data-driven power grid stability analytics. It renders the intractable TSA straightforward and effortless in the Hilbert space, and therefore provides stability information for power system operations. Extensive experiments on quantum simulators and real quantum computers verify the accuracy, noise-resilience, scalability and universality of QTSA.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Machine learning and atomic layer deposition: Predicting saturation times from reactor growth profiles using artificial neural networks

In this work, we explore the application of deep neural networks to the optimization of atomic layer deposition (ALD) processes. In particular, we focus on a one-shot optimization problem, where we try to predict the optimal dose time that leads to saturation everywhere in the reactor based on thickness values measured at different points of an ALD reactor after a single trial growth. In order to tackle this problem, we introduce a dataset designed to train neural networks to predict saturation times based on these inputs for a cross-flow ALD reactor. Here, we then explore the predictive ability of artificial neural networks of different depths and sizes using a separate testing dataset to evaluate their accuracies. The results obtained show that networks trained using stochastic gradient descent methods can accurately predict saturation times without requiring any additional information on the surface kinetics. This provides a viable approach to minimize the number of experiments required to optimize new ALD processes in a known reactor, and it highlights the way machine learning can be leveraged for thin film growth and manufacturing. While the datasets and training procedure depend on the reactor geometry, the trained neural networks provide a general surrogate model connecting thickness values and trial dose times with optimal saturation times that can be reused for different ALD processes within the same reactor.

36 MATERIALS SCIENCE↗

Autonomous hybrid optimization of a SiO 2 plasma etching mechanism

Computational modeling of plasma etching processes at the feature scale relevant to the fabrication of nanometer semiconductor devices is critically dependent on the reaction mechanism representing the physical processes occurring between plasma produced reactant fluxes and the surface, reaction probabilities, yields, rate coefficients, and threshold energies that characterize these processes. The increasing complexity of the structures being fabricated, new materials, and novel gas mixtures increase the complexity of the reaction mechanism used in feature scale models and increase the difficulty in developing the fundamental data required for the mechanism. This challenge is further exacerbated by the fact that acquiring these fundamental data through more complex computational models or experiments is often limited by cost, technical complexity, or inadequate models. In this paper, we discuss a method to automate the selection of fundamental data in a reduced reaction mechanism for feature scale plasma etching of SiO 2 using a fluorocarbon gas mixture by matching predictions of etch profiles to experimental data using a gradient descent (GD)/Nelder–Mead (NM) method hybrid optimization scheme. These methods produce a reaction mechanism that replicates the experimental training data as well as experimental data using related but different etch processes.

36 MATERIALS SCIENCE↗

Nonlinear registration of serial coronary CT angiography (CCTA) for assessment of changes in atherosclerotic plaque

Purpose: Coronary CT angiography (CCTA) is a high-resolution three-dimensional imaging technique for the evaluation of coronary arteries in suspected or confirmed coronary artery disease (CAD). Coregistration of serial CCTA scans would allow precise superimposition of images obtained at two different points in time, which could aid in recognition of subtle changes and precise monitoring of coronary plaque progression or regression. To this end, the authors aimed at developing a fully automatic nonlinear volume coregistration for longitudinal CCTA scan pairs. Methods: The algorithm combines global displacement and local deformation using nonlinear volume coregistration with a volume-preserving constraint. Histogram matching of intensities between two serial scans is performed prior to nonlinear coregistration with dense nonparametric local deformation in which sum of squared differences is used as a similarity measure. The approximate segmentation of coronary arteries obtained from commercially available software provides initial anatomical landmarks for the coregistration algorithm that help localize and emphasize the structure of interest. To avoid possible bias caused by incorrect segmentation, the authors convolve the Gaussian kernel with the segmented binary coronary tree mask and define an extended weighted region of interest. A multiresolution approach is employed to represent coarse-to-fine details of both volumes and the energy function is optimized using a gradient descent method. The authors applied the algorithm in ten paired CCTA datasets (20 scans in total) obtained within 10.7{+-}5.7 months from each other on a dual source CT scanner to monitor progression of CAD. Results: Serial CCTA coregistration was successful in 9/10 cases as visually confirmed. The global displacement and local deformation of target registration error obtained from four anatomical landmarks were 2.22{+-}1.15 and 1.56{+-}0.74 mm, respectively, and the inverse consistency error of local deformation was 0.14{+-}0.06 mm. The observer variability between two expert observers was 1.31{+-}0.91 mm. Conclusions: The proposed coregistration algorithm demonstrates potential to accurately register serial CCTA scans, which may allow direct comparison of calcified and noncalcified atherosclerotic plaque changes between the two scans.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Metal artifact reduction in x-ray computed tomography (CT) by constrained optimization

Purpose: The streak artifacts caused by metal implants have long been recognized as a problem that limits various applications of CT imaging. In this work, the authors propose an iterative metal artifact reduction algorithm based on constrained optimization. Methods: After the shape and location of metal objects in the image domain is determined automatically by the binary metal identification algorithm and the segmentation of ''metal shadows'' in projection domain is done, constrained optimization is used for image reconstruction. It minimizes a predefined function that reflects a priori knowledge of the image, subject to the constraint that the estimated projection data are within a specified tolerance of the available metal-shadow-excluded projection data, with image non-negativity enforced. The minimization problem is solved through the alternation of projection-onto-convex-sets and the steepest gradient descent of the objective function. The constrained optimization algorithm is evaluated with a penalized smoothness objective. Results: The study shows that the proposed method is capable of significantly reducing metal artifacts, suppressing noise, and improving soft-tissue visibility. It outperforms the FBP-type methods and ART and EM methods and yields artifacts-free images. Conclusions: Constrained optimization is an effective way to deal with CT reconstruction with embedded metal objects. Although the method is presented in the context of metal artifacts, it is applicable to general ''missing data'' image reconstruction problems.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Scatter correction in cone-beam CT via a half beam blocker technique allowing simultaneous acquisition of scatter and image information

Purpose: X-ray scatter incurred to detectors degrades the quality of cone-beam computed tomography (CBCT) and represents a problem in volumetric image guided and adaptive radiation therapy. Several methods using a beam blocker for the estimation and subtraction of scatter have been proposed. However, due to missing information resulting from the obstruction of the blocker, such methods require dual scanning or dynamically moving blocker to obtain a complete volumetric image. Here, we propose a half beam blocker-based approach, in conjunction with a total variation (TV) regularized Feldkamp-Davis-Kress (FDK) algorithm, to correct scatter-induced artifacts by simultaneously acquiring image and scatter information from a single-rotation CBCT scan. Methods: A half beam blocker, comprising lead strips, is used to simultaneously acquire image data on one side of the projection data and scatter data on the other half side. One-dimensional cubic B-Spline interpolation/extrapolation is applied to derive patient specific scatter information by using the scatter distributions on strips. The estimated scatter is subtracted from the projection image acquired at the opposite view. With scatter-corrected projections where this subtraction is completed, the FDK algorithm based on a cosine weighting function is performed to reconstruct CBCT volume. To suppress the noise in the reconstructed CBCT images produced by geometric errors between two opposed projections and interpolated scatter information, total variation regularization is applied by a minimization using a steepest gradient descent optimization method. The experimental studies using Catphan504 and anthropomorphic phantoms were carried out to evaluate the performance of the proposed scheme. Results: The scatter-induced shading artifacts were markedly suppressed in CBCT using the proposed scheme. Compared with CBCT without a blocker, the nonuniformity value was reduced from 39.3% to 3.1%. The root mean square error relative to values inside the regions of interest selected from a benchmark scatter free image was reduced from 50 to 11.3. The TV regularization also led to a better contrast-to-noise ratio. Conclusions: An asymmetric half beam blocker-based FDK acquisition and reconstruction technique has been established. The proposed scheme enables simultaneous detection of patient specific scatter and complete volumetric CBCT reconstruction without additional requirements such as prior images, dual scans, or moving strips.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Improving best-phase image quality in cardiac CT by motion correction with MAM optimization

Purpose: Research in image reconstruction for cardiac CT aims at using motion correction algorithms to improve the image quality of the coronary arteries. The key to those algorithms is motion estimation, which is currently based on 3-D/3-D registration to align the structures of interest in images acquired in multiple heart phases. The need for an extended scan data range covering several heart phases is critical in terms of radiation dose to the patient and limits the clinical potential of the method. Furthermore, literature reports only slight quality improvements of the motion corrected images when compared to the most quiet phase (best-phase) that was actually used for motion estimation. In this paper a motion estimation algorithm is proposed which does not require an extended scan range but works with a short scan data interval, and which markedly improves the best-phase image quality. Methods: Motion estimation is based on the definition of motion artifact metrics (MAM) to quantify motion artifacts in a 3-D reconstructed image volume. The authors use two different MAMs, entropy, and positivity. By adjusting the motion field parameters, the MAM of the resulting motion-compensated reconstruction is optimized using a gradient descent procedure. In this way motion artifacts are minimized. For a fast and practical implementation, only analytical methods are used for motion estimation and compensation. Both the MAM-optimization and a 3-D/3-D registration-based motion estimation algorithm were investigated by means of a computer-simulated vessel with a cardiac motion profile. Image quality was evaluated using normalized cross-correlation (NCC) with the ground truth template and root-mean-square deviation (RMSD). Four coronary CT angiography patient cases were reconstructed to evaluate the clinical performance of the proposed method. Results: For the MAM-approach, the best-phase image quality could be improved for all investigated heart phases, with a maximum improvement of the NCC value by 100% and of the RMSD value by 81%. The corresponding maximum improvements for the registration-based approach were 20% and 40%. In phases with very rapid motion the registration-based algorithm obtained better image quality, while the image quality of the MAM algorithm was superior in phases with less motion. The image quality improvement of the MAM optimization was visually confirmed for the different clinical cases. Conclusions: The proposed method allows a software-based best-phase image quality improvement in coronary CT angiography. A short scan data interval at the target heart phase is sufficient, no additional scan data in other cardiac phases are required. The algorithm is therefore directly applicable to any standard cardiac CT acquisition protocol.

60 APPLIED LIFE SCIENCES↗

SU-E-J-37: Combining Proton Radiography and X-Ray CT Information to Better Estimate Relative Proton Stopping Power in a Clinical Environment

Purpose: In standard proton therapy clinical practice, proton stopping power uncertainties are in the order of 3.5%, which affects the ability of placing the proton Bragg peak at the edge of the tumor. The innovating idea of this project is to approach the uncertainty problem in RSP by using combined information from X-ray CT and proton radiography along a few beam angles. In addition, this project aims to quantify the systematic error introduced by the theoretical models (Janni, ICRU49, Bischel) for proton stopping power in media. Methods: A 3D phantom of 36 cm3 composed of 9 materials randomly placed is created. Measured RSP values are obtained using a Gammex phantom with a proton beam. Theoretical RSP values are calculated with Beth-Block equation in combination with three databases (Janni, ICRU49 and Bischel). Clinical RSP errors are simulated by introducing a systematic (1.5%, 2.5%, 3.5%) and a random error (+/−0.5%) to the theoretical RSP. A ray-tracing algorithm uses each of these RSP tables to calculate energy loss for proton crossing the phantom through various directions. For each direction, gradient descent (GD) method is done on the clinical RSP table to minimize the residual energy difference between the simulation with clinical RSP and with theoretical RSP. The possibility of a systematic material dependent error is investigated by comparing measured RSP to theoretical RSP as calculated from the three models. Results: Using 10,000 iterations on GD algorithm, RSP differences between theoretical values and clinical RSP have converged (<1%) for each error introduced. Results produced with ICRU49 have the smallest average difference (0.021%) to the measured RSP. Janni (1.168%) and Bischel (−0.372%) database shows larger systematic errors. Conclusion: Based on these results, ray-tracing optimisation using information from proton radiography and X-ray CT demonstrates a potential to improve the proton range accuracy in a clinical environment.

07 ISOTOPE AND RADIATION SOURCES↗

SU-E-J-83: Ion Imaging to Better Estimate In-Vivo Relative Stopping Powers Using X-Ray CT Prior-Knowledge Information

Purpose: To reduce uncertainties in relative stopping power (RSP) estimates for ions (alpha and carbon) by using Ion radiographic-imaging and X-ray CT prior-knowledge. Methods: A 36×36 phantom matrix composed of 9 materials with different thicknesses and randomly placed is generated. Theoretical RSPs are calculated using stopping power (SP) data from three references (Janni, ICRU49 and Bischel). We introduced an artificial systematic error (1.5%, 2.5% or 3.5%) and a random error (<0.5%) to the SP to simulated patient ion-range errors present in clinic environment. Carbon/alpha final energy for each RSPs set (theoretical and from CT images) is obtained with a ray-tracing algorithm. A gradient descent (GD) method is used to minimize the difference in exit particle energy, between theory and X-ray CT RSP maps, by iteratively correcting the RSP map from X-ray CT. Once a new set of RSPs is obtained for a direction a new optimization is done over other direction using the RSPs from the previous optimization. Theoretical RSPs are compared with experimental RSPs obtained with Gammex Phantom. Results: Preliminary results show that optimized RSP values can be obtained with smaller uncertainties (<1%) than clinical RSPs (1.5% to 3.5%). Theoretical values from three different references show uncertainties, up to 3% from experimental values. Further investigation will consider prior-knowledge from RSP obtained with CT images and ion radiographies from Monte Carlo Simulations. Conclusion: GD and ray-tracing methods have been implemented to reduce RSP uncertainties from values obtained for clinical treatment. Experimental RSPs will be obtained using carbon/alpha beams to consider the existence of material dependent systematic errors. Based on the results, it is hoped to show that using ray-tracing optimization with ion radiography and prior knowledge on RPSs, treatment planning accuracy and cost-effectiveness can be improved.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

SU-E-J-195: Quantification of Rotations and Deformations in Head and Heck Radiotherapy

Purpose: To quantify the magnitude of patient rotations and deformations over the course of head and neck radiotherapy. Methods: 45 CBCT images acquired weekly were selected retrospectively from 5 head and neck cancer patients. CBCT images were registered to the corresponding planning CT images using translation, rigid-body, and cubic B-spline deformable transformations. Elastix ( elastix.isi.uu.nl ), an open-source public domain registration algorithm, was used for all registrations with mutual information as similarity metric and gradient descent as optimization. Four-level multiresolution approach was employed with 2 cm B-spline grid spacing at the finest resolution. Registration qualities were visually verified. Rotational occurrences were quantified from the rigid-body registrations. The magnitude of deformations in planning target volumes (PTV) and organs were quantified from the deformation vector fields with rigid registrations as baseline. Results: The measured rotations (mean±std) were 1.0±1.0 deg, - 0.3±1.0 deg, and 0.1±0.8 deg in the sagittal, coronal, and axial plains respectively. The estimated magnitudes of deformations (mean±std) were 3.4±2.1 mm for PTVs, 2.9±1.2 mm for parotid glands, 2.8±1.2 mm for mandible, and 2.2±1.1 mm for spinal cord. The overall deformation in the body contour was 4.3±4.3 mm. The organ spatial deviation due to rotation was small and approximately 8.4% of that from deformations. Results varied among patients. Conclusion: Rotations occurred about 1.0 deg (1SD) in the H'N area. Daily geometric deviation of target volumes was considerable and mainly from tissue deformation than rotations. Proper safety margin is necessary for adequate target coverage. Online plan adaptation may mitigate the daily differences between the plan and actual patient poses. Partially supported by a grant from Varian Medical Systems, Palo Alto, CA.

60 APPLIED LIFE SCIENCES↗

Semiautomatic registration of 3D transabdominal ultrasound images for patient repositioning during postprostatectomy radiotherapy

Purpose: The aim of the present work is to propose and evaluate registration algorithms of three-dimensional (3D) transabdominal (TA) ultrasound (US) images to setup postprostatectomy patients during radiation therapy. Methods: Three registration methods have been developed and evaluated to register a reference 3D-TA-US image acquired during the planning CT session and a 3D-TA-US image acquired before each treatment session. The first method (method A) uses only gray value information, whereas the second one (method B) uses only gradient information. The third one (method C) combines both sets of information. All methods restrict the comparison to a region of interest computed from the dilated reference positioning volume drawn on the reference image and use mutual information as a similarity measure. The considered geometric transformations are translations and have been optimized by using the adaptive stochastic gradient descent algorithm. Validation has been carried out using manual registration by three operators of the same set of image pairs as the algorithms. Sixty-two treatment US images of seven patients irradiated after a prostatectomy have been registered to their corresponding reference US image. The reference registration has been defined as the average of the manual registration values. Registration error has been calculated by subtracting the reference registration from the algorithm result. For each session, the method has been considered a failure if the registration error was above both the interoperator variability of the session and a global threshold of 3.0 mm. Results: All proposed registration algorithms have no systematic bias. Method B leads to the best results with mean errors of −0.6, 0.7, and −0.2 mm in left–right (LR), superior–inferior (SI), and anterior–posterior (AP) directions, respectively. With this method, the standard deviations of the mean error are of 1.7, 2.4, and 2.6 mm in LR, SI, and AP directions, respectively. The latter are inferior to the interoperator registration variabilities which are of 2.5, 2.5, and 3.5 mm in LR, SI, and AP directions, respectively. Failures occur in 5%, 18%, and 10% of cases in LR, SI, and AP directions, respectively. 69% of the sessions have no failure. Conclusions: Results of the best proposed registration algorithm of 3D-TA-US images for postprostatectomy treatment have no bias and are in the same variability range as manual registration. As the algorithm requires a short computation time, it could be used in clinical practice provided that a visual review is performed.

60 APPLIED LIFE SCIENCES↗

Sparsity-based acoustic inversion in cross-sectional multiscale optoacoustic imaging

Purpose: With recent advancement in hardware of optoacoustic imaging systems, highly detailed cross-sectional images may be acquired at a single laser shot, thus eliminating motion artifacts. Nonetheless, other sources of artifacts remain due to signal distortion or out-of-plane signals. The purpose of image reconstruction algorithms is to obtain the most accurate images from noisy, distorted projection data. Methods: In this paper, the authors use the model-based approach for acoustic inversion, combined with a sparsity-based inversion procedure. Specifically, a cost function is used that includes the L1 norm of the image in sparse representation and a total variation (TV) term. The optimization problem is solved by a numerically efficient implementation of a nonlinear gradient descent algorithm. TV–L1 model-based inversion is tested in the cross section geometry for numerically generated data as well as for in vivo experimental data from an adult mouse. Results: In all cases, model-based TV–L1 inversion showed a better performance over the conventional Tikhonov regularization, TV inversion, and L1 inversion. In the numerical examples, the images reconstructed with TV–L1 inversion were quantitatively more similar to the originating images. In the experimental examples, TV–L1 inversion yielded sharper images and weaker streak artifact. Conclusions: The results herein show that TV–L1 inversion is capable of improving the quality of highly detailed, multiscale optoacoustic images obtained in vivo using cross-sectional imaging systems. As a result of its high fidelity, model-based TV–L1 inversion may be considered as the new standard for image reconstruction in cross-sectional imaging.

60 APPLIED LIFE SCIENCES↗

SU-D-204-01: A Methodology Based On Machine Learning and Quantum Clustering to Predict Lung SBRT Dosimetric Endpoints From Patient Specific Anatomic Features

Purpose: To develop a data-mining methodology based on quantum clustering and machine learning to predict expected dosimetric endpoints for lung SBRT applications based on patient-specific anatomic features. Methods: Ninety-three patients who received lung SBRT at our clinic from 2011–2013 were retrospectively identified. Planning information was acquired for each patient, from which various features were extracted using in-house semi-automatic software. Anatomic features included tumor-to-OAR distances, tumor location, total-lung-volume, GTV and ITV. Dosimetric endpoints were adopted from RTOG-0195 recommendations, and consisted of various OAR-specific partial-volume doses and maximum point-doses. First, PCA analysis and unsupervised quantum-clustering was used to explore the feature-space to identify potentially strong classifiers. Secondly, a multi-class logistic regression algorithm was developed and trained to predict dose-volume endpoints based on patient-specific anatomic features. Classes were defined by discretizing the dose-volume data, and the feature-space was zero-mean normalized. Fitting parameters were determined by minimizing a regularized cost function, and optimization was performed via gradient descent. As a pilot study, the model was tested on two esophageal dosimetric planning endpoints (maximum point-dose, dose-to-5cc), and its generalizability was evaluated with leave-one-out cross-validation. Results: Quantum-Clustering demonstrated a strong separation of feature-space at 15Gy across the first-and-second Principle Components of the data when the dosimetric endpoints were retrospectively identified. Maximum point dose prediction to the esophagus demonstrated a cross-validation accuracy of 87%, and the maximum dose to 5cc demonstrated a respective value of 79%. The largest optimized weighting factor was placed on GTV-to-esophagus distance (a factor of 10 greater than the second largest weighting factor), indicating an intuitively strong correlation between this feature and both endpoints. Conclusion: This pilot study shows that it is feasible to predict dose-volume endpoints based on patient-specific anatomic features. The developed methodology can potentially help to identify patients at risk for higher OAR doses, thus improving the efficiency of treatment planning. R01-184173.

60 APPLIED LIFE SCIENCES↗

WE-AB-209-09: Optimization of Rotational Arc Station Parameter Optimized Radiation Therapy

Purpose: To develop a fast optimization method for station parameter optimized radiation therapy (SPORT) and show that SPORT is capable of improving VMAT in both plan quality and delivery efficiency. Methods: The angular space from 0° to 360° was divided into 180 station points (SPs). A candidate aperture was assigned to each of the SPs based on the calculation results using a column generation algorithm. The weights of the apertures were then obtained by optimizing the objective function using a state-of-the-art GPU based Proximal Operator Graph Solver (POGS) within seconds. Apertures with zero or low weight were thrown out. To avoid being trapped in a local minimum, a stochastic gradient descent method was employed which also greatly increased the convergence rate of the objective function. The above procedure repeated until the plan could not be improved any further. A weighting factor associated with the total plan MU also indirectly controlled the complexities of aperture shapes. The number of apertures for VMAT and SPORT was confined to 180. The SPORT allowed the coexistence of multiple apertures in a single SP. The optimization technique was assessed by using three clinical cases (prostate, H&N and brain). Results: Marked dosimetric quality improvement was demonstrated in the SPORT plans for all three studied cases. Prostate case: the volume of the 50% prescription dose was decreased by 22% for the rectum. H&N case: SPORT improved the mean dose for the left and right parotids by 15% each. Brain case: the doses to the eyes, chiasm and inner ears were all improved. SPORT shortened the treatment time by ∼1 min for the prostate case, ∼0.5 min for brain case, and ∼0.2 min for the H&N case. Conclusion: The superior dosimetric quality and delivery efficiency presented here indicates that SPORT is an intriguing alternative treatment modality.

60 APPLIED LIFE SCIENCES↗

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Physics-Informed Neural Networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD) algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, Hamilton-Jacobi-Bellman (HJB), viscous Burgers’, Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

97 MATHEMATICS AND COMPUTING↗