SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
In this article we review recent and ongoing research in quantum communications and networking within the Quantum Information Science Section at Oak Ridge National Laboratory. Our research spans applications such as quantum key distribution as well as more fundamental aspects needed to develop scalable quantum networks including quantum repeaters and space-based quantum communications platforms.
This document describes the algorithm used for the Surface Cloud Grid Value-Added Product (VAP). This VAP uses as input the 15-min. output from the Shortwave (SW) Flux Analysis VAP (see Long 2001; Long and Ackerman 2000; Long et al. 1999) from the Atmospheric Radiation Measurement (ARM) Climate Reseach Facility (ACRF) Southern Great Plains (SGP) Central Facility and extended facilities. This network of 21 sites is unevenly spaced over northern Oklahoma into southern Kansas, covering an area from 95.5° to 99.5° west longitude and 34.5° to 38.5° north latitude. For research applications such as single-column modeling, an estimate of the cloud and cloud effects distribution over this entire domain is desirable. The Surface Cloud Grid VAP applies a multi-pass weighted sum analytic approximation technique (Caracena 1987), which uses Gaussian weighting and an imposed scale length, to interpolate to a 0.25° by 0.25° lat/long grid over the SGP domain. The output, like the input, includes solar elevation angles of 10° or greater.
The ability to establish accurate occupancy is a key enabler for improved energy efficiency in human-occupied indoor spaces. The accurate counts can be used to adjust, in real-time, air exchange, heating, and cooling, to closely match the environmental needs of the occupants rather than to use a single setting based on maximum room occupancy. This project focuses on the use of overhead fisheye cameras and artificial intelligence (AI) algorithms as the technology to realize accurate people counting in large commercial spaces. We developed Computational Occupancy Sensing SYstem ("COSSY''), which supports large-scale deployment (thousands of square feet) and straightforward installation (using conventional power over ethernet), while providing high accuracy and low cost. COSSY has been validated at Boston University and by a third party, demonstrating people-counting accuracy of at least 89.5% in large spaces (2,000 sqft). A cost/benefit analysis performed for Boston commercial market indicates that a full payback of COSSY deployment costs due to energy savings would occur within 3-6 months from installation. Applications of COSSY reach far beyond saving energy. The precise people counting in real time could also be useful for emergency response (e.g., fire), in spatial analytics (e.g., office space management) and in some health applications (e.g., social distance assessment during pandemics).
Natural language processing (NLP) tasks (text classification, named entity recognition, etc.) have seen amazing improvements over the last few years. This is due to models such as BERT that achieve deep knowledge transfer by using a large pre-trained model, then fine-tuning the model on specific tasks. The BERT architecture has shown even better performance on domain-specific tasks when the model is pre-trained using domain-relevant texts. Here, inspired by these recent advancements, we have developed NukeLM, a nuclear-domain BERT model pre-trained on 1.5 million abstracts from the DOE Office of Scientific and Technical Information (OSTI) database. This NukeLM model is then fine-tuned for the classification of research articles into either binary classes (related to the nuclear fuel cycle (NFC) or not) or multiple categories related to the subject of the article. We show that continued pre-training of a BERT-style architecture prior to fine-tuning results in greater performance in both article classification tasks. This information is critical for properly triaging manuscripts, a necessary task for better understanding citation networks that publish in the nuclear space and uncovering new areas of research in the nuclear (or nuclear relevant) domain.
Using a one-dimensional code, we computed the power (enthalpy discharge rate) of a twelve-cell mechanical draft cooling tower (MDCT) using over two hundred visible condensed water vapor plume volume measurements derived from images, weather data, and tower operating conditions. The plume images were simultaneously captured by multiple stationary digital cameras surrounding the cooling tower. An analysis technique combining structure from motion (SfM), a neural-network-based image segmentation algorithm, and space carving was used to quantify the volumes. Afterwards, the power output was computed using novel techniques in the one-dimensional code that included cooling tower exhaust plume adjacency effects implemented with a modified version of the entrainment function, weather data averaged from eleven stations, and fan operations at the times when plume volumes were measured. The model was then compared with the averaged observed power output, and it validated well with an average error ranging from 6 to 12%, depending on the meteorological data used in the simulations. This methodology can possibly determine power plant fuel consumption rates by applying visible imagery.
We developed Distilled Graph Attention Policy Network (DGAPN), a reinforcement learning model to generate novel graph-structured chemical representations that optimize user-defined objectives by efficiently navigating a physically constrained domain. The framework is examined on the task of generating molecules that are designed to bind, noncovalently, to functional sites of SARS-CoV-2 proteins. We present a spatial Graph Attention (sGAT) mechanism that leverages self-attention over both node and edge attributes as well as encoding the spatial structure --- this capability is of considerable interest in synthetic biology and drug discovery. An attentional policy network is introduced to learn the decision rules for a dynamic, fragment-based chemical environment, and state-of-the-art policy gradient techniques are employed to train the network with stability. Exploration is driven by the stochasticity of the action space design and the innovation reward bonuses learned and proposed by random network distillation. In experiments, our framework achieved outstanding results compared to state-of-the-art algorithms, while reducing the complexity of paths to chemical synthesis.
We developed Distilled Graph Attention Policy Network (DGAPN), a reinforcement learning model to generate novel graph-structured chemical representations that optimize user-defined objectives by efficiently navigating a physically constrained domain. The framework is examined on the task of generating molecules that are designed to bind, noncovalently, to functional sites of SARS-CoV-2 proteins. We present a spatial Graph Attention (sGAT) mechanism that leverages self-attention over both node and edge attributes as well as encoding the spatial structure - this capability is of considerable interest in synthetic biology and drug discovery. An attentional policy network is introduced to learn the decision rules for a dynamic, fragment-based chemical environment, and state-of-the-art policy gradient techniques are employed to train the network with stability. Exploration is driven by the stochasticity of the action space design and the innovation reward bonuses learned and proposed by random network distillation. In experiments, our framework achieved outstanding results compared to state-of-the-art algorithms, while reducing the complexity of paths to chemical synthesis.
In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.
Background: Predicting neurological recovery after spinal cord injury (SCI) is challenging. Using topological data analysis, we have previously shown that mean arterial pressure (MAP) during SCI surgery predicts long-term functional recovery in rodent models, motivating the present multicenter study in patients. Methods: Intra-operative monitoring records and neurological outcome data were extracted (n = 118 patients). We built a similarity network of patients from a low-dimensional space embedded using a non-linear algorithm, Isomap, and ensured topological extraction using persistent homology metrics. Confirmatory analysis was conducted through regression methods. Results: Network analysis suggested that time outside of an optimum MAP range (hypotension or hypertension) during surgery was associated with lower likelihood of neurological recovery at hospital discharge. Logistic and LASSO (least absolute shrinkage and selection operator) regression confirmed these findings, revealing an optimal MAP range of 76–[104-117] mmHg associated with neurological recovery. Conclusions: We show that deviation from this optimal MAP range during SCI surgery predicts lower probability of neurological recovery and suggest new targets for therapeutic intervention. Funding: NIH/NINDS: R01NS088475 (ARF); R01NS122888 (ARF); UH3NS106899 (ARF); Department of Veterans Affairs: 1I01RX002245 (ARF), I01RX002787 (ARF); Wings for Life Foundation (ATE, ARF); Craig H. Neilsen Foundation (ARF); and DOD: SC150198 (MSB); SC190233 (MSB); DOE: DE-AC02-05CH11231 (DM).
Abstract We show that deep generative neural networks, based on global optimization networks (GLOnets), can be configured to perform the multiobjective and categorical global optimization of photonic devices. A residual network scheme enables GLOnets to evolve from a deep architecture, which is required to properly search the full design space early in the optimization process, to a shallow network that generates a narrow distribution of globally optimal devices. As a proof-of-concept demonstration, we adapt our method to design thin-film stacks consisting of multiple material types. Benchmarks with known globally optimized antireflection structures indicate that GLOnets can find the global optimum with orders of magnitude faster speeds compared to conventional algorithms. We also demonstrate the utility of our method in complex design tasks with its application to incandescent light filters. These results indicate that advanced concepts in deep learning can push the capabilities of inverse design algorithms for photonics.
In physical networks trained using supervised learning, physical parameters are adjusted to produce desired responses to inputs. An example is an electrical contrastive local learning network of nodes connected by edges that adjust their conductances during training. When an edge conductance changes, it upsets the current balance of every node. In response, physics adjusts the node voltages to minimize the dissipated power. Learning in these systems is therefore a coupled double-optimization process, in which the network descends both a cost landscape in the high-dimensional space of edge conductances and a physical landscape—the power dissipation—in the high-dimensional space of node voltages. Because of this coupling, the physical landscape of a trained network contains information about the learned task. Here, we derive a structure-function relation for trained tunable networks and demonstrate that all the physical information relevant to the trained input-output relation can be captured by a tuning susceptibility, an experimentally measurable quantity. We supplement our theoretical results with simulations to show that the tuning susceptibility is correlated with functional importance and that we can extract physical insight into how the system performs the task from the conductances of highly susceptible edges. Our analysis is general and can be applied directly to mechanical networks, such as networks trained for protein-inspired function such as allostery.
We present a differentiable predictive control (DPC) methodology for learning constrained control laws for unknown nonlinear systems. DPC poses an approximate solution to multiparametric programming problems emerging from explicit nonlinear model predictive control (MPC). Contrary to approximate MPC, DPC does not require supervision by an expert controller. Instead, a system dynamics model is learned from a small dataset of recorded observations of the perturbed system's dynamics and the control law is optimized offline by interaction with the learned system model. The DPC method is based on two sequential steps, i) system identification using a constrained neural state-space model, and ii) optimization of an explicit control law parametrized by another neural network in closed-loop simulation with the identified neural state-space model. The combination of a differentiable closed-loop system and penalty methods for constraint handling of system outputs and inputs allows us to optimize the control law's parameters directly by backpropagating economic MPC loss through the learned system model. By incorporating domain knowledge and leveraging established techniques from optimal control, our method leverages deep neural networks as nonlinear function approximators for system identification and control while avoiding concomitant costs of intractably large datasets, and computationally expensive over-parametrized models. The scalability, data efficiency, and constrained optimal control capability of the proposed DPC method are demonstrated in simulation using a multi-zone building emulator.
Heating, Ventilation, and Air Conditioning (HVAC) systems can maintain the space air temperature of residential buildings, either directly by heating/cooling the air, or indirectly via heat transfer to and from the building structure that acts as a thermal mass. Hence, HVAC systems can help achieve load shifting, peak load reduction, and/or energy cost saving, thus enabling grid-interactive HVAC operation. A home thermal model that can accurately reflect the dynamics of the space air and interior wall surface temperatures, is therefore valuable. This paper develops such a model using the standard RC (resistance-capacitance) approach. The model contains a virtual envelope node and an internal space node and is thus second-order. A hybrid parameter identification scheme, made up of the least-squares and optimal search methods, is also developed. The proposed model and scheme were validated using data collected from a test home. It was found that a modest amount of training data was sufficient to yield reliable parameter estimates and accurate prediction. It was also found that when making 24-hour-ahead prediction of the space air temperature, both methods had comparable performances when the training data began in a transition season. However, when they began in an HVAC season, the optimal search method performed better. Furthermore, the least-squares method is recommended during a transition season due to its lower computational burden, while the optimal search method is recommended during an HVAC season due to its better estimation performance.
One of the main purposes of hydraulic fracturing in unconventional resources such as shale is to improve the connectivity between existing natural fractures and the production well. Since the inherent permeability of shale is extremely low, this newly formed fracture connectivity provides faster paths for the hydrocarbon stored in preexisting natural fractures. In this study, we performed numerical simulations of the free gas production at large scale fractured reservoir with different pre-existing natural fracture network intensities and with variable hydraulic fracture sizes and staging. We explored production characteristics as a function of the properties of both, natural fractures (intensity) and hydraulic fractures (size and staging). Our hypothesis is that once maximum fracture connectivity is achieved, additional stimulation does not increase production significantly. In this numerical study we use an advanced modeling tool to study fractures connectivity, Discrete Fracture Network (DFN) approach, which can represent fracture networks similar to those observed on individual reservoir sites. In our model the horizontal production well and vertical hydraulic fractures were deterministically defined and surrounded by stochastically generated natural fracture networks. The performed numerical experiments show how fracture connectivity depends on hydraulic fracture settings in fractured reservoir with different preexisting fracture densities. We numerically showed that cumulative free gas production is not affected significantly by hydraulic fractures size (25 m–150 m) or spacing (15 m–45 m) for dense natural fracture networks. However, the hydraulic fracture size is important in reservoirs with sparse natural fractures, although spacing (number of stages) does not play a significant role.
The capability to utilize neural networks to predict macroscopic and microscopic cross section parametric spaces has been developed for the Griffin reactor physics application. The LibTorch interface enables Griffin's MOOSE-based materials to interact with LibTorch-trained models, allowing for the evaluation of complex macroscopic or microscopic cross section spaces, which are then used to evaluate the neutronic properties of the Griffin finite element model. This study benchmarks traditional ISOXML-formatted tabulation libraries against neural network-based models for 279 nuclides on 20,160 grid points for zero-dimensional and two-dimensional reactor models. Benchmark metrics include the fundamental mode eigenvalue, fission and absorption rates, and various temperature coefficients of reactivity (isothermal, fuel, and moderator). From the perspective of storage space, the complete set of LibTorch models uses 11 MB on disk, compared to the 10 GB for the ISOXML multigroup library that covers the same grid space. For the two-dimensional performance case considered in Griffin, the Torch model uses 97% less RAM than the reference ISOXML dataset while runtime increases by a factor of 3 when using the LibTorch model compared to the ISOXML dataset with multi-linear interpolation. The LibTorch model consistently yields errors within 0.01% for most analyzed quantities except for the temperature coefficients of reactivity where the maximum discrepancies are up to 0.3 $\frac{pcm}{K}$. Due to the neural network attempting to best predict quantities with no regard for a positive or negative bias for any given quantity, predictions may experience random fluctuations, resulting in both positive and negative errors. Future work will entail both depletion and coupled transient analysis to determine the predictive capabilities of Griffin with neural network-based cross sections.
We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.
Within the decade, many new ground and space-based observatories will become operational, generating massive amounts of data on short timescales. New surveys like Rubin Observatory's Legacy Survey of Space and Time (LSST) will be capable of observing objects with greater resolution than ever before, but processing and analyzing these datasets optimally will pose a significant challenge. In an effort to prepare for this, we explore how incorporating Deep Neural Networks can better support future data-intensive Astrophysics tasks such as galaxy morphology classification.
Allen–Cahn equation is a reaction–diffusion equation and is widely used for modeling phase separation. Machine learning methods for solving the Allen–Cahn equation in its strong form suffer from inaccuracies in collocation techniques, errors in computing higher-order spatial derivatives, and the large system size required by the space–time approach. To overcome these challenges, we propose solving the gradient flow of the Ginzburg–Landau free energy functional, which is equivalent to the Allen–Cahn equation, thereby avoiding the second-order spatial derivatives associated with the Allen–Cahn equation. A minimizing movement scheme is employed to solve the gradient flow problem, eliminating the complexities of a space–time approach. We utilize a separable neural network that efficiently represents the phase field through low-rank tensor decomposition. As we use the minimizing movement scheme to numerically solve the gradient flow problem, we thus, refer to the proposed method as the Separable Deep Minimizing Movement (SDMM) method. The evaluation of the functional in the minimizing movement scheme using the Gauss quadrature technique bypasses the inaccuracies associated with collocation techniques traditionally used to solve partial differential equations. A hyperbolic tangent transformation is introduced on the phase field prior to the evaluation of the functional to ensure that it remains strictly bounded within the values of the two phases. For this transformation, theoretical guarantee for energy stability of the minimizing movement scheme is established. Our results suggest that this transformation helps to improve the accuracy and efficiency significantly. The proposed method resolves the challenges faced by state-of-the-art machine learning techniques, outperforming them in both accuracy and efficiency. It is also the first machine learning method to achieve an order of magnitude speed improvement over the finite element method. In addition to its formulation and computational implementation, several case studies illustrate the applicability of the proposed method.