Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “TensorFlow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Towards High-Performance AI4NP Applications on Modern GPU Platforms

The evolution of modern heterogeneous accelerators, such as GPUs, has significantly advanced the landscape of artificial intelligence (AI). There is a notable surge to adopt AI within the nuclear physics domain (AI4NP). While most AI4NP studies focus on feasibility analysis, our attention is directed towards evaluating their performance on contemporary GPUs that integrate tensor cores. We first benchmark the throughput of hyperparameterized multi-layer perceptron (MLP) models. We then examine the performance of an AI4NP application: Hydra. We assess the performance gain and accuracy loss caused by the tensor cores for low-precision floating-point operations. Our experiments encompass the PyTorch and TensorFlow Keras frameworks on NVIDIA’s T4 and A100 GPUs. We explore the behavior of different GPU hardware platforms and AI software tools. This study can be a valuable resource for guiding the performance optimization of larger-scale deployments of AI4NP applications.

Mei, Xinxin↗

FlameNEST: explicit profile likelihoods with the Noble Element Simulation Technique

We present FlameNEST, a framework providing explicit likelihood evaluations in noble element particle detectors using data-driven models from the Noble Element Simulation Technique. FlameNEST provides a way to perform statistical analyses on real data with no dependence on large, computationally expensive Monte Carlo simulations by evaluating the likelihood on an event-by-event basis using analytic probability elements convolved together in a single TensorFlow multiplication. Furthermore, this robust framework creates opportunities for simple inter-collaboration analyses which will be fundamental for the future of experimental dark matter physics.

47 OTHER INSTRUMENTATION↗

ZMPY3D: accelerating protein structure volume analysis through vectorized 3D Zernike moments and Python-based GPU integration

Abstract Motivation Volumetric 3D object analyses are being applied in research fields such as structural bioinformatics, biophysics, and structural biology, with potential integration of artificial intelligence/machine learning (AI/ML) techniques. One such method, 3D Zernike moments, has proven valuable in analyzing protein structures (e.g., protein fold classification, protein–protein interaction analysis, and molecular dynamics simulations). Their compactness and efficiency make them amenable to large-scale analyses. Established methods for deriving 3D Zernike moments, however, can be inefficient, particularly when higher order terms are required, hindering broader applications. As the volume of experimental and computationally-predicted protein structure information continues to increase, structural biology has become a “big data” science requiring more efficient analysis tools. Results This application note presents a Python-based software package, ZMPY3D, to accelerate computation of 3D Zernike moments by vectorizing the mathematical formulae and using graphical processing units (GPUs). The package offers popular GPU-supported libraries such as CuPy and TensorFlow together with NumPy implementations, aiming to improve computational efficiency, adaptability, and flexibility in future algorithm development. The ZMPY3D package can be installed via PyPI, and the source code is available from GitHub. Volumetric-based protein 3D structural similarity scores and transform matrix of superposition functionalities have both been implemented, creating a powerful computational tool that will allow the research community to amalgamate 3D Zernike moments with existing AI/ML tools, to advance research and education in protein structure bioinformatics. Availability and implementation ZMPY3D, implemented in Python, is available on GitHub (https://github.com/tawssie/ZMPY3D) and PyPI, released under the GPL License.

Lai, Jhih-Siang (ORCID:0000000156775890)↗

A Quantitative Study of Deep Learning Training on Heterogeneous Supercomputers

Deep learning (DL) has become a key technique for solving complex problems in scientific research and discovery. DL training for science is substantially challenging because it has to deal with massive quantities of multi-dimensional data. High- performance computing (HPC) supercomputers are increasingly being employed for meeting the exponentially growing demand for DL. Multiple GPUs and high-speed interconnect network are needed for supporting DL on HPC systems. However, the excessive use of GPUs without considering effective benefits leads to inefficient resource utilization of these expensive setups. In this paper, we conduct a quantitative analysis to gauge the efficacy of DL workloads on the latest HPC system and identify viability of next-generation DL-optimized heterogeneous supercomputers for enabling researchers to develop more efficient resource management and distributed DL middleware. We evaluate well- known DL models with large-scale datasets using the popular TensorFlow framework, and provide a thorough evaluation in- cluding scalability, accuracy, variability, storage resource, GPU- GPU/GPU-CPU data transfer, and GPU utilization. Our analysis reveals that the latest heterogeneous supercomputing cluster shows varying performance trend as compared to the existing literature for single- and multi-node training. To the best of our knowledge, this is the first work to conduct such a quantitative and comprehensive study of DL training on a supercomputing system with multiple GPUs.

Han, Jingoo↗

HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization

We present a new software, HYPPO, that enables the automatic tuning of hyperparameters of various deep learning (DL) models. Unlike other hyperparameter optimization (HPO) methods, HYPPO uses adaptive surrogate models and directly accounts for uncertainty in model predictions to find accurate and reliable models that make robust predictions. Using asynchronous nested parallelism, we are able to significantly alleviate the computational burden of training complex architectures and quantifying the uncertainty. HYPPO is implemented in Python and can be used with both TensorFlow and PyTorch libraries. We demonstrate various software features on time-series prediction and image classification problems as well as a scientific application in computed tomography image reconstruction. Finally, we show that (1) we can reduce by an order of magnitude the number of evaluations necessary to find the most optimal region in the hyperparameter space and (2) we can reduce by two orders of magnitude the throughput for such HPO process to complete.

adaptation models↗

Three dimensions, two microscopes, one code: Automatic differentiation for x-ray nanotomography beyond the depth of focus limit

Conventional tomographic reconstruction algorithms assume that one has obtained pure projection images, involving no within-specimen diffraction effects nor multiple scattering. Advances in x-ray nanotomography are leading toward the violation of these assumptions, by combining the high penetration power of x-rays, which enables thick specimens to be imaged, with improved spatial resolution that decreases the depth of focus of the imaging system. We describe a reconstruction method where multiple scattering and diffraction effects in thick samples are modeled by multislice propagation and the 3D object function is retrieved through iterative optimization. We show that the same proposed method works for both full-field microscopy and for coherent scanning techniques like ptychography. Our implementation uses the optimization toolbox and the automatic differentiation capability of the open-source deep learning package TensorFlow, demonstrating a straightforward way to solve optimization problems in computational imaging with flexibility and portability.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks

Going deeper and wider in neural architectures improves their accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) practitioners either need to change to less de- sired network architectures, or nontrivially dissect a network across multiGPUs. These distract DL practitioners from concentrating on their original machine learning tasks. We present SuperNeurons: a dynamic GPU memory scheduling runtime to enable the network training far beyond the GPU DRAM capacity. SuperNeurons features 3 memory optimizations, Liveness Analysis, Unified Tensor Pool, and Cost-Aware Recomputation; together they effectively reduce the network-wide peak memory usage down to the maximal memory usage among layers. We also address the performance issues in these memory-saving techniques. Given the limited GPU DRAM, SuperNeurons not only provisions the necessary memory for the training, but also dynamically allocates the memory for convolution workspaces to achieve the high performance. Evaluations against Caffe, Torch, MXNet and TensorFlow have demonstrated that SuperNeurons trains at least 3.2432 deeper network than current ones with the leading performance. Particularly, SuperNeurons can train ResNet2500 that has 104 basic network layers on a 12GB K40c.

Wang, Linnan↗

A Self-Sustained CPS Design for Reliable Wildfire Monitoring

Continuous monitoring of areas nearby the electric grid is critical for preventing and early detection of devastating wildfires. Existing wildfire monitoring systems are intermittent and oblivious to local ambient risk factors, resulting in poor wildfire awareness. Ambient sensor suites deployed near the gridlines can increase the monitoring granularity and detection accuracy. However, these sensors must address two challenging and competing objectives at the same time. First, they must remain powered for years without manual maintenance due to their remote locations. Second, they must provide and transmit reliable information if and when a wildfire starts. The first objective requires aggressive energy savings and ambient energy harvesting, while the second requires continuous operation of a range of sensors. To the best of our knowledge, this paper presents the first self-sustained cyber-physical system that dynamically co-optimizes the wildfire detection accuracy and active time of sensors. The proposed approach employs reinforcement learning to train a policy that controls the sensor operations as a function of the environment (i.e., current sensor readings), harvested energy, and battery level. Here, the proposed cyber-physical system is evaluated extensively using real-life temperature, wind, and solar energy harvesting datasets and an open-source wildfire simulator. In long-term (5 years) evaluations, the proposed framework achieves 89% uptime, which is 46% higher than a carefully tuned heuristic approach. At the same time, it averages a 2-minute initial response time, which is at least 2.5× faster than the same heuristic approach. Furthermore, the policy network consumes 0.6 mJ per day on the TI CC2652R microcontroller using TensorFlow Lite for Micro, which is negligible compared to the daily sensor suite energy consumption.

54 ENVIRONMENTAL SCIENCES↗

Cycle Consistent Surrogate for Inertial Confinement Fusion

We provide a self-contained package to load, process, and test the ICF JAG dataset. In particular, we include a Neural Network designed to act as a surrogate for the JAG 1D Simulator. The neural network is implemented in Tensorflow. We include a Jupyter notebook which allows a user to load the dataset, load the neural network and train it such that given just the 5 input parameters, it predicts the scalars and images accurately. This can be done directly in the notebook, without any additional modifications. During training, intermediate predictions are also saved to disk (as specified by the user). We hope this serves as a starting point to build, test and play with the ICF-JAG simulation dataset.

Anirudh, Rushil↗

ALFABET™ and nfp (Neural Fingerprint) [SWR-19-13]

Keras layers for end-to-end learning on molecular structure. Based on Keras, Tensorflow, and RDKit. Source code used in the study Message-passing neural networks for high-throughput polymer screening. SEE ALSO https://github.com/NREL/alfabet

St. John, Peter↗

eeyore

The software "eeyore" solves the problem of running Markov chain Monte Carlo (MCMC) algorithms on Bayesian neural networks. There is software for MCMC algorithms and software for neural network (such as PyTorch and TensorFlow), but there are limited solutions for combining the two. The main idea is to create a subclass of the 'torch.nn.Module' for defining Bayesian neural networks with the desired functionality required by MCMC taking the form of additional methods in that sub-class. This way, neural network functionality of PyTorch is exposed to the developer and user, allowing to develop MCMC methods on top of it through a simple API. The code implements Langevin Monte Carlo algorithms and population MCMC algorithms, as well as combinations of these, for Bayesian neural networks. This kind of functionality is not available elsewhere, to the best of the authors' knowledge. Moreover, a simple API is available that allows to implement new MCMC algorithms in the future.

Papamarkou, Theodore↗

FasTensor (FT) v0.0.1

The FasTensor implements the native array data programming mode namely SLOPE for modern data analysis tasks. It is designed to work in parallel on supercomputer and can handle terabyte data analysis tasks. It is thousand times faster than other data analysis systems such as Apache Spark and 38% faster than TensorFlow in dealing with large scale scientific data analysis. It has wide applications in image data analysis and mesh data analysis tasks in physical simulations and other fields.

Wu, Kesheng↗

Diverse Super-Resolution (diversity_SR) [SWR-21-60]

Deep learning tools for enhancing the spatial resolution of wind data. The software is developed in Python using the TensorFlow deep learning package. Models for diversity super-resolution is provided. Included in the package are pretrained models with example code/data to perform the super-resolution as well as tools of training models for different enhancement- or data-types. The super-resolution is an inherently ill-conditioned problem, with multiple high-resolution fields plausibly mapping to the same coarse field. Considitional GANs provide a framework for generating a distribution of high-resolution realizations from a given low-resolution input. Stochastic estimation is used to inform the network of the expected degree and location of sub-grid diversity. The package includes a pretrained network to generate distributions of 10x-enhanced fields of wind data.

Glaws, Andrew↗

Functional Mock-up Interface - Machine Learning Center (FMI-MLC) v1

The Functional Mock-up Interface - Machine Learning Center is a tool to interface the Python-Tensorflow library with building simulation models, to develop and train machine learning-based controls. In particular, this framework allows the use of reinforcement learning through the standardized Functional Mock-up Interface, an open industry standard for co-simulation. First application was the development of a controller for dynamic façade and heating ventilation and air conditioning (HVAC) systems.

Gehbauer, Christoph↗

NEURAL NETWORK FOR COHERENT DIFFRACTION IMAGE INVERSION

A deep neural network model plus automatic differentiation is developed for retrieving phase information from 3D coherent diffraction images. The model is implemented using Tensorflow and the training dataset is generated using physics-based atomistic simulations. Custom codes are written to handle the resampling of diffraction images to oversampling ratios appropriate for the neural network model.

CHAN, HENRY↗

stacked predictive sparse decomposition (SPSD) v1.0

It is an unsupervised representation algorithm used to learn morphometric properties from cellular objects. Many similar/identical open source implementations are widely available in tensorflow, pytorch, etc. And ours is implemented in matlab.

Chang, Hang↗

R-Stream

R-Stream version 3.21.3, including its TensorFlow front-end (TFRCC) and LLVM 9 front-end. This version includes the features developed in the Portacy Phase II contract. R-Stream is a source-to-source automatic parallelization engine. For usage, please refer to the manuals (In particular, user guide and release notes) available in the package.

Meister, Benoit [Reservoir Labs, Inc., Portland, O↗