Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Application of deep neural networks to improve diagnostic accuracy of rheumatoid arthritis using diffuse optical tomography

A set of deep neural network models for rheumatoid arthritis (RA) classification using a highway network, a convolutional neural network and a residual network is proposed based on the data of diffuse optical tomography (DOT) utilising near-infrared light, which ensures early diagnosis of pathophysiological changes resulting from inflammation. A numerical model of the finger is used to generate images to overcome the inherent problem of insufficient clinical DOT images available. The proposed deep neural network models are applied to automatically classify simulated DOT images of inflamed and non-inflamed joints and transfer learning is also used to improve the performance of the classification. The results demonstrate that all three deep neural network methods improve the diagnostic accuracy as compared to the widely applied support vector machine (SVM), especially for high inter-subject variability databases. In cases of distinct modelled severity of disease, residual network achieved the highest accuracy (> 99 %), and both of highway and convolutional neural networks reached 99 %, respectively. However, as the severity of the modelled disease is reduced, this accuracy is reduced to 75.2 % for residual networks. The results indicate that transfer learning can improve the performance of deep neural network methods on RA classification from DOT data and highlight their potential as a computer aided tool in DOT diagnostic systems. (paper)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2014

A molecular line scan in the Hubble deep field north

We present a molecular line scan in the Hubble Deep Field North (HDF-N) that covers the entire 3 mm window (79-115 GHz) using the IRAM Plateau de Bure Interferometer. Our CO redshift coverage spans z ≲ 0.45, 1 ≲ z ≲ 1.9 and all z ≳ 2. We reach a CO detection limit that is deep enough to detect essentially all z > 1 CO lines reported in the literature so far. We have developed and applied different line-searching algorithms, resulting in the discovery of 17 line candidates. We estimate that the rate of false positive line detections is ∼2/17. We identify optical/NIR counterparts from the deep ancillary database of the HDF-N for seven of these candidates and investigate their available spectral energy distributions. Two secure CO detections in our scan are identified with star-forming galaxies at z = 1.784 and at z = 2.047. These galaxies have colors consistent with the 'BzK' color selection and they show relatively bright CO emission compared with galaxies of similar dust continuum luminosity. We also detect two spectral lines in the submillimeter galaxy HDF 850.1 at z = 5.183. We consider an additional nine line candidates as high quality. Our observations also provide a deep 3 mm continuum map (1σ noise level = 8.6 μJy beam{sup –1}). Via a stacking approach, we find that optical/MIR bright galaxies contribute only to <50% of the star formation rate density at 1 < z < 3, unless high dust temperatures are invoked. The present study represents a first, fundamental step toward an unbiased census of molecular gas in 'normal' galaxies at high-z, a crucial goal of extragalactic astronomy in the ALMA era.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2013

THE 2012 HUBBLE ULTRA DEEP FIELD (UDF12): OBSERVATIONAL OVERVIEW

We present the 2012 Hubble Ultra Deep Field campaign (UDF12), a large 128 orbit Cycle 19 Hubble Space Telescope program aimed at extending previous Wide Field Camera 3 (WFC3)/IR observations of the UDF by quadrupling the exposure time in the F105W filter, imaging in an additional F140W filter, and extending the F160W exposure time by 50%, as well as adding an extremely deep parallel field with the Advanced Camera for Surveys (ACS) in the F814W filter with a total exposure time of 128 orbits. The principal scientific goal of this project is to determine whether galaxies reionized the universe; our observations are designed to provide a robust determination of the star formation density at z ∼> 8, improve measurements of the ultraviolet continuum slope at z ∼ 7-8, facilitate the construction of new samples of z ∼ 9-10 candidates, and enable the detection of sources up to z ∼ 12. For this project we committed to combining these and other WFC3/IR imaging observations of the UDF area into a single homogeneous dataset to provide the deepest near-infrared observations of the sky. In this paper we present the observational overview of the project and describe the procedures used in reducing the data as well as the final products that were produced. We present the details of several special procedures that we implemented to correct calibration issues in the data for both the WFC3/IR observations of the main UDF field and our deep 128 orbit ACS/WFC F814W parallel field image, including treatment for persistence, correction for time-variable sky backgrounds, and astrometric alignment to an accuracy of a few milliarcseconds. We release the full, combined mosaics comprising a single, unified set of mosaics of the UDF, providing the deepest near-infrared blank-field view of the universe currently achievable, reaching magnitudes as deep as AB ∼ 30 mag in the near-infrared, and yielding a legacy dataset on this field.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Deep neural network uncertainty quantification for LArTPC reconstruction

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. Here we assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.

47 OTHER INSTRUMENTATION↗
DOE OSTI2013

A PILOT FOR A VERY LARGE ARRAY H I DEEP FIELD

High-resolution 21 cm H I deep fields provide spatially and kinematically resolved images of neutral hydrogen at different redshifts, which are key to understanding galaxy evolution across cosmic time and testing predictions of cosmological simulations. Here we present results from a pilot for an H I deep field done with the Karl G. Jansky Very Large Array (VLA). We take advantage of the newly expanded capabilities of the telescope to probe the redshift interval 0 < z < 0.193 in one observation. We observe the COSMOS field for 50 hr, which contains 413 galaxies with optical spectroscopic redshifts in the imaged field of 34' Multiplication-Sign 34' and the observed redshift interval. We have detected neutral hydrogen gas in 33 galaxies in different environments spanning the probed redshift range, including three without a previously known spectroscopic redshift. The detections have a range of H I and stellar masses, indicating the diversity of galaxies we are probing. We discuss the observations, data reduction, results, and highlight interesting detections. We find that the VLA's B-array is the ideal configuration for H I deep fields since its long spacings mitigate radio frequency interference. This pilot shows that the VLA is ready to carry out such a survey, and serves as a test for future H I deep fields planned with other Square Kilometer Array pathfinders.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Improving deep learning-based protein distance prediction in CASP14

Abstract Motivation Accurate prediction of residue–residue distances is important for protein structure prediction. We developed several protein distance predictors based on a deep learning distance prediction method and blindly tested them in the 14th Critical Assessment of Protein Structure Prediction (CASP14). The prediction method uses deep residual neural networks with the channel-wise attention mechanism to classify the distance between every two residues into multiple distance intervals. The input features for the deep learning method include co-evolutionary features as well as other sequence-based features derived from multiple sequence alignments (MSAs). Three alignment methods are used with multiple protein sequence/profile databases to generate MSAs for input feature generation. Based on different configurations and training strategies of the deep learning method, five MULTICOM distance predictors were created to participate in the CASP14 experiment. Results Benchmarked on 37 hard CASP14 domains, the best performing MULTICOM predictor is ranked 5th out of 30 automated CASP14 distance prediction servers in terms of precision of top L/5 long-range contact predictions [i.e. classifying distances between two residues into two categories: in contact (<8 Angstrom) and not in contact otherwise] and performs better than the best CASP13 distance prediction method. The best performing MULTICOM predictor is also ranked 6th among automated server predictors in classifying inter-residue distances into 10 distance intervals defined by CASP14 according to the precision of distance classification. The results show that the quality and depth of MSAs depend on alignment methods and sequence databases and have a significant impact on the accuracy of distance prediction. Using larger training datasets and multiple complementary features improves prediction accuracy. However, the number of effective sequences in MSAs is only a weak indicator of the quality of MSAs and the accuracy of predicted distance maps. In contrast, there is a strong correlation between the accuracy of contact/distance predictions and the average probability of the predicted contacts, which can therefore be more effectively used to estimate the confidence of distance predictions and select predicted distance maps. Availability and implementation The software package, source code and data of DeepDist2 are freely available at https://github.com/multicom-toolbox/deepdist and https://zenodo.org/record/4712084#.YIIM13VKhQM. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Large-scale deep learning for metastasis detection in pathology reports

Objectives No existing algorithm can reliably identify metastasis from pathology reports across multiple cancer types and the entire US population. In this study, we develop a deep learning model that automatically detects patients with metastatic cancer by using pathology reports from many laboratories and of multiple cancer types. Materials and Methods We use 60 471 unstructured pathology reports from 4 Surveillance, Epidemiology, and End Results (SEER) registries. The reports were coded into 1 of 3 labels: metastasis negative, metastases positive, or metastasis undetermined. We utilize a task-specific deep neural network trained from scratch and compare its performance with a widely used large language model (LLM). Results Our deep learning architecture trained on task-specific data outperforms a general-purpose LLM, with a recall of 0.894 compared to 0.824. We quantified model uncertainty and used it to defer reports for human review. We found that retaining 72.9% of reports increased recall from 0.894 to 0.969. Discussion A smaller deep learning architecture trained on task-specific data outperforms a general LLM. Equally critical to model performance is the incorporation of uncertainty quantification, achieved here through an abstention mechanism. Conclusions This study’s finding demonstrate the feasibility of developing algorithms to automatically identify metastatic cancer cases from unstructured pathology reports.

machine learning↗
DOE OSTI2017

Deep inelastic scattering as a probe of entanglement

Using nonlinear evolution equations of QCD, we compute the von Neumann entropy of the system of partons resolved by deep inelastic scattering at a given Bjorken x and momentum transfer q 2 = - Q 2 . We interpret the result as the entropy of entanglement between the spatial region probed by deep inelastic scattering and the rest of the proton. At small x the relation between the entanglement entropy S ( x ) and the parton distribution x G ( x ) becomes very simple: S ( x ) = ln [ x G ( x ) ] . In this small x , large rapidity Y regime, all partonic microstates have equal probabilities—the proton is composed by an exponentially large number exp ( Δ Y ) of microstates that occur with equal and exponentially small probabilities exp ( - Δ Y ) , where Δ is defined by x G ( x ) ~ 1 / x Δ . For this equipartitioned state, the entanglement entropy is maximal—so at small x , deep inelastic scattering probes a maximally entangled state. Here, we propose the entanglement entropy as an observable that can be studied in deep inelastic scattering. This will then require event-by-event measurements of hadronic final states, and would allow to study the transformation of entanglement entropy into the Boltzmann one. We estimate that the proton is represented by the maximally entangled state at x ≤ 10 -3 ; this kinematic region will be amenable to studies at the Electron Ion Collider.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2019

Towards Native Execution of Deep Learning on a Leadership-Class HPC System

Large parallel machines generally offer the best parallel performance with "native execution" that is achieved using codes developed with the optimized compilers, communication libraries, and runtimes offered on the machines. In this paper, we report and analyze performance results from native execution of deep learning on a leadership-class high-performance computing (HPC) system. Using our new code called DeepEx, we present a study of the parallel speed up and convergence rates of learning achieved with native parallel execution. In the trade-off between computational parallelism and synchronized convergence, we first focus on maximizing parallelism while still obtaining convergence. Scaling results are reported from execution on up to 15,000 GPUs using two scientific data sets from atom microscopy and protein folding applications, and also using the popular ImageNet data set. In terms of the traditional measure of parallel speed up, excellent scaling is observed up to 12,000 GPUs. Additionally, accounting for convergence rates of deep learning accuracy or error, a deep learning-specific metric called "learning speed up" is also tracked. The performance results indicate the need to evaluate parallel deep learning execution in terms of learning speed up, and point to additional directions for improved exploitation of high-end HPC systems.

Yoginath, Srikanth↗
DOE OSTI2016

A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers

Current Deep Learning models use highly optimized convolutional neural networks (CNN) trained on large graphical processing units (GPU)-based computers with a fairly simple layered network topology, i.e., highly connected layers, without intra-layer connections. Complex topologies have been proposed, but are intractable to train on current systems. Building the topologies of the deep learning network requires hand tuning, and implementing the network in hardware is expensive in both cost and power.In this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing (HPC) to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. Due to input size limitations of current quantum computers we use the MNIST dataset for our evaluation.The results show the possibility of using the three architectures in tandem to explore complex deep learning networks that are untrainable using a von Neumann architecture. We show that a quantum computer can find high quality values of intra-layer connections and weights, while yielding a tractable time result as the complexity of the network increases; a high performance computer can find optimal layer-based topologies; and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware.This represents a new capability that is not feasible with current von Neumann architecture. It potentially enables the ability to solve very complicated problems unsolvable with current computing technologies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Comparison and Interpretation of Admittance Spectroscopy and Deep Level Transient Spectroscopy from Co-Evaporated and Solution-Deposited Cu2ZnSn(Sx, Se1-x)4 Solar Cells

Cu2ZnSn(S, Se)4 (CZTSe) is an earth-abundant semiconductor with potential for economical thin-film photovoltaic devices. Short minority carrier lifetimes contribute to low open circuit voltage and efficiency. Deep level defects that may contribute to lower minority carrier lifetimes in kesterites have been theoretically predicted, however very little experimental characterization of these deep defects exists. In this work we use admittance spectroscopy (AS) and deep level transient spectroscopy (DLTS) to characterize devices built using CZTSSe absorber layers deposited via both coevaporation and solution processing. AS reveals a band of widely-distributed activation energies for traps or energy barriers for transport, especially in the solution deposited case. DLTS reveals signatures of deep majority and minority traps within both types of samples.

14 SOLAR ENERGY↗
DOE OSTI2024

Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications

Run to run variability in parallel programs caused by floating-point non-associativity has been known to significantly affect reproducibility in iterative algorithms, due to accumulating errors. Non-reproducibility can critically affect the efficiency and effectiveness of correctness testing for stochastic programs. Recently, the sensitivity of deep learning training and inference pipelines to floating-point non-associativity has been found to sometimes be extreme. It can prevent certification for commercial applications, accurate assessment of robustness and sensitivity, and bug detection. New approaches in scientific computing applications have coupled deep learning models with high-performance computing, leading to an aggravation of debugging and testing challenges. Here we perform an investigation of the statistical properties of floating-point non-associativity within modern parallel programming models, and analyze performance and productivity impacts of replacing atomic operations with deterministic alternatives on GPUs. We examine the recently-added deterministic options in PyTorch within the context of GPU deployment for deep learning, uncovering and quantifying the impacts of input parameters triggering run to run variability and reporting on the reliability and completeness of the documentation. Finally, we evaluate the strategy of exploiting automatic determinism that could be provided by deterministic hardware, using the Groq LPUTM accelerator for inference portions of the deep learning pipeline. We demonstrate the benefits that a hardware-based strategy can provide within reproducibility and correctness efforts.

Shanmugavelu, Sanjif↗
DOE OSTI2018

Scalable FPGA Accelerator for Deep Convolutional Neural Networks with Stochastic Streaming

Here, FPGA-based heterogeneous computing platform, due to its extreme logic reconfigurability, emerges to be a strong contender as computing fabric in modern AI. As a result, various FPGA-based accelerators for deep CNN—the key driver of modern AI—have been proposed due to their advantages of high performance, reconfigurability, and fast development round, etc. In general, the consensus among researchers is that, although FPGA-based accelerator can achieve much higher energy efficiency, its raw computing performance lags behind when compared with GPUs with similar logic density. In this paper, we develop an alternative methodology to efficiently implement CNNs with FPGAs that outperform GPUs in terms of both power consumption and performance. Our key idea is to design a scalable hardware architecture and circuit design for large-scale CNNs that leverages a stochastic-based computing principle. Specifically, there are three major performance advantages. First, all key components of our deep learning CNN are designed and implemented to compute stochastically, thus achieving excellent computing performance and energy efficiency. Second, because our proposed CNN architecture enables a stream-mode computing, all of its stages can process even the partial results from preceding stages, therefore not incurring unnecessary latency due to data dependency. Finally, our FPGA-based deep CNN also provides a superior hardware scalability when compared with conventional FPGA implementations by reducing the bandwidth requirement between layers. The results show that our proposed CNN architecture significantly outperforms all previous FPGA-based deep CNN implementation approaches. It achieves 1.58x more GOPS, 6.42x more GOPS/Slice, and 10.92x more GOPS/W when compared with state-of-the-art CNN architecture. The top-5 accuracy of stochastic VGG-16 CNN is 86.77 percent with 18.91 fps frame rate.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Deeply learning deep inelastic scattering kinematics

We study the use of deep learning techniques to reconstruct the kinematics of the neutral current deep inelastic scattering (DIS) process in electron–proton collisions. In particular, we use simulated data from the ZEUS experiment at the HERA accelerator facility, and train deep neural networks to reconstruct the kinematic variables Q 2 and x. Our approach is based on the information used in the classical construction methods, the measurements of the scattered lepton, and the hadronic final state in the detector, but is enhanced through correlations and patterns revealed with the simulated data sets. We show that, with the appropriate selection of a training set, the neural networks sufficiently surpass all classical reconstruction methods on most of the kinematic range considered. Rapid access to large samples of simulated data and the ability of neural networks to effectively extract information from large data sets, both suggest that deep learning techniques to reconstruct DIS kinematics can serve as a rigorous method to combine and outperform the classical reconstruction methods.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Deep Learning and Structural Imaging of Materials

Deep learning has had a transformative effect on numerous domains and is actively utilized by many scientists in data-intensive fields such as high-energy physics and cosmology. Materials science, and in particular, the structural imaging of materials with electrons and X-rays are projected to enter the age of scientific data torrents, positioning them as new application spaces for modern artificial intelligence. In this contribution, we provide a synopsis on the foundations and latest progress in deep learning and present an in-depth application of utilizing modern deep artificial neural networks in scanning transmission electron microscopy to extract structural material properties. We use this case study to expose the strengths of deep learning-based models and discuss their current limitations, in the process highlighting their potential use in other data-intensive structural imaging modalities.

Laanait, Nouamane↗
DOE OSTI2017

ShapeShop: Towards Understanding Deep Learning Representations via Interactive Experimentation

Deep learning is the driving force behind many recent technologies; however, deep neural networks are often viewed as “black-boxes” due to their internal complexity that is hard to understand. Little research focuses on helping people explore and understand the relationship between a user’s data and the learned representations in deep learning models. We present our ongoing work, ShapeShop, an interactive system for visualizing and understanding what semantics a neural network model has learned. Built using standard web technologies, ShapeShop allows users to experiment with and compare deep learning models to help explore the robustness of image classifiers.

Hohman, Frederick M.↗
DOE OSTI2024

A Deep Multimodal Representation Learning Framework for Accurate Molecular Properties Prediction

Drug discovery is a complex and challenging process, requiring the optimization of candidate compounds to identify those with the potential to become safe and effective drugs. Predicting molecular properties is an indispensable step in the drug discovery pipeline. Traditionally, this process is costly and time-intensive, involving multiple rounds of experiments and clinical trials, rendering it impractical for every candidate compound. Deep learning techniques have emerged as a promising approach to drug discovery to reduce the cost and time required to identify novel drugs. However, prevalent research in deep learning models focused on predicting molecular properties has primarily fixated on single-modal models, which utilize a single modality of data, neglecting the potential benefits of combining different data modalities. To overcome this limitation, we introduce MRL-Mol: a deep \textbf{M}ultimodal \textbf{R}epresentation \textbf{L}earning framework for accurate \textbf{Mol}ecular properties prediction. MRL-Mol harnesses three data modalities: sequence, graph, and image, augmenting the depth of comprehension. Leveraging a large-scale unlabeled dataset~($\sim$1M unique molecules), we pretrain MRL-Mol to extract inter- and intra-modal information. Our study demonstrates the superior performance of MRL-Mol in predicting molecular properties across six benchmark datasets, including both classification and regression tasks. Notably, MRL-Mol outperforms other state-of-the-art molecular properties prediction models. These findings suggest that by combining information from multiple data modalities, MRL-Mol can comprehend molecules better than single-modal deep learning models and identify molecular properties with better accuracy.

Yang, Yuxin↗
DOE OSTI2022

Research on Deformation Law of Deep Foundation Pit of Station in Core Region of Saturated Soft Loess Based on Monitoring

Saturated soft loess has a large pore structure, high compressibility, low strength, fluid plastic state, and poor engineering properties. It is still one of the key problems that engineering needs to solve. In order to study the influence of deep foundation pit excavation in the saturated soft loess area on the deformation of foundation pits, the deformation laws of ground settlement, enclosure structure, and supporting axial force were clarified based on the field tests on the deformation characteristics of deep foundation pits in areas with the high-water level in saturated soft loess, combined with geological conditions and on-site construction procedures. The results indicate the following: water supply and construction process were found to be the main factor in changing the surface settlement curve of deep foundation pits in saturated soft loess; increasing the construction speed of the pit bottom floor, inverted braces, floor frame beams, and sidewall frame beams to close the structure, which is conducive to restraining the deformation of the continuous underground wall and foundation pits in similar areas. In the initial stage of support layout, the axial force of steel support tends to increase too fast or even exceed the standard control value. Therefore, a reasonable preadding axial force is an effective means to control the deformation of the continuous underground wall and the axial force of the steel support. The current research results may provide a reference for constructing deep foundation pits in similar areas.

Wang, Xueyan↗