Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computational imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Evaluation of U-net-based Image Segmentation Model to Digital Mammography

Detecting suspicious lesions in medical imaging is the important first step in computer-aided detection (CAD) systems. However, detecting abnormalities in breast tissue is difficult due to the lesion's varying size, shape, margin, and contrast with the background tissue. We focused on mass segmentation, a method that provides notable morphological features by outlining contours of masses. Accurate segmentation is crucial for correct diagnosis. Recent advancements in deep learning have improved object detection and segmentation, and these techniques are also being applied to medical imaging studies. We focused on U-net, which is a recently developed mass segmentation algorithm based on a fully convolutional network. The U-net architecture consists of (1) a contracting path to increase the resolution of the output and (2) a symmetric expanding path to better locate the region of interest. The performance of a U-net model was tested with 63 digital mammograms from INbreast, a publicly available database. We trained the model with images resized to 40x40 pixels and conducted 10-fold cross-validation to prevent overfitting. The model's performance with respect to breast density and the lesion's BI-RADS rating was also investigated. Dice coefficients (DC) were used as a performance measure to compare the predicted segmentation of the model with the ground truth. Logistic regression and an analysis of variance were performed to determine the significance of the DCs with regards to breast density and lesion behavior and to calculate the 95% confidence interval. The average DC was 0.80. The difference between DCs for BI-RADS 2 and 4c and for BI-RADS 2 and 5 were significant, suggesting that the model has more difficulty in segmenting benign lesions.

Cho, Priscilla↗
DOE OSTI2021

Surface-sensitive photon avalanche behavior revealed by single-avalanching-nanoparticle imaging

Avalanching nanoparticles (ANPs) are a new class of lanthanide-based upconverting material demonstrating steep optical nonlinearities with the potential to advance applications ranging from subwavelength bioimaging to neuromorphic computing, nanothermometry, and pressure transduction. Here, we use single-nanocrystal imaging to uncover design-dependent heterogeneity in ANP threshold intensity (I th ). Quantitative comparisons between distributions of I th and ANP shell properties reveal correlations between mean I th values, histogram widths, and nanocrystal shell thickness. Evaluating avalanching behaviors using an established model of shell-dependent surface energy transfer shows that variations in shell thickness-and the resultant energy transfer through the shell to the surface and environment-are likely the primary contributor to ANP-to-ANP I th heterogeneity. Further, nanocrystals with an ~6 nm average shell thickness show I th heterogeneity beyond the extent expected from statistical measurements of shell size and variability using transmission electron microscopy (TEM). Here these results provide a principal guide for the design and application of ANPs to environmental sensing.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Deep-learning-aided forward optical coherence tomography endoscope for percutaneous nephrostomy guidance

Percutaneous renal access is the critical initial step in many medical settings. In order to obtain the best surgical outcome with minimum patient morbidity, an improved method for access to the renal calyx is needed. In our study, we built a forward-view optical coherence tomography (OCT) endoscopic system for percutaneous nephrostomy (PCN) guidance. Porcine kidneys were imaged in our experiment to demonstrate the feasibility of the imaging system. Three tissue types of porcine kidneys (renal cortex, medulla, and calyx) can be clearly distinguished due to the morphological and tissue differences from the OCT endoscopic images. To further improve the guidance efficacy and reduce the learning burden of the clinical doctors, a deep-learning-based computer aided diagnosis platform was developed to automatically classify the OCT images by the renal tissue types. Convolutional neural networks (CNN) were developed with labeled OCT images based on the ResNet34, MobileNetv2 and ResNet50 architectures. Nested cross-validation and testing was used to benchmark the classification performance with uncertainty quantification over 10 kidneys, which demonstrated robust performance over substantial biological variability among kidneys. ResNet50-based CNN models achieved an average classification accuracy of 82.6%±3.0%. The classification precisions were 79%±4% for cortex, 85%±6% for medulla, and 91%±5% for calyx and the classification recalls were 68%±11% for cortex, 91%±4% for medulla, and 89%±3% for calyx. Interpretation of the CNN predictions showed the discriminative characteristics in the OCT images of the three renal tissue types. The results validated the technical feasibility of using this novel imaging platform to automatically recognize the images of renal tissue structures ahead of the PCN needle in PCN surgery.

Wang, Chen↗
DOE OSTI2024

Towards interpretable Cryo-EM: disentangling latent spaces of molecular conformations

Molecules are essential building blocks of life and their different conformations (i.e., shapes) crucially determine the functional role that they play in living organisms. Cryogenic Electron Microscopy (cryo-EM) allows for acquisition of large image datasets of individual molecules. Recent advances in computational cryo-EM have made it possible to learn latent variable models of conformation landscapes. However, interpreting these latent spaces remains a challenge as their individual dimensions are often arbitrary. The key message of our work is that this interpretation challenge can be viewed as an Independent Component Analysis (ICA) problem where we seek models that have the property of identifiability. That means, they have an essentially unique solution, representing a conformational latent space that separates the different degrees of freedom a molecule is equipped with in nature. Thus, we aim to advance the computational field of cryo-EM beyond visualizations as we connect it with the theoretical framework of (nonlinear) ICA and discuss the need for identifiable models, improved metrics, and benchmarks. Moving forward, we propose future directions for enhancing the disentanglement of latent spaces in cryo-EM, refining evaluation metrics and exploring techniques that leverage physics-based decoders of biomolecular systems. Moreover, we discuss how future technological developments in time-resolved single particle imaging may enable the application of nonlinear ICA models that can discover the true conformation changes of molecules in nature. The pursuit of interpretable conformational latent spaces will empower researchers to unravel complex biological processes and facilitate targeted interventions. This has significant implications for drug discovery and structural biology more broadly. More generally, latent variable models are deployed widely across many scientific disciplines. Thus, the argument we present in this work has much broader applications in AI for science if we want to move from impressive nonlinear neural network models to mathematically grounded methods that can help us learn something new about nature.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Optimal 3D chemical imaging with multimodal electron tomography

Accurate mapping of nanoscale chemistry in three dimensions (3D) has been a longstanding challenge. Modern electron microscopy provides chemical images by electron energy loss spectroscopy (EELS) and energy dispersive x-ray spectrometry (EDX) but requires high fluences that damage specimens. In 3D, the requirements are worse; electron tomography demands many high-fluence chemical maps for reconstruction, creating a tradeoff between resolution, accuracy, and sample survival. Fused multimodal electron tomography (MM-ET) alleviates this requirement by leveraging lower-fluence high-angle annular dark-field (HAADF) images alongside a few chemical maps to dramatically improve chemical resolution. Here, experimental and computational parameter space is systematically explored to determine when MM-ET performs best. Ideal imaging conditions balance sample survival with resolution and chemical specificity; we recommend a tilt range of at least ± 70°, acquiring 40 equally spaced HAADF projections (signal-to-noise > 10), and 7 EELS/EDX maps of each chemistry (signal-to-noise > 4).

36 MATERIALS SCIENCE↗
DOE OSTI2025

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗
DOE OSTI2021

Multiscale Data-Driven Seismic Full-Waveform Inversion With Field Data Study

Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a nonconvex problem and can encounter local minima due to the limited accuracy of the initial velocity models or the absence of low frequencies in the measurements. To overcome these computational issues, we develop a multiscale data-driven FWI method based on fully convolutional networks (FCNs). In preparing the training data, we first develop a real-time style transform method to create a large set of synthetic subsurface velocity models from natural images. We then develop two convolutional neural networks with encoder-decoder structures to reconstruct the low- and high-frequency components of the subsurface velocity models, separately. To validate the performance of our data-driven inversion method and the effectiveness of the synthesized training set, we compare it with conventional physics-based waveform inversion approaches using both synthetic and field data. Finally, these numerical results demonstrate that, once our model is fully trained, it can significantly reduce the computation time and yield more accurate subsurface velocity models in comparison with conventional FWI.

58 GEOSCIENCES↗
DOE OSTI2021

X-ray microscopy enables multiscale high-resolution 3D imaging of plant cells, tissues, and organs

Capturing complete internal anatomies of plant organs and tissues within their relevant morphological context remains a key challenge in plant science. While plant growth and development are inherently multiscale, conventional light, fluorescence, and electron microscopy platforms are typically limited to imaging of plant microstructure from small flat samples that lack a direct spatial context to, and represent only a small portion of, the relevant plant macrostructures. We demonstrate technical advances with a lab-based X-ray microscope (XRM) that bridge the imaging gap by providing multiscale high-resolution three-dimensional (3D) volumes of intact plant samples from the cell to the whole plant level. Serial imaging of a single sample is shown to provide sub-micron 3D volumes co-registered with lower magnification scans for explicit contextual reference. High-quality 3D volume data from our enhanced methods facilitate sophisticated and effective computational segmentation. Advances in sample preparation make multimodal correlative imaging workflows possible, where a single resin-embedded plant sample is scanned via XRM to generate a 3D cell-level map, and then used to identify and zoom in on sub-cellular regions of interest for high-resolution scanning electron microscopy. In total, we present the methodologies for use of XRM in the multiscale and multimodal analysis of 3D plant features using numerous economically and scientifically important plant systems.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Enhancing the Range and Reliability of the Spacer Layer Imaging Method

The spacer layer imaging method (SLIM) is widely used to measure the thickness of additive and lubricant films, in lubricant development and evaluation, and for fundamental research into elastohydrodynamic lubrication and tribofilm formation mechanisms. The film thickness measurement, as implemented on several popular tribometers, provides powerful, non-destructive in-situ mapping of film topography with nanometre-scale height sensitivity. However, the results can be highly sensitive to experimental procedure, machine condition, and image analysis, in some cases reporting unphysical film thickness trends. The prevailing image analysis techniques make it challenging to interrogate these errors, often hiding their multivariate nonlinear behaviour from the user by spatial averaging. Herein, several common ‘silent errors’ in the SLIM measurement, including colour matching to incorrect fringe orders, and colour drift due to the optical properties of the system or film itself, are discussed, with examples. A robust suite of novel a priori and a posteriori methods to address these issues, and to improve the accuracy and reliability of the measurement, are also presented, including a novel, computationally inexpensive circle-finding algorithm for automated image processing. In combination, these methods allow reliable mapping of films up to at least 800 nm in thickness, representing a significant milestone for the utility of SLIM applied to elastohydrodynamic contact.

EHL film geometry↗
DOE OSTI2024

Uncovering the three-dimensional structure of upconverting core–shell nanoparticles with multislice electron ptychography

In photon upconverting core–shell nanoparticles, structure strongly dictates performance. Typical imaging in scanning transmission electron microscopy has sufficient resolution to probe the atomic structure of these nanoparticles, but contrast, dose, and projection limitations make conventional methods insufficient for fully characterizing these structures. Phase retrieval techniques provide a promising alternative imaging mode, and, in particular, multislice electron ptychography can recover depth-dependent information. Here, we study beam-sensitive photon upconverting core–shell nanoparticles with a multislice ptychography approach using a low electron dose to avoid damage. Large strain fields arise in these heterostructures due to the mismatch in lattice parameter between the core and the shell. We reconstruct both a nanoparticle that appears defect-free and one that has a large break in the side and map the distribution of strain in 3D by computing distortion fields from high-resolution potential images of each slice. In the defect-free nanoparticle, we observe twisting of the shell, while in the broken nanoparticle, we measure the 3D position of the crack, the core, and dislocations. These results highlight the advantage of multislice electron ptychography to recover 3D information from a single scan, even under strict electron dose requirements from beam-sensitive samples.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2025

Integrating Experiments and Simulations to Reveal Anisotropic Growth Mechanisms and Interfaces of a One-Dimensional Zeolite

Zeolites are nanoporous crystalline materials critical for diverse industrial applications, yet their growth mechanisms are poorly understood. Here, this study presents a novel integrated framework combining experimental synthesis, high-resolution imaging, coarse-grained molecular dynamics simulations, and computer vision to uncover the mechanisms of growth of SSZ-24, a 1D channel zeolite. We demonstrate how synthesis conditions, such as temperature and reactant concentration, govern crystal anisotropy and surface roughness with growth dynamics differing markedly by crystallographic orientation. Along the channels, growth involves minimal energy barriers and rapid nucleation, resulting in rough surfaces. In contrast, growth perpendicular to the channels requires cooperative molecular organization and is highly sensitive to thermodynamic and kinetic conditions, yielding smooth anisotropic surfaces under low driving forces. By simulating transmission electron microscopy (TEM) images, we bridge molecular-scale simulations with experimental observations, identifying distinct growth mechanisms along different crystal planes. This work offers molecular-level insights into zeolite crystallization, advancing the rational design of nanoporous materials. The integration of cross-disciplinary methodologies establishes a transformative framework for optimizing zeolite synthesis, with implications for broader classes of materials.

Bertolazzo, Andressa A. [Univ. of Utah, Salt Lake ↗
DOE OSTI2021

Automatic information extraction from neutron radiography imaging to estimate axial fuel expansion in $\mathrm{EBR-II}$

Approximately 130,000 metal fuel pins were irradiated in the Experimental Breeder Reactor II (EBR-II) during its 30 years of operation to develop and characterize existing and prospective fuels. For many of the metal fuel irradiation experiments, neutron radiography imaging was performed to characterize fuel behavior, like fuel swelling. However, due to the lack of technology or resources, many of the images have not been processed or were processed manually through visual examination. This paper represents first-attempt to develop an image processing algorithm capable of automatically extracting information regarding the degree of fuel swelling from neutron radiography imaging. The algorithm was applied to 120 images of three different metallic fuel pin compositions—U-10Zr, U-8Pu-10Zr, and U-19Pu-10Zr. The algorithm performs operations of image intensity adjustment, image binarization, region finding, and labeling to extract information about fuel swelling. The average growth for U-10Zr was found to be 8.49% with 95% Confidence Interval (CI) [8.33 –8.66%], for U-8Pu-10Zr — 7.50% with 95% CI [7.23 – 7.78%], and for U-19Pu-10Zr — 3.15%, with 95% CI [2.40 – 3.91%]. The results obtained by applying this automatic image processing algorithm are consistent with previously reported studies of the same types of fuels. The automatic image processing algorithm will be expanded to include thousands of available neutron radiography images and different types of fuels to investigate empirical dependencies of fuel swelling, which can be subsequently applied to advanced fuel modeling. Results from this study can later be compared to BISON simulations to further benchmark modeling efforts and develop assessment cases.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Abstract Graph learning, when used as a semi-supervised learning (SSL) method, performs well for classification tasks with a low label rate. We provide a graph-based batch active learning pipeline for pixel/patch neighborhood multi- or hyperspectral image segmentation. Our batch active learning approach selects a collection of unlabeled pixels that satisfy a graph local maximum constraint for the active learning acquisition function that determines the relative importance of each pixel to the classification. This work builds on recent advances in the design of novel active learning acquisition functions (e.g., the Model Change approach in arXiv:2110.07739) while adding important further developments including patch-neighborhood image analysis and batch active learning methods to further increase the accuracy and greatly increase the computational efficiency of these methods. In addition to improvements in the accuracy, our approach can greatly reduce the number of labeled pixels needed to achieve the same level of the accuracy based on randomly selected labeled pixels.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Spatiotemporal Learning in Power Modules: Wavelet-Enhanced Forecasting of Thermomechanical Degradation

Detecting internal defects in power electronics packages is critical for their performance and reliability, especially under extreme operating conditions, as these defects can lead to catastrophic failure if not properly addressed. Confocal scanning acoustic microscopy (C-SAM) plays a key role in the nondestructive evaluation of bond layer degradation within a power electronics package by detecting defects such as delamination, voids, and cracks. However, accurately quantifying and predicting these defects from C-SAM images remains a significant challenge due to the low noise-to-signal ratio, which typically arises from both imaging process and bond patterns itself. In this paper, we explore machine learning strategies for processing C-SAM images and providing predictive models of defect growth. We use C-SAM images of sintered copper and sintered silver samples, which are obtained under accelerated thermal experiments, as the representative dataset for our study. We investigate the effect of Fourier transforms and wavelet transforms on these datasets to remove high-frequency noise and address noise across multiple scales with histogram equalization to enhance the contrast and improve the visibility of defects. As a result, defect boundaries can be clearly distinguished, enabling more accurate tracking of their growth over time. We then employ different time-series forecasting algorithms on the denoised images to formulate an image-based lifetime prediction model. Statistical models and deep-learning techniques are trained on images obtained in the early stages of thermal shock, and defect growth in the later stages is predicted. Our work serves as a preliminary attempt to improve the accuracy of lifetime prediction models of power electronics packages, which is critical under extreme operating environments.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Analog In-Memory Computing for the Synthetic Aperture Radar Polar Format Algorithm

As the utility of synthetic aperture radar (SAR) systems increases in autonomous vehicles, satellites, and other power- and space-constrained edge applications, there is a growing need for processors that can form SAR images at low power. In recent years, analog in-memory compute (AIMC) has shown immense promise for accelerating neural networks and other matrix-vector multiplication (MVM) heavy workloads at the edge. Here, in this work, we examine how the polar format algorithm (PFA), a popular SAR image formation algorithm, can be mapped to these AIMC systems. The PFA maps readily onto analog MVMs because it primarily consists of two linear operations: interpolation of frequency-domain data to a Cartesian grid, followed by a 2-D Fourier transform. This work presents two approaches to map the interpolation operation onto MVMs in analog hardware: a chirp transform and a modified form of sinc interpolation. These mappings introduce algorithmic errors, and their effect on the quality of SAR image formation is examined, both quantitatively and qualitatively. In addition, the impact of errors introduced by the analog hardware is explored to determine which approach is optimal under varying assumptions about the underlying analog memory devices and circuits.

Analog computing↗
DOE OSTI2023

Analysis of an optical imaging system prototype for autonomously monitoring zooplankton in an aquaculture facility

Traditional approaches to biomonitoring in aquatic systems, such as sample collection, sorting, and identification, require significant time and effort, thereby limiting the spatiotemporal resolution of sample collection. Additionally, collection and preservation of samples for subsequent taxonomic identification and enumeration leads to mortality of organisms. Recent advances in technologies that utilize optical imaging and machine learning have provided new opportunities to expedite biomonitoring and lead to significant cost savings. These technologies can be advantageous to scientists or managers that conduct routine biomonitoring to inform operations, as in the case of aquaculture facilities. The Small Aquatic Organism optical imaging system (SAO) is a high-throughput optical imaging and classification prototype system that relies on computer vision and machine learning (Support Vector Machines, or SVMs) to autonomously identify and enumerate aquatic organisms. The SAO provides a more sustainable method of collecting large volumes of data and has the benefit of being used in situ. In this study, we tested the performance of the SAO in providing comparable results to manual zooplankton community monitoring in ten ponds at an aquaculture facility. We performed a side-by-side study comparing the sampling methods of plankton tow nets, where major zooplankton taxonomic classes were manually identified and enumerated, to sampling with the SAO. Vouchered samples were used to develop a training library for the SAO, where classes consisted of water boatman and zooplankton groups: cladocerans, copepod adults, copepod nauplii, and rotifers. SAO imagery was manually classified and compared with predicted results for validation. Accuracy for the SVM classifier of the SAO was 37.4 %. Convolutional Neural Networks (CNN) and Random Forest classifiers were also applied to SAO imagery and image features for comparison. The best CNN model and our Random Forest model had accuracies of 80.4 % and 46.6 % respectively. Challenges faced included the small size of copepod nauplii and rotifers and the limited resolution of the imaging camera, although there are tradeoffs between imaging resolution and the sample processing rate. Furthermore, our comparison shows that advancement in both optical imaging and ML are needed in order for the SAO prototype to yield comparable results to manual community monitoring in an aquaculture facility.

54 ENVIRONMENTAL SCIENCES↗