Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Nuclear Physics Network Requirements Review (Final Report)

The Energy Sciences Network (ESnet) is the high-performance network user facility for the US Department of Energy (DOE) Office of Science (SC) and delivers highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the DOE science mission by connecting all its laboratories and facilities in the US and abroad. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) program and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet interconnects DOE national laboratories, user facilities, and major experiments so that scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large datasets, and access distributed data repositories. ESnet is specifically built to provide Between July 2023 and October 2023, ESnet and the Nuclear Physics program (NP) of the DOE SC organized an ESnet requirements review of NP-supported activities. Preparation for these events included identification of key stakeholders: program and facility management, research groups, and technology providers. Each stakeholder group was asked to prepare formal case study documents about its relationship to the NP program to build a complete understanding of the current, near-term, and long-term status, expectations, and processes that will support the science going forward.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Recommended Nuclear Structure and Decay Data for A = 200 Isobars

Evaluated nuclear structure and decay data for all nuclei with mass number A = 200 ( 200 Os, 200 Ir, 200 Pt, 200 Au, 200 Hg, 200 Tl, 200 Pb, 200 Bi, 200 Po, 200 At, 200 Rn, and 200 Fr), are presented. All available experimental data are compiled and evaluated, and best values for level and gamma-ray energies, quantum numbers, lifetimes, gamma-ray intensities and transition probabilities, as well as other nuclear properties, are recommended. Inconsistencies and discrepancies that exist in the literature are discussed. A number of computer codes (https://www-nds.iaea.org/public/ensdf pgm/) developed by members of the NSDD network were used during the evaluation process. In conclusion, this work supersedes the earlier evaluation by F.G. Kondev and S. Lalkovski (2007Ko42), published in Nuclear Data Sheets 108, 1471 (2007).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

pyTCR: A tropical cyclone rainfall model for python

pyTCR is a climatology software package developed in the Python programming language. It integrates the capabilities of several legacy physical models and increases computational efficiency to allow rapid estimation of tropical cyclone (TC) rainfall consistent with the large-scale environment. Specifically, pyTCR implements a horizontally distributed and vertically integrated model [Zhu et al., 2013] for simulating rainfall driven by TCs. Along storm tracks, rainfall is estimated by computing the cross-boundary-layer, upward water vapor transport caused by different mechanisms including frictional convergence, vortex stretching, large-scale baroclinic effect (i.e., wind shear), topographic forcing, and radiative cooling [Lu et al., 2018]. The package provides essential functionalities for modeling and interpreting spatio-temporal TC rainfall data. pyTCR requires a limited number of model input parameters, making it a convenient and useful tool for analyzing rainfall mechanisms driven by TCs. To sample rare (most intense) rainfall events that are often of great societal interest, pyTCR adapts and leverages outputs from a statistical-dynamical TC downscaling model [Lin et al., 2023] capable of rapidly generating a large number of synthetic TCs given a certain climate. As a result, pyTCR significantly reduces computational effort and improves the efficiency in capturing extreme TC rainfall events at the tail of the distributions from limited datasets. Furthermore, the TC downscaling model is forced entirely by large-scale environmental conditions from reanalysis data or coupled General Circulation Models (GCMs), simplifying the projection of TC-induced rainfall and wind speed under future climate using pyTCR. Finally, pyTCR can be coupled with hydrological and wind models to assess risks associated with independent and compound events (e.g., storm surges and freshwater flooding).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Federated learning for 2D synchrotron x-ray diffractometry: a cross-institutional approach for phase quantification of Ti–6Al–4V alloy

High-energy Two dimensional (2D) synchrotron x-ray diffractometry provides important insights into the atomistic structure and phase evolution of materials, yet traditional analysis methods remain complex, knowledge-intensive, and computationally demanding. Deep-learning models offer a powerful alternative for automating their analysis. Institutions that hold these datasets may be unwilling to share their data due to privacy and security policies, as well as the challenges associated with large-scale data transfer. As a result, models trained on local datasets often perform well only on their own data but exhibit bias and poor generalization across different instruments or facilities. To overcome these limitations, we explore federated learning (FL) for 2D synchrotron diffractograms, enabling collaborative model training without exchanging raw data. In this study, 2D synchrotron diffractograms of Ti–6Al–4V alloy collected from two independent facilities are used to train convolutional neural networks for predicting the β-phase volume fraction. Experimental results show that federated global models significantly outperform locally trained models in terms of generalization and achieve accuracy comparable to centralized trained models. These findings demonstrate the potential of FL to enable secure, cross-institutional collaboration and enhance the scalability of deep-learning-based materials characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Streaming readout for next generation electron scattering experiments

Current and future experiments at the high-intensity frontier are expected to produce an enormous amount of data that needs to be collected and stored for offline analysis. Thanks to the continuous progress in computing and networking technology, it is now possible to replace the standard ‘triggered’ data acquisition systems with a new, simplified and outperforming scheme. ‘Streaming readout’ (SRO) DAQ aims to replace the hardware-based trigger with a much more powerful and flexible software-based one, that considers the whole detector information for efficient real-time data tagging and selection. Considering the crucial role of DAQ in an experiment, validation with on-field tests is required to demonstrate SRO performance. In this paper, we report results of the on-beam validation of the Jefferson Lab SRO framework. In this work, we exposed different detectors (PbWO-based electromagnetic calorimeters and a plastic scintillator hodoscope) to the Hall-D electron-positron secondary beam and to the Hall-B production electron beam, with increasingly complex experimental conditions. By comparing the data collected with the SRO system against the traditional DAQ, we demonstrate that the SRO performs as expected. Furthermore, we provide evidence of its superiority in implementing sophisticated AI-supported algorithms for real-time data analysis and reconstruction.

47 OTHER INSTRUMENTATION↗
DOE OSTI2014

Approaches to advancescientific understanding of macrosystems ecology

Macrosystem ecological studies inherently investigate processes that interact across multiple spatial and temporal scales, requiring intensive sampling and massive amounts of data from diverse sources to incorporate complex cross-scale and hierarchical interactions. Inherent challenges associated with these characteristics include high computational demands, data standardization and assimilation, identification of important processes and scales without prior knowledge, and the need for large, cross-disciplinary research teams that conduct long-term studies. Therefore, macrosystem ecology studies must utilize a unique set of approaches that are capable of encompassing these methodological characteristics and associated challenges. Several case studies demonstrate innovative methods used in current macrosystem ecology studies.

data management↗
DOE OSTI2022

Energy Efficient Material Processing through Automated Process Monitoring and Controls

Smart manufacturing is bound to play a crucial role in reducing global energy consumption while accelerating economic development. This phenomenon is evident in advanced sensing technology developments, data analytics and machine learning/AI, automated controls, cloud computing, etc. The immediate opportunity for smart manufacturing is to improve the energy efficiency of manufacturing operations through innovations in processes and controls. The manufacturing industry still consumes about 30% of total global energy production, which is significant. This program focused on the recommendation of heterogeneous sensors to monitor the Chemical Vapor Infiltration (CVI) process parameters, states, and key performance indices and utilize cloud computing to sort and analyze the real-time data. Learning through data analytics helps guide the control parameters affecting energy usage. The target process in the program is the CVI process at the Honeywell South Bend facility, which is one of the most energy-intensive manufacturing industries.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2017

Divide and Conquer (DC) BLAST: fast and easy BLAST execution within HPC environments

Bioinformatics is currently faced with very large-scale data sets that lead to computational jobs, especially sequence similarity searches, that can take absurdly long times to run. For example, the National Center for Biotechnology Information (NCBI) Basic Local Alignment Search Tool (BLAST and BLAST+) suite, which is by far the most widely used tool for rapid similarity searching among nucleic acid or amino acid sequences, is highly central processing unit (CPU) intensive. While the BLAST suite of programs perform searches very rapidly, they have the potential to be accelerated. In recent years, distributed computing environments have become more widely accessible and used due to the increasing availability of high-performance computing (HPC) systems. Therefore, simple solutions for data parallelization are needed to expedite BLAST and other sequence analysis tools. However, existing software for parallel sequence similarity searches often requires extensive computational experience and skill on the part of the user. In order to accelerate BLAST and other sequence analysis tools, Divide and Conquer BLAST (DCBLAST) was developed to perform NCBI BLAST searches within a cluster, grid, or HPC environment by using a query sequence distribution approach. Scaling from one (1) to 256 CPU cores resulted in significant improvements in processing speed. Thus, DCBLAST dramatically accelerates the execution of BLAST searches using a simple, accessible, robust, and parallel approach. DCBLAST works across multiple nodes automatically and it overcomes the speed limitation of single-node BLAST programs. DCBLAST can be used on any HPC system, can take advantage of hundreds of nodes, and has no output limitations. Thus, this freely available tool simplifies distributed computation pipelines to facilitate the rapid discovery of sequence similarities between very large data sets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

A Bayesian Approach for Quantifying Data Scarcity when Modeling Human Behavior via Inverse Reinforcement Learning

Computational models that formalize complex human behaviors enable study and understanding of such behaviors. However, collecting behavior data required to estimate the parameters of such models is often tedious and resource intensive. Thus, estimating dataset size as part of data collection planning (also known as Sample Size Determination) is important to reduce the time and effort of behavior data collection while maintaining an accurate estimate of model parameters. In this paper, we present a sample size determination method based on Uncertainty Quantification (UQ) for a specific Inverse Reinforcement Learning (IRL) model of human behavior, in two cases: 1) pre-hoc experiment design—conducted in the planning stage before any data is collected, to guide the estimation of how many samples to collect; and 2) post-hoc dataset analysis—performed after data is collected, to decide if the existing dataset has sufficient samples and whether more data is needed. Here, we validate our approach in experiments with a realistic model of behaviors of people with Multiple Sclerosis (MS) and illustrate how to pick a reasonable sample size target. Our work enables model designers to perform a deeper, principled investigation of effects of dataset size on IRL.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Improving Large-scale Storage System Performance via Topology-aware and Balanced Data Placement

With the advent of big data, the I/O subsystems of large-scale compute clusters are becoming a center of focus, with more applications putting greater demands on end-to-end I/O performance. These subsystems are often complex in design. They comprise of multiple hardware and software layers to cope with the increasing capacity, capability and scalability requirements of data intensive applications. The sharing nature of storage resources and the intrinsic interactions across these layers make it to realize user-level, end-to-end performance gains a great challenge. We propose a topology-aware resource load balancing strategy to improve per-application I/O performance. We demonstrate the effectiveness of our algorithm on an extreme-scale compute cluster, Titan, at the Oak Ridge Leadership Computing Facility (OLCF). Our experiments with both synthetic benchmarks and a real-world application show that, even under congestion, our proposed algorithm can improve large-scale application I/O performance significantly, resulting in both the reduction of application run times and higher resolution simulation runs.

Wang, Feiyi↗
DOE OSTI2023

Computing rank‐revealing factorizations of matrices stored out‐of‐core

This paper describes efficient algorithms for computing rank-revealing factorizations of matrices that are too large to fit in main memory (RAM), and must instead be stored on slow external memory devices such as disks (out-of-core or out-of-memory). Traditional algorithms for computing rank-revealing factorizations (such as the column pivoted QR factorization and the singular value decomposition) are very communication intensive as they require many vector-vector and matrix-vector operations, which become prohibitively expensive when data is not in RAM. Randomization allows to reformulate new methods so that large contiguous blocks of the matrix are processed in bulk. The paper describes two distinct methods. The first is a blocked version of column pivoted Householder QR, organized as a “left-looking” method to minimize the number of the expensive write operations. The second method results employs a UTV factorization. It is organized as an algorithm-by-blocks to overlap computations and I/O operations. As it incorporates power iterations, it is much better at revealing the numerical rank. Numerical experiments on several computers demonstrate that the new algorithms are almost as fast when processing data stored on slow memory devices as traditional algorithms are for data stored in RAM.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Breaking Down the Computational Barriers to Real-Time Urban Flood Forecasting

Flooding impacts are on the rise globally, and concentrated in urban areas. Currently, there are no operational systems to forecast flooding at spatial resolutions that can facilitate emergency preparedness and response actions mitigating flood impacts. We present a framework for real-time flood modeling and uncertainty quantification that combines the physics of fluid motion with advances in probabilistic methods. The framework overcomes the prohibitive computational demands of high-fidelity modeling in real-time by using a probabilistic learning method relying on surrogate models that are trained prior to a flood event. This shifts the overwhelming burden of computation to the trivial problem of data storage, and enables forecasting of both flood hazard and its uncertainty at scales that are vital for time-critical decision-making before and during extreme events. The framework has the potential to improve flood prediction and analysis and can be extended to other hazard assessments requiring intense high-fidelity computations in real-time.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Multi phenomena melt pool sensor data fusion for enhanced process monitoring of laser powder bed fusion additive manufacturing

Finding actionable trends in laser-based metal additive manufacturing process monitoring data is challenging owing to the diversity and complexity of the underlying physical interactions. A single monitoring solution that captures a particular process phenomenon, such as a photodiode that tracks melt pool intensity, is not alone capable of evaluating process stability or detecting flaw formation with sufficient precision for routine application in industry. In this work, to improve flaw detection performance, we adopted a data fusion approach that captures multiple process phenomena. To demonstrate this, we acquired data from laser powder bed fusion (LPBF) builds of cylindrical specimens produced with different laser spot sizes, emulating defocusing due to process faults such as thermal lensing. The resulting specimens had porosity of varying types and severity, quantified by post-build non-destructive X-ray computed tomography, Archimedes density measurements, and destructive metallographic characterization. During the build, the melt pool state was monitored with two coaxial high-speed video cameras and a temperature field imaging system. Physically intuitive low-level melt pool signatures, such as melt pool temperature, shape and size, and spatter intensity were extracted from this high-dimensional, image-based sensor data. These process signatures were subsequently used as input features in relatively simple machine learning models, such as a support vector machine, which were trained to detect laser defocusing, and in addition, predict porosity type and severity. The results show that the data fusion approach significantly enhanced system performance by reducing the overall false positive rate from ~ 0.1 to ~ 0.001 without sacrificing the true positive rate (~0.90). These results were at par with a black-box, deep machine learning approach (convolutional neural network).

36 MATERIALS SCIENCE↗
DOE OSTI2014

Improving large-scale storage system performance via topology-aware and balanced data placement

With the advent of big data, the I/O subsystems of large-scale compute clusters are becoming a center of focus. More applications are putting greater demands on end-to-end I/O performance. These subsystems are often complex in design. They comprise of multiple hardware and software layers to cope with the increasing capacity, capability, and scalability requirements of data intensive applications. However, the sharing nature of storage resources and the intrinsic interactions across these layers make it a great challenge to realize end-to-end performance gains. This paper proposes a topology-aware strategy to balance the load across resources, to improve the per-application I/O performance. We demonstrate the effectiveness of our algorithm on an extreme-scale compute cluster, Titan, at the Oak Ridge Leadership Computing Facility (OLCF). Our experiments with both synthetic benchmarks and a real-world application show that, even under congestion, our proposed algorithm can improve large-scale application I/O performance significantly, resulting in both a reduction in application run time as well as a higher resolution of simulation run.

Wang, Feiyi↗
DOE OSTI2026

Advancing specialized biofoundries via automated adaptive laboratory evolution

Adaptive laboratory evolution (ALE) is a powerful strategy for improving microbial phenotypes by harnessing natural selection under defined environmental conditions. Through applying selection regimes, beneficial mutations accumulate, enabling the generation of strains with enhanced properties. However, conventional ALE is labor-intensive and difficult to scale, limiting reproducibility and broader discovery of evolutionary principles. Recent advances in robotics, automation, and computational infrastructure are transforming ALE into a scalable, data-rich experimental paradigm. Automated platforms enable standardized and complex protocols, real-time monitoring, and highly parallel evolution campaigns, improving consistency while generating longitudinal datasets that reveal convergent adaptive mechanisms. Here, we discuss the role of specialized biofoundries in advancing automated ALE and enabling large-scale evolutionary engineering. We review major automated ALE formats and outline key design principles for effective ALE biofoundries, highlighting how automated ALE can support autonomous experimentation and AI-guided strain engineering.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Conditional diffusion machine-learning framework for mapping valence electron distribution from convergent beam electron diffraction

Quantitative convergent beam electron diffraction (CBED) enables determination of aspherical valence electron distributions through refinement of low-order structure factors, which are highly sensitive to chemical bonding and charge density variations. However, conventional quantitative CBED (QCBED) requires solving a highly nonlinear inverse problem with many coupled parameters, and computationally intensive dynamical diffraction calculations, making it time-consuming and difficult to apply to complex systems. More broadly, reconstructing charge density and orbital electron distribution from diffraction data has long been a central challenge in both x-ray and electron crystallography. Here, in this study, we introduce an artificial-intelligence (AI)-based framework that replaces traditional refinement with a data-driven inverse solver. Using a large synthetic CBED dataset generated by Bloch-wave simulations, we train a conditional diffusion model to directly infer crystal structural parameters and multipole density formalism parameters, and hence valence electron distributions, from CBED patterns alone. By learning from forward simulations across realistic parameter space, the model effectively solves the inverse problem. Compared with direct regression approaches, the diffusion-based framework provides posterior parameter distributions for rigorous uncertainty quantification while preserving quantitative fidelity and reducing analysis time by orders of magnitude. By eliminating the need for external single-crystal x-ray diffraction data and complex nonlinear refinement, this approach enables practical, high-throughput, and in situ quantitative CBED, enabling real-time mapping of valence electron distributions and their correlation with functional responses in quantum and energy materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2013

An Efficient Deterministic Parallel Algorithm for Adaptive Multidimensional Numerical Integration on GPUs

Recent development in Graphics Processing Units (GPUs) has enabled a new possibility for highly efficient parallel computing in science and engineering. Their massively parallel architecture makes GPUs very effective for algorithms where processing of large blocks of data can be executed in parallel. Multidimensional integration has important applications in areas like computational physics, plasma physics, computational fluid dynamics, quantum chemistry, molecular dynamics and signal processing. The computationally intensive nature of multidimensional integration requires a high-performance implementation. In this study, we present an efficient deterministic parallel algorithm for adaptive multidimensional numerical integration on GPUs. Various optimization techniques are applied to maximize the utilization of the GPU. GPU-based implementation outperforms the best known sequential methods and achieves a speed-up of up to 100. It also shows good scalability with the increase in dimensionality.

Arumugam, Kamesh↗
DOE OSTI2018

CAT: computer aided triage improving upon the Bayes risk through ε-refusal triage rules

Manual extraction of information from electronic pathology (epath) reports to populate the Surveillance, Epidemiology, and End Result (SEER) database is labor intensive. Systematizing the data extraction automatically using machine-learning (ML) and natural language processing (NLP) is desirable to reduce the human labor required to populate the SEER database and to improve the timeliness of the data. This enables scaling up registry efficiency and collection of new data elements. To ensure the integrity, quality, and continuity of the SEER data, the misclassification error of ML and NPL algorithms needs to be negligible. Current algorithms fail to achieve the precision of human experts who can bring additional information in their assessments. Differences in registry format and the desire to develop a common information extraction platform further complicate the ML/NLP tasks. The purpose of our study is to develop triage rules to partially automate registry workflow to improve the precision of the auto-extracted information. This paper presents a mathematical framework to improve the precision of a classifier beyond that of the Bayes classifier by selectively classifying item that are most likely to be correct. This results in a triage rule that only classifies a subset of the item. We characterize the optimal triage rule and demonstrate its usefulness in the problem of classifying cancer site from electronic pathology reports to achieve a desired precision. From the mathematical formalism, we propose a heuristic estimate for triage rule based on post-processing the soft-max output from standard machine learning algorithms. We show, in test cases, that the triage rule significantly improve the classification accuracy.

60 APPLIED LIFE SCIENCES↗