SEARCH · Engineering Papers

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

DOE OSTI2014

The Science DMZ: A Network Design Pattern for Data-Intensive Science

The ever-increasing scale of scientific data has become a significant challenge for researchers that rely on networks to interact with remote computing systems and transfer results to collaborators worldwide. Despite the availability of high-capacity connections, scientists struggle with inadequate cyberinfrastructure that cripples data transfer performance, and impedes scientific progress. The Science DMZ paradigm comprises a proven set of network design patterns that collectively address these problems for scientists. We explain the Science DMZ model, including network architecture, system configuration, cybersecurity, and performance tools, that creates an optimized network environment for science. We describe use cases from universities, supercomputing centers and research laboratories, highlighting the effectiveness of the Science DMZ model in diverse operational settings. In all, the Science DMZ model is a solid platform that supports any science workflow, and flexibly accommodates emerging network technologies. As a result, the Science DMZ vastly improves collaboration, accelerating scientific discovery.

Dart, Eli↗

DOE OSTI2013

The Science DMZ: A Network Design Pattern for Data-Intensive Science

42 ENGINEERING↗

DOE OSTI2013

What is Data-Intensive Science?

What is Data Intensive Science? Today we are living in a digital world, where scientists often no longer interact directly with the physical object of their research, but do so via digitally captured, reduced, calibrated, analyzed, synthesized and, at times, visualized data. Advances in experimental and computational technologies have lead to an exponential growth in the volumes, variety and complexity of this data and while the deluge is not happening everywhere in an absolute sense, it is in a relative one. Science today is data intensive. Data intensive science has the potential to transform not only how we do science, but how quickly we can translate scientific progress into complete solutions, policies, decisions and ultimately economic success. Critically, data intensive science touches some of the most important challenges we are facing. Consider a few of the grand challenges outlined by the U.S. National Academy of Engineering: make solar energy economical, provide energy from fusion, develop carbon sequestration methods, advance health informatics, engineer better medicines, secure cyberspace, and engineer the tools of scientific discovery. Arguably, meeting any of these challenges requires the collaborative effort of trans-disciplinary teams, but also significant contributions from enabling data intensive technologies. Indeed for many of them, advances in data intensive research will be the single most important factor in developing successful and timely solutions. Simple extrapolations of how we currently interact with and utilize data and knowledge are not sufficient to meet this need. Given the importance of these challenges, a new, bold vision for the role of data in science, and indeed how research will be conducted in a data intensive environment is evolving.

Data Intensive Science↗

DOE OSTI2020

Data Driven Online Learning of Power System Dynamics

With the advancement of sensing and communication in power networks, high-frequency real-time data from a power network can be used as a resource to develop better monitoring capabilities. In this work, a systematic approach based on data-driven operator theoretic methods involving Koopman operator is proposed for the online identification of power system dynamics. In particular, a new algorithm is provided, which unlike any previously existing algorithms, updates the Koopman operator iteratively as new data points are acquired. The proposed algorithm has three advantages: a) allows for real-time monitoring of the power system dynamics b) linear power system dynamics (this linear system is usually in a higher dimensional feature space and is not same as linearization of the underlying nonlinear dynamics) and c) computationally fast and less intensive when compared to the popular Extended Dynamic Mode Decomposition (EDMD) algorithm. The efficiency of the proposed algorithm is illustrated on an IEEE 9 bus system using synthetic data from the nonlinear model and on IEEE 39 bus system using synthetic data from the linearized model.

Sinha, Subhrajit↗

DOE OSTI2026

Uncertain quantum computing futures and potential energy and physical resource impacts at scale

Considerable attention has recently focused on the vast energy and water demands of supercomputing, namely large-scale data centers that underpin artificial intelligence (AI), one of the great disruptors of contemporary society. Looking ahead some years from now, quantum computing is poised to disrupt established computing paradigms once again. Scientists and engineers are now working intensely to bring this century-old dream of physicists to fruition. Yet, as quantum computers begin to be integrated with classical supercomputing architectures, the implications for energy and physical resource use also need to be understood, especially how they compare to today’s AI data centers. These impacts have not yet been quantified by the research community – a notable gap in the literature, even if commercial-scale deployment of Quantum-Accelerated Computing Infrastructure (QuACI) is not expected for a few more years. This study is the first to conduct such an assessment. Using publicly available information from academic sources and private industry, we characterize multiple configurations of superconducting qubit-based, fault-tolerant quantum computers (FTQC) that could plausibly be deployed at scale in the 2030s and into the 2040s. By parameterizing these FTQC systems at a process level, we conduct a prospective scenario analysis to quantify their energy and physical resource needs. While these estimates are uncertain, given the current state of quantum technologies and their unknown future trajectories, important insights can already be drawn. One key finding is that while the electricity needs for a fleet of FTQCs are within the bounds of previous modeling studies that have explored high electricity demand futures, the needs for certain physical resources, namely water and helium-3, could pose bottlenecks to QuACI scale-up.

Computing↗

DOE OSTI2014

Big-Data RHEED analysis for understanding epitaxial film growth processes

Reflection high energy electron diffraction (RHEED) has by now become a standard tool for in-situ monitoring of film growth by pulsed laser deposition and molecular beam epitaxy. Yet despite the widespread adoption and wealth of information in RHEED image, most applications are limited to observing intensity oscillations of the specular spot, and much additional information on growth is discarded. With ease of data acquisition and increased computation speeds, statistical methods to rapidly mine the dataset are now feasible. Here, we develop such an approach to the analysis of the fundamental growth processes through multivariate statistical analysis of RHEED image sequence. This approach is illustrated for growth of LaxCa1-xMnO3 films grown on etched (001) SrTiO3 substrates, but is universal. The multivariate methods including principal component analysis and k-means clustering provide insight into the relevant behaviors, the timing and nature of a disordered to ordered growth change, and highlight statistically significant patterns. Fourier analysis yields the harmonic components of the signal and allows separation of the relevant components and baselines, isolating the assymetric nature of the step density function and the transmission spots from the imperfect layer-by-layer (LBL) growth. These studies show the promise of big data approaches to obtaining more insight into film properties during and after epitaxial film growth. Furthermore, these studies open the pathway to use forward prediction methods to potentially allow significantly more control over growth process and hence final film quality.

Big-Data↗

DOE OSTI2026

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

The DOE High Energy Physics (HEP) program in Neutrino and Collider science drives data-intensive science and simulation on extreme-scale platforms. Modeling and optimizing the complex distributed components from experimental to leadership computing facilities are essential for HEP workflows to achieve required response times and resilience under various conditions. Tachyon proposes a framework for scalable modeling, simulation, and validation of key performance characteristics for the distributed infrastructure between FNAL and ALCF, along with associated HEP workflows.

Carothers, Chris [Rensselaer Poly.]↗

DOE OSTI2026

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

Carothers, Chris [Rensselaer Poly.]↗

DOE OSTI2025

Quantum Time Dynamics Mediated by the Yang–Baxter Equation and Artificial Neural Networks

Quantum computing shows great potential, but errors pose a significant challenge. This study explores new strategies for mitigating quantum errors using artificial neural networks (ANNs) and the Yang–Baxter equation (YBE). Unlike traditional error mitigation methods, which are computationally intensive, we investigate artificial error mitigation. We developed a novel method that combines ANNs for noise mitigation combined with the YBE to generate noisy data. This approach effectively reduces noise in quantum simulations, enhancing the accuracy of the results. The YBE rigorously preserves quantum correlations and symmetries in spin chain simulations in certain classes of integrable lattice models, enabling effective compression of quantum circuits while retaining linear scalability with the number of qubits. This compression facilitates both full and partial implementations, allowing the generation of noisy quantum data on hardware alongside noiseless simulations using classical platforms. By introducing controlled noise through the YBE, we enhance the data set for error mitigation. We train an ANN model on partial data from quantum simulations, demonstrating its effectiveness in mitigating errors in time-evolving quantum states, providing a scalable framework to enhance quantum computation fidelity, particularly in noisy intermediate-scale quantum (NISQ) systems. We demonstrate the efficacy of this approach by performing quantum time dynamics simulations using the Heisenberg XY Hamiltonian on real quantum devices.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2016

SURROGATE MODEL DEVELOPMENT AND VALIDATION FOR RELIABILITY ANALYSIS OF REACTOR PRESSURE VESSELS

In nuclear light water reactors (LWRs), the reactor coolant, core and shroud are contained within a massive, thick walled steel vessel known as a reactor pressure vessel (RPV). Given the tremendous size of these structures, RPVs typically contain a large population of pre-existing flaws introduced in the manufacturing process. After many years of operation, irradiation-induced embrittlement makes these vessels increasingly susceptible to fracture initiation at the locations of the pre-existing flaws. Because of the uncertainty in the loading conditions, flaw characteristics and material properties, probabilistic methods are widely accepted and used in assessing RPV integrity. The Fracture Analysis of Vessels – Oak Ridge (FAVOR) computer program developed by researchers at Oak Ridge National Laboratory is widely used for this purpose. This program can be used in order to perform deterministic and probabilistic risk-informed analyses of the structural integrity of an RPV subjected to a range of thermal-hydraulic events. FAVOR uses a one-dimensional representation of the global response of the RPV, which is appropriate for the beltline region, which experiences the most embrittlement, and employs an influence coefficient technique to rapidly compute stress intensity factors for axis-aligned surface-breaking flaws. The Grizzly code is currently under development at Idaho National Laboratory (INL) to be used as a general multiphysics simulation tool to study a variety of degradation mechanisms in nuclear power plant components. The first application of Grizzly has been to study fracture in embrittled RPVs. Grizzly can be used to model the thermo-mechanical response of an RPV under transient conditions observed in a pressurized thermal shock (PTS) scenario. The global response of the vessel provides boundary conditions for local 3D models of the material in the vicinity of a flaw. Fracture domain integrals are computed to obtain stress intensity factors, which can in turn be used to assess whether a fracture would initiate at a pre-existing flaw. To use Grizzly for probabilistic analysis, it is necessary to have a way to rapidly evaluate stress intensity factors. To accomplish this goal, a reduced order model (ROM) has been developed to efficiently represent the behavior of a detailed 3D Grizzly model used to calculate fracture parameters. This approach uses the stress intensity factor influence coefficient method that has been used with great success in FAVOR. Instead of interpolating between tabulated solutions, as FAVOR does, the ROM approach uses a response surface methodology to compute fracture solutions based on a sampled set of results used to train the ROM. The main advantages of this approach are that the process of generating the training data can be fully automated, and the procedure can be readily used to consider more general flaw configurations. This paper demonstrates the procedure used to generate a ROM to rapidly compute stress intensity factors for axis-aligned flaws. The results from this procedure are in good agreement with those produced using the traditional influence coefficient interpolation procedure, which gives confidence in this method. This paves the way for applying this procedure for more general flaw configurations.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2016

The Improved Simple Corner Balance Method and Efforts to Enhance its Computational Performance

The diamond difference (DD) spatial discretization has long been a work horse of the neutron transport community. For nearly as long as DD has been used in neutron transport, it has been observed that DD produces undamped, non-physical angular fluxes in spatially under resolved problems and can fail in the thick diffusion limit. A long list of alternative spatial discretizations, all with better physics properties: accuracy, robustness, and/or preservation of the thick diffusion limit, have been proposed as alternatives to DD. Unfortunately, these alternative spatial discretizations also require significantly more floating point operations (FLOPs) per unknown compared to DD. Historically, neutron transport problems of interest have been able to be resolved sufficiently that DD is accurate enough and FLOPs have been the computational performance bottleneck controlling time to solution. Thus, in practice DD has been and remains a preferred spatial discretization for neutron transport simulations. Trends in computing architecture are leading to a new paradigm where data movement, not FLOPs will control time to solution. This presents an opportunity to improve neutron transport physics by reconsidering more FLOP intensive spatial discretization than DD without incurring a time to solution penalty. In this work, we consider the improved simple corner balance (ISCB) spatial discretization, a unique alternative to DD that has the same memory footprint as DD, and detail efforts to improve its computational performance. The ISCB spatial discretization is a more FLOP intensive spatial discretization than DD with the same memory footprint as DD. Though designed to maximize FLOP intensity, third party libraries such as LAPACK are not tuned for the unique, frequent, small linear algebra needs of neutron transport problems. While hand coding a sparse Gauss elimination is likely to be faster than a dense matrix solve (CM vs. HGE), were LAPACK suitable for small linear algebra problems, we would not have seen a performance increase with decreased numbers of LAPACK calls for the Schur complement solve (SC4 vs. SC1). Significant effort has reduced the relative compute time of ISCB to DD, but on current computer architectures DD is still faster than ISCB. Future work comparing the overall efficiency of ISCB to DD may reveal that while ISCB is slower than DD on current computers, ISCB is more accurate than DD by a factor greater than the compute time slowdown. As new computer architectures become available, we will test our hypothesis that neutron transport simulations will be bandwidth limited, ideally rendering the additional FLOPs required by ISCB to be 'free' [in terms of time to solution]. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

MICCO: An Enhanced Multi-GPU Scheduling Framework for Many-Body Correlation Functions

Calculation of many-body correlation functions is one of the critical kernels utilized in many scientific computing areas, especially in Lattice Quantum Chromodynamics (Lattice QCD). It is formalized as a sum of a large number of contraction terms each of which can be represented by a graph consisting of vertices describing quarks inside a hadron node and edges designating quark propagations at specific time intervals. Due to its computation- and memory-intensive nature, real-world physics systems (e.g., multi-meson or multi-baryon systems) explored by Lattice QCD prefer to leverage multi-GPUs. Different from general graph processing, many-body correlation function calculations show two specific features: a large number of computation-/data-intensive kernels and frequently repeated appearances of original and intermediate data. The former results in expensive memory operations such as tensor movements and evictions. The latter offers data reuse opportunities to mitigate the data-intensive nature of many-body correlation function calculations. However, existing graph-based multi-GPU schedulers cannot capture these data-centric features, thus resulting in a sub-optimal performance for many-body correlation function calculations. To address this issue, this paper presents a multi-GPU scheduling framework, MICCO, to accelerate contractions for correlation functions particularly by taking the data dimension (e.g., data reuse and data eviction) into account. This work first performs a comprehensive study on the interplay of data reuse and load balance, and designs two new concepts: local reuse pattern and reuse bound to study the opportunity of achieving the optimal trade-off between them. Based on this study, MICCO proposes a heuristic scheduling algorithm and a machine-learning-based regression model to generate the optimal setting of reuse bounds. Specifically, MICCO is integrated into a real-world Lattice QCD system, Redstar, for the first time running on multiple GPUs. The evaluation demonstrates MICCO outperforms other state-of-art works, achieving up to 2.25× speedup in synthesized datasets, and 1.49× speedup in real-world correlation functions.

Wang, Qihan↗

DOE OSTI2020

Overland flow numerical model prediction, Lower Triangle Region in East River Watershed, Colorado, 3 days

This data package contains numerical simulation results of surface flow variables such as flow velocity and water depth in Lower Triangle Region in East River Watershed, Colorado. The surface flow is a consequence of a high intensity rainfall event with a total duration of 3 days, available at a resolution of 10 minutes. The results are computed on triangular multiresolution meshes with resolutions ranging from 10 meter to 80 meter. The data package also contains a simulation on a uniform triangular mesh with a resolution of 10 meter. The simulations consider surface flow only and neglect subsurface flow, infiltration, and evapotranspiration. The purpose of the data is to assess the quality of a mesh refinement strategy.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2016

New constraints on modelling the random magnetic field of the MW

We extend the description of the isotropic and anisotropic random component of the small-scale magnetic field within the existing magnetic field model of the Milky Way from Jansson and Farrar, by including random realizations of the small-scale component. Using a magnetic-field power spectrum with Gaussian random fields, the NE2001 model for the thermal electrons and the Galactic cosmic-ray electron distribution from the current GALPROP model we derive full-sky maps for the total and polarized synchrotron intensity as well as the Faraday rotation-measure distribution. While previous work assumed that small-scale fluctuations average out along the line-of-sight or which only computed ensemble averages of random fields, we show that these fluctuations need to be carefully taken into account. Comparing with observational data we obtain not only good agreement with 408 MHz total and WMAP7 22 GHz polarized intensity emission maps, but also an improved agreement with Galactic foreground rotation-measure maps and power spectra, whose amplitude and shape strongly depend on the parameters of the random field. We demonstrate that a correlation length of 0≈22 pc (05 pc being a 5σ lower limit) is needed to match the slope of the observed power spectrum of Galactic foreground rotation-measure maps. Using multiple realizations allows us also to infer errors on individual observables. We find that previously-used amplitudes for random and anisotropic random magnetic field components need to be rescaled by factors of ≈0.3 and 0.6 to account for the new small-scale contributions. Our model predicts a rotation measure of −2.8±7.1 rad/m{sup 2} and 04.4±11. rad/m{sup 2} for the north and south Galactic poles respectively, in good agreement with observations. Applying our model to deflections of ultra-high-energy cosmic rays we infer a mean deflection of ≈3.5±1.1 degree for 60 EeV protons arriving from CenA.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗

DOE OSTI2014

Dosimetric evaluation of MapCHECK 2 and 3DVH in the IMRT delivery quality assurance process

Based on per-field data, 3 dose-volume histogram (DVH) is designed to calculate 3-dimensional (3D) dose using patient-computed tomography (CT) for an intensity-modulated radiation therapy (IMRT) delivery quality assurance (DQA) process measured with diode detector arrays. In this study, the characteristics and suitability of 3DVH for an IMRT DQA process were evaluated by assessment of clinically applied results. The DQA plans were prepared with a 2D diode detector array (MapCHECK 2) for 20 IMRT patients (10 with head and neck cancer and 10 with prostate cancer), and the matching rate between the calculated dose and the measured dose was analyzed. The relative dose differences between the original IMRT plan and the recalculated plan by 3DVH were evaluated for tumor target and organs at risk (OARs). Confidence limits (CLs) were determined for quantification of the degree of agreement that should be expected in each DQA method. The individual CLs for DQA accuracy of 3D dosimetric evaluation in the tumor target and OARs were also calculated to evaluate the differences compared with conventional 2D-based DQA results. The matching rates were sufficient to validate the accuracy of IMRT DQA analyses and the calculated CL values were reasonable values considering the normal criteria of our institution. The 3DVH analysis generally had better matching rate than per-field measurement analysis and the CL values for OARs were better than those for the planning target volume (PTV). The usefulness of the 3DVH tool was verified for IMRT DQA from the point of view of dose evaluation in the corresponding patient's CT data. It also enables dosimetric evaluation in each important structure (PTV and OARs), which can promote more effective evaluation of DQA error in terms of tumor treatment and side effects.

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2014

WE-E-17A-03: FDG-PET-Based Radiomics to Predict Local Control and Survival Following Radiotherapy

Purpose: An exploding field in cancer research is “radiomics,” based on the hypothesis that there is statistical (hidden) information in medical images that is prognostic or predictive of outcomes. Our group has developed an efficient pipeline to extract and analyze quantitative image features from medical images as related to outcomes or diagnosis. In this work, we summarize our previous studies with positron emission tomography (PET) images and show the potential of the use of radiomics for outcomes research. Methods: We analyzed two cancer datasets, each consisting of pre-radiotherapy-treatment PET scans: 163 T1-2N0M0 non-small cell lung cancer (NSCLC) patients and 174 head and neck (H and N) cancer patients with stage III–IV. The PET scans were converted to Computational Environment for Radiological Research (CERR) format, and CERR was used to generate 24 shape, texture, and intensity-histogram based image features. Data-mining and logistic regression methods were then used to model local failure (LF) and overall survival (OS). Unbiased estimates of performance were generated using leave-one-out cross-validation (LOOCV). Results: For predicting LF, the models with biologically equivalent dose (BED) and TLG (metabolic tumor volume (MTV) x SUVmean) in NSCLC, and skewness and MTV in H and N, achieved the best performance with AUC=0.818 (p<0.0001) and AUC=0.826 (p=0.0002), respectively. For predicting OS, the models with kurtosis and volume in NSCLC and SUVmax and homogeneity in H and N achieved the best performance with AUC=0.706 (p<0.0001) and AUC=0.656 (p=0.0003), respectively. On LOOCV, all these models retained significant predictive power. Interestingly, MTV was highly correlated with LF in both sites. Conclusion: PET-based imaged features are promising tools for improving treatment management decision making. Much more research is needed to identify optimal radiomics metrics and to correlate imaging phenotype with other clinical or genomic information.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2024

Phase Diagrams of Alloys and Their Hydrides via On-Lattice Graph Neural Networks and Limited Training Data

Efficient prediction of sampling-intensive thermodynamic properties is needed to evaluate material performance and permit high-throughput materials modeling for a diverse array of technology applications. To alleviate the prohibitive computational expense of high-throughput configurational sampling with density functional theory (DFT), surrogate modeling strategies like cluster expansion are many orders of magnitude more efficient but can be difficult to construct in systems with high compositional complexity. We therefore employ minimal-complexity graph neural network models that accurately predict and can even extrapolate to out-of-train distribution formation energies of DFT-relaxed structures from an ideal (unrelaxed) crystallographic representation. This enables the large-scale sampling necessary for various thermodynamic property predictions that may otherwise be intractable and can be achieved with small training data sets. Two exemplars, optimizing the thermodynamic stability of low-density high-entropy alloys and modulating the plateau pressure of hydrogen in metal alloys, demonstrate the power of this approach, which can be extended to a variety of materials discovery and modeling problems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗