Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Intensive Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Phase-change heterostructure enables ultralow noise and drift for memory operation

Artificial intelligence and other data-intensive applications have escalated the demand for data storage and processing. New computing devices, such as phase-change random access memory (PCRAM)–based neuro-inspired devices, are promising options for breaking the von Neumann barrier by unifying storage with computing in memory cells. However, current PCRAM devices have considerable noise and drift in electrical resistance that erodes the precision and consistency of these devices. We designed a phase-change heterostructure (PCH) that consists of alternately stacked phase-change and confinement nanolayers to suppress the noise and drift, allowing reliable iterative RESET and cumulative SET operations for high-performance neuro-inspired computing. Our PCH architecture is amenable to industrial production as an intrinsic materials solution, without complex manufacturing procedure or much increased fabrication cost.

Science & Technology - Other Topics↗
DOE OSTI2019

Impacts of Spatial Heterogeneity and Temporal Non-Stationarity on Intensity-Duration-Frequency Estimates—A Case Study in a Mountainous California-Nevada Watershed

Changes in extreme precipitation events may require revisions of civil engineering standards to prevent water infrastructures from performing below the designated guidelines. Climate change may invalidate the intensity-duration-frequency (IDF) computation that is based on the assumption of data stationarity. Efforts in evaluating non-stationarity in the annual maxima series are inadequate, mostly due to the lack of long data records and convenient methods for detecting trends in the higher moments. In this study, using downscaled high resolution climate simulations of the historical and future periods under different carbon emission scenarios, we tested two solutions to obtain reliable IDFs under non-stationarity: (1) identify quasi-stationary time windows from the time series of interest to compute the IDF curves using data for the corresponding time windows; (2) introduce a parameter representing the trend in the means of the extreme value distributions. Focusing on a mountainous site, the Walker Watershed, the spatial heterogeneity and variability of IDFs or extremes are evaluated, particularly in terms of the terrain and elevation impacts. We compared observations-based IDFs that use the stationarity assumption with the two approaches that consider non-stationarity. The IDFs directly estimated based on the traditional stationarity assumption may underestimate the 100-year 24-h events by 10% to 60% towards the end of the century at most grids, resulting in significant under-designing of the engineering infrastructure at the study site. Strong spatial heterogeneity and variability in the IDF estimates suggest a preference for using high resolution simulation data for the reliable estimation of exceedance probability over data from sparsely distributed weather stations. Discrepancies among the three IDFs analyses due to non-stationarity are comparable to the spatial variability of the IDFs, underscoring a need to use an ensemble of non-stationary approaches to achieve unbiased and comprehensive IDF estimates.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

BigPanDA: PanDA Workload Management System and its Applications beyond ATLAS

Modern experiments collect peta-scale volumes of data and utilize vast, geographically distributed computing infrastructure that serves thousands of scientists around the world. Requirements for rapid, near real-time data processing, fast analysis cycles and need to run massive detector simulations to support data analysis pose special premium on efficient use of available computational resources. A sophisticated Workload Management System (WMS) is needed to coordinate the distribution and processing of data and jobs in such environment. The ATLAS experiment at CERN uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. While PanDAcurrently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, it runs around 2 million jobs per day on hundreds of Grid sites and serving thousands of ATLAS users. In 2017 about 1.5 exabytes of data were processed with PanDA.In 2012 BigPanDA project project was started with aim to introduce new types of computing resources into ATLAS computing infrastructure, but also to offering PanDA features to different data-intensive applications for projects and experiments outside of ATLAS and High-Energy and Nuclear Physics. In this article we will present accomplishments and discuss possible directions for future work.

Svirin, Pavlo↗
DOE OSTI2017

Skills and Knowledge for Data-Intensive Environmental Research

The scale and magnitude of complex and pressing environmental issues lend urgency to the need for integrative and reproducible analysis and synthesis, facilitated by data-intensive research approaches. However, the recent pace of technological change has been such that appropriate skills to accomplish data-intensive research are lacking among environmental scientists, who more than ever need greater access to training and mentorship in computational skills. Here, we provide a roadmap for raising data competencies of current and next-generation environmental researchers by describing the concepts and skills needed for effectively engaging with the heterogeneous, distributed, and rapidly growing volumes of available data. We articulate five key skills: (1) data management and processing, (2) analysis, (3) software skills for science, (4) visualization, and (5) communication methods for collaboration and dissemination. We provide an overview of the current suite of training initiatives available to environmental scientists and models for closing the skill-transfer gap.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2012

CMS resource utilization and limitations on the grid after the first two years of LHC collisions

After years of development, the CMS distributed computing system is now in full operation. The LHC continues to set records for operational performance, and CMS records data at more than 300 Hz. Because of the intensity of the beams, there are multiple proton-proton interactions per beam crossing, leading to ever-larger event sizes and processing times. The CMS computing system has responded admirably to these challenges, but some reoptimization of the computing model has been required to maximize the efficient delivery of data analysis results by the collaboration in the face of increasingly constrained computing resources. We present the current status of the system, describe the recent performance, and discuss the challenges ahead and how CMS intends to meet them.

Bagliesi, Giuseppe↗
DOE OSTI2022

Extending and Smoothing Two-dimension Equation of State Simulation Data

The Multiphase Equation of State (MEOS) project works to produce high quality equation of state tables which are used in computationally intensive simulations of materials in different conditions. An equation of state table describes the state of matter under certain physical conditions such as pressure, volume, temperature, or internal energy. MEOS uses many different models and combinations thereof to produce accurate tables consisting of continuous smooth data, derivatives, and higher-order derivatives. Accurate and smooth data are important factors in producing precise simulations. This report focuses on improving the electron tables produced from Purgatorio and Thomas-Fermi data. Purgatorio data is extremely accurate but often jagged and discontinuous in the lower temperature and density region. Thomas-Fermi is a model that provides smooth data throughout. Together, this produces an EOS data table that is both accurate and smooth. This report describes the implementation of a new feature that allows users to define more detailed regions in the Purgatorio table to be replaced with Thomas-Fermi. As a result, MEOS can generate a table that is both more smooth and still accurate.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Heterogeneous Machine Learning on High Performance Computing for End to End Driving of Autonomous Vehicles

Current artificial intelligence techniques for end to end driving of autonomous vehicles typically rely on a single form of learning or training processes along with a corresponding dataset or simulation environment. Relatively speaking, success has been shown for a variety of learning modalities in which it can be shown that the machine can successfully “drive” a vehicle. However, the realm of real-world driving extends significantly beyond the realm of limited test environments for machine training. This creates an enormous gap in capability between these two realms. With their superior neural network structures and learning capabilities, humans can be easily trained within a short period of time to proceed from limited test environments to real world driving. For machines though, this gap is guarded by at least two challenges: 1) machine learning techniques remain brittle and unable to generalize to a wide range of scenarios, and 2) effective training data that enhances generalization and generates the desired driving behavior. Further, each challenge can be computationally intensive on its own thereby exasperating the gap. Moreover, is has not yet been shown that a single form of learning or training is capable of addressing a large range of scenarios. As a result, solving the first challenge does not inherently solve the second and vice versa. The work described here discusses an approach to address the first challenge that would also provide a foundation for solving the second. Our approach utilizes a combination of conditional imitation learning with a static dataset, reinforcement learning with a simulation environment, and high-performance computing to train a neural network. As a result, this reduces the “time to solution” from to the existing techniques for autonomous driving and provides an extensible framework to address the second key challenge.

autonomous vehicles↗
DOE OSTI2019

CMS Computing Resources: Meeting the demands of the high-luminosity LHC physics program

The high-luminosity program has seen numerous extrapolations of its needed computing resources that each indicate the need for substantial changes if the desired HL-LHC physics program is to be supported within the current level of computing resource budgets. Drivers include large increases in event complexity (leading to increased processing time and analysis data size) and trigger rates needed (5-10 fold increases) for the HL-LHC program. The CMS experiment has recently undertaken an effort to merge the ideas behind short-term and long-term resource models in order to make easier and more reliable extrapolations to future needs. Near term computing resource estimation requirements depend on numerous parameters: LHC uptime and beam intensities; detector and online trigger performance; software performance; analysis data requirements; data access, management, and retention policies; site characteristics; and network performance. Longer term modeling is affected by the same characteristics, but with much larger uncertainties that must be considered to understand the most interesting handles for increasing the "physics per computing dollar" of the HL-LHC. In this presentation, we discuss the current status of long term modeling of the CMS computing resource needs for HL-LHC with emphasis on techniques for extrapolations, uncertainty quantification, and model results. We illustrate potential ways that high-luminosity CMS could accomplish its desired physics program within today's computing budgets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

AUTONOMOUS GAUSSIAN DECOMPOSITION

We present a new algorithm, named Autonomous Gaussian Decomposition (AGD), for automatically decomposing spectra into Gaussian components. AGD uses derivative spectroscopy and machine learning to provide optimized guesses for the number of Gaussian components in the data, and also their locations, widths, and amplitudes. We test AGD and find that it produces results comparable to human-derived solutions on 21 cm absorption spectra from the 21 cm SPectral line Observations of Neutral Gas with the EVLA (21-SPONGE) survey. We use AGD with Monte Carlo methods to derive the H i line completeness as a function of peak optical depth and velocity width for the 21-SPONGE data, and also show that the results of AGD are stable against varying observational noise intensity. The autonomy and computational efficiency of the method over traditional manual Gaussian fits allow for truly unbiased comparisons between observations and simulations, and for the ability to scale up and interpret the very large data volumes from the upcoming Square Kilometer Array and pathfinder telescopes.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

A Deep Learning Modeling Framework to Capture Mixing Patterns in Reactive-Transport Systems

Prediction and control of chemical mixing are vital for many scientific areas such as subsurface reactive transport, climate modeling, combustion, epidemiology, and pharmacology. Due to the complex nature of mixing in heterogeneous and anisotropic media, the mathematical models related to this phenomenon are not analytically tractable. Numerical simulations often provide a viable route to predict chemical mixing accurately. However, contemporary modeling approaches for mixing cannot utilize available spatial-temporal data to improve the accuracy of the future prediction and can be compute-intensive, especially when the spatial domain is large and for long-term temporal predictions. To address this knowledge gap, in this work we will present in this paper a deep learning (DL) modeling framework applied to predict the progress of chemical mixing under fast bimolecular reactions. This framework uses convolutional neural networks (CNN) for capturing spatial patterns and long short-term memory (LSTM) networks for forecasting temporal variations in mixing. By careful design of the framework—placement of non-negative constraint on the weights of the CNN and the selection of activation function, the framework ensures non-negativity of the chemical species at all spatial points and for all times. Our DL-based framework is fast, accurate, and requires minimal data for training. The time needed to obtain a forecast using the model is a fraction (≈ O(-6)) of the time needed to obtain the result using a high-fidelity simulation. To achieve an error of 10% (measured using the infinity norm) for capturing local-scale mixing features such as interfacial mixing, only 24% to 32% of the sequence data for model training is required. To achieve the same level of accuracy for capturing global-scale mixing features, the sequence data required for model training is 64% to 70% of the total spatial-temporal data. Hence, the proposed approach—a fast and accurate way to forecast long-time spatial-temporal mixing patterns in heterogeneous and anisotropic media—will be a valuable tool for modeling reactive-transport in a wide range of applications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2015

Next Generation Workload Management System For Big Data on Heterogeneous Distributed Computing

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS and ALICE are the largest collaborations ever assembled in the sciences and are at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, both experiments rely on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System (WMS) for managing the workflow for all data processing on hundreds of data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. The scale is demonstrated by the following numbers: PanDA manages O(10 2 ) sites, O(10 5 ) cores, O(10 8 ) jobs per year, O(10 3 ) users, and ATLAS data volume is O(10 17 ) bytes. In 2013 we started an ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF). The project titled 'Next Generation Workload Management and Analysis System for Big Data' (BigPanDA) is funded by DOE ASCR and HEP. Extending PanDA to clouds and LCF presents new challenges in managing heterogeneity and supporting workflow. The BigPanDA project is underway to setup and tailor PanDA at the Oak Ridge Leadership Computing Facility (OLCF) and at the National Research Center "Kurchatov Institute" together with ALICE distributed computing and ORNL computing professionals. Our approach to integration of HPC platforms at the OLCF and elsewhere is to reuse, as much as possible, existing components of the PanDA system. Finally, we will present our current accomplishments with running the PanDA WMS at OLCF and other supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility (OLCF), MIRA supercomputer at Argonne Leadership Computing Facilities (ALCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava and others). Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full production for the ATLAS experiment since September 2015. We will present our current accomplishments with running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

Data-intensive science↗
DOE OSTI2014

Communication: Helium nanodroplet isolation and rovibrational spectroscopy of hydroxymethylene

Hydroxymethylene (HCOH) and its d{sub 1}-isotopologue (HCOD) are isolated in low temperature helium nanodroplets following pyrolysis of glyoxylic acid. Transitions identified in the infrared spectrum are assigned exclusively to the trans-conformation based on previously reported anharmonic frequency computations [P. R. Schreiner, H. P. Reisenauer, F. C. Pickard, A. C. Simmonett, W. D. Allen, E. Mátyus, and A. G. Császár, Nature 453, 906 (2008); L. Koziol, Y. M. Wang, B. J. Braams, J. M. Bowman, and A. I. Krylov, J. Chem. Phys. 128, 204310 (2008)]. For the OH(D) and CH stretches, a- and b-type transitions are observed, and when taken in conjunction with CCSD(T)/cc-pVTZ computations, lower limits to the vibrational band origins are determined. The relative intensities of the a- and b-type transitions provide the orientation of the transition dipole moment in the inertial frame. The He nanodroplet data are in excellent agreement with anharmonic frequency computations reported here and elsewhere, confirming an appreciable Ar-matrix shift of the OH and OD stretches and strong anharmonic resonance interactions in the high-frequency stretch regions of the mid-infrared.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2010

Designing Energy Efficient Communication Runtime Systems for Data Centric Programming Models

The insatiable demand of high performance computing is being driven by the most computationally intensive applications such as computational chemistry, climate modeling, nuclear physics, etc. The last couple of decades have observed a tremendous rise in supercomputers with architectures ranging from traditional clusters to system-on-a-chip in order to achieve the petaflop computing barrier. However, with advent of petaflop-plus computing, we have ushered in an era where power efficient system software stack is imperative for execution on exascale systems and beyond. At the same time, computationally intensive applications are exploring programming models beyond traditional message passing, as a combination of Partitioned Global Address Space (PGAS) languages and libraries, providing one-sided communication paradigm with put, get and accumulate primitives. To support the PGAS models, it is critical to design power efficient and high performance one-sided communication runtime system. In this paper, we design and implement PASCoL, a high performance power aware one-sided communication library using Aggregate Remote Memory Copy Interface (ARMCI), the communication runtime system of Global Arrays. For various communication primitives provided by ARMCI, we study the impact of Dynamic Voltage/Frequency Scaling (DVFS) and a combination of interrupt (blocking)/polling based mechanisms provided by most modern interconnects. We implement our design and evaluate it with synthetic benchmarks using an InfiniBand cluster. Our results indicate that PASCoL can achieve significant reduction in energy consumed per byte transfer without additional penalty for various one-sided communication primitives and various message sizes and data transfer patterns.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2011

Planning With Intensity-Modulated Radiotherapy and Tomotherapy to Modulate Dose Across Breast to Reflect Recurrence Risk (IMPORT High Trial)

Purpose: To establish planning solutions for a concomitant three-level radiation dose distribution to the breast using linear accelerator- or tomotherapy-based intensity-modulated radiotherapy (IMRT), for the U.K. Intensity Modulated and Partial Organ (IMPORT) High trial. Methods and Materials: Computed tomography data sets for 9 patients undergoing breast conservation surgery with implanted tumor bed gold markers were used to prepare three-level dose distributions encompassing the whole breast (36 Gy), partial breast (40 Gy), and tumor bed boost (48 or 53 Gy) treated concomitantly in 15 fractions within 3 weeks. Forward and inverse planned IMRT and tomotherapy were investigated as solutions. A standard electron field was compared with a photon field arrangement encompassing the tumor bed boost volume. The out-of-field doses were measured for all methods. Results: Dose-volume constraints of volume >90% receiving 32.4 Gy and volume >95% receiving 50.4 Gy for the whole breast and tumor bed were achieved. The constraint of volume >90% receiving 36 Gy for the partial breast was fulfilled in the inverse IMRT and tomotherapy plans and in 7 of 9 cases of a forward planned IMRT distribution. An electron boost to the tumor bed was inadequate in 8 of 9 cases. The IMRT methods delivered a greater whole body dose than the standard breast tangents. A contralateral lung volume >2.5 Gy was increased in the inverse IMRT and tomotherapy plans, although it did not exceed the constraint. Conclusion: We have demonstrated a set of widely applicable solutions that fulfilled the stringent clinical trial requirements for the delivery of a concomitant three-level dose distribution to the breast.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2021

QUBO formulations for training machine learning models

Abstract Training machine learning models on classical computers is usually a time and compute intensive process. With Moore’s law nearing its inevitable end and an ever-increasing demand for large-scale data analysis using machine learning, we must leverage non-conventional computing paradigms like quantum computing to train machine learning models efficiently. Adiabatic quantum computers can approximately solve NP-hard problems, such as the quadratic unconstrained binary optimization (QUBO), faster than classical computers. Since many machine learning problems are also NP-hard, we believe adiabatic quantum computers might be instrumental in training machine learning models efficiently in the post Moore’s law era. In order to solve problems on adiabatic quantum computers, they must be formulated as QUBO problems, which is very challenging. In this paper, we formulate the training problems of three machine learning models—linear regression, support vector machine (SVM) and balanced k-means clustering—as QUBO problems, making them conducive to be trained on adiabatic quantum computers. We also analyze the computational complexities of our formulations and compare them to corresponding state-of-the-art classical approaches. We show that the time and space complexities of our formulations are better (in case of SVM and balanced k-means clustering) or equivalent (in case of linear regression) to their classical counterparts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Attractor comparisons based on density

Recognizing a chaotic attractor can be seen as a problem in pattern recognition. Some feature vector must be extracted from the attractor and used to compare to other attractors. The field of machine learning has many methods for extracting feature vectors, including clustering methods, decision trees, support vector machines, and many others. In this work, feature vectors are created by representing the attractor as a density in phase space and creating polynomials based on this density. Density is useful in itself because it is a one dimensional function of phase space position, but representing an attractor as a density is also a way to reduce the size of a large data set before analyzing it with graph theory methods, which can be computationally intensive. The density computation in this paper is also fast to execute. In this paper, as a demonstration of the usefulness of density, the density is used directly to construct phase space polynomials for comparing attractors. Comparisons between attractors could be useful for tracking changes in an experiment when the underlying equations are too complicated for vector field modeling.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗