Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA INTENSIVE COMPUTING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

A review of water valuation metrics: Supporting sustainable water use in manufacturing

In the manufacturing sector, water has been often considered too cheap to conserve. Such thinking relies on water valuations that limit the value of water to the price paid. Using such simple methods, the share of water cost to total manufacturing cost is significantly small, <3%. As a result, conserving water and enabling technology uptake is difficult to justify economically and slow to advance, hindering progress toward sustainable water use. However, the value of water to a manufacturer is far greater than the price paid. Valuations such as the true cost of water consider the additional in-plant treatment and energy costs and have been gaining greater traction in the manufacturing sector. However, true cost alone still undervalues water by not accounting for economic and social costs related to scarcity and environmental externalities. This paper makes the case and presents a framework for valuing manufacturing water beyond the price paid and the true cost. The proposed fuller valuation of manufacturing water takes into account the internal and opportunity costs associated with the realization of water risks. The paper follows with a review of a wide range of water valuation metrics, both at the specific industry level and regional/economy-wide level. The use of various valuation metrics incorporating the relationship between the change in value with change in water use, such as marginal value of water, shadow price, and elasticity at the specific industry level, has been limited in the U.S. manufacturing sector. Further, a limited number of studies exist on data-intensive subjective evaluation techniques such as computable general equilibrium modeling and input-output modeling for regional water valuation. After reviewing water value metrics, several recent case studies from manufacturers from the literature are presented to illustrate both the promise and challenges of a fuller valuation of water as proposed here. Some large multinational corporations have moved toward assessing the value of water via supply chain sustainability initiatives, environmental profit and loss accounting, estimating risk-adjusted values of water, hydro-economic modeling, natural capital asset valuation, and developing value chain indices. This paper provides policymakers and technology developers a framework for monetizing water value beyond its true cost and current metrics. If adopted, such fuller water valuations can help make the business case for the development and deployment of cost-effective water-conserving technologies, thereby improving the sustainability of the manufacturing sector with respect to water.

54 ENVIRONMENTAL SCIENCES↗

Empirically-calibrated H100 node power models for accurate AI training energy estimation

Accurately quantifying the energy use of artificial intelligence (AI) training is critical for infrastructure planning, carbon accounting, and sustainable data center operation, but few studies have directly measured the power consumption of production workloads on contemporary hardware. By combining empirical measurements from Brookhaven National Laboratory during AI training on 8-graphics-processing-unit H100 systems with open-source benchmarking data, we develop statistical models relating computational intensity to node-level power consumption. We measure the gap between manufacturer-rated thermal design power (TDP) and actual power demand during AI training. Our analysis reveals that even computationally intensive workloads operate at only 76% of the 10.2 kW TDP rating. Our architecture-specific model, calibrated to floating-point operations, predicts energy consumption with 11.4% mean absolute percentage error, significantly outperforming TDP-based approaches (27%–37% error). We identified distinct power signatures between transformer and convolutional neural network architectures, with transformers showing characteristic fluctuations that may impact grid stability. These results provide a measurement-grounded basis for improving AI training energy estimates, enabling more reliable infrastructure sizing, cost projections, and environmental impact assessments.

Newkirk, Alex C↗

TomoPyUI : a user-friendly tool for rapid tomography alignment and reconstruction

The management and processing of synchrotron and neutron computed tomography data can be a complex, labor-intensive and unstructured process. Users devote substantial time to both manually processing their data ( i.e. organizing data/metadata, applying image filters etc. ) and waiting for the computation of iterative alignment and reconstruction algorithms to finish. In this work, we present a solution to these problems: TomoPyUI , a user interface for the well known tomography data processing package TomoPy . This highly visual Python software package guides the user through the tomography processing pipeline from data import, preprocessing, alignment and finally to 3D volume reconstruction. The TomoPyUI systematic intermediate data and metadata storage system improves organization, and the inspection and manipulation tools (built within the application) help to avoid interrupted workflows. Notably, TomoPyUI operates entirely within a Jupyter environment. Herein, we provide a summary of these key features of TomoPyUI , along with an overview of the tomography processing pipeline, a discussion of the landscape of existing tomography processing software and the purpose of TomoPyUI , and a demonstration of its capabilities for real tomography data collected at SSRL beamline 6-2c.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Analytics for Massive Heat Maps

High throughput instrumentation for genomics is producing data orders of magnitude greater than even a decade before. Biologists often visualize the data of these experiments through the use of heat maps. For large datasets, heat map visualizations do not scale. These visualizations are only capable of displaying a portion of the data, making it difficult for scientists to find and detect patterns that span more than a subsection of the data. We present a novel method that provides an interactive visual display for massive heat maps [O(108)]. Our process shows how a massive heat map can be decomposed into multiple levels of abstraction to represent the underlying macrostructures. We aggregate these abstractions into a framework that can allow real-time navigation of the space. To further assist pattern discovery, we ground our system on the principle of focus+context. Our framework also addresses the issue of balancing the memory and display resolution and heat map size. We will show that this technique for biologists provides a powerful new visual metaphor for analyzing massive datasets.

59 BASIC BIOLOGICAL SCIENCES↗

Advanced Scientific Computing Research Network Requirements: ASCR Network Requirements Review Final Report

The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the U.S. Department of Energy (DOE) Office of Science (SC), the single largest supporter of basic research in the physical sciences in the United States. In support of SC programs, ESnet regularly updates and refreshes its understanding of the networking requirements of the instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet to be a highly successful enabler of scientific discovery for over 25 years. In October 2012, ESnet and the Office of Advanced Scientific Computing Research (ASCR) of the DOE SC organized a review to characterize the networking requirements of the programs funded by the ASCR program office. The requirements identified at the review are summarized in the Findings section, and are described in more detail in the body of the report.

97 MATHEMATICS AND COMPUTING↗

Community Input on the Need, Scope, and Development Roadmap of a Proposed National Virtual Biosecurity for Bioenergy Crops Center (NVBBCC)

Brookhaven National Laboratory (BNL) was awarded a pilot project in FY22 under the U.S. Department of Energy (DOE) Office of Science Biopreparedness Research Virtual Environment (BRaVE) initiative, to define research priorities, needs, and requirements for a national virtual center devoted to the biosecurity of bioenergy crops. The mission of the proposed center, referred to as the National Virtual Biosecurity for Bioenergy Crop Center (NVBBCC), would be to provide the scientific basis and tools to detect, characterize, model, and mitigate biothreats to bioenergy crops. This function will be essential to ensure the projected increased US reliance over the next few decades on key plant-based energy products, such as biojet fuel. The NVBBCC is envisioned as a distributed, virtual center with multiple national laboratories at its core to maximize the use of existing unique facilities and expertise across the DOE complex. A major goal of the pilot project was to develop a roadmap for establishing NVBBCC through a series of meetings to gather community input. A total of about 150 individuals, drawn from DOE laboratories, the USDA, academia, NIH, DHS and the private sector participated in six planning meetings held in FY23. Four of the meetings were focused on specific research topics (disease detection, dispersion and disease propagation, biomolecular characterization of plant-pathogen interaction, and mitigation strategies). These four meetings were followed by a meeting that focused on computational needs to support collaborative, data-intensive research within a distributed center as well as workforce development. A final meeting focused on establishing and maintaining preparedness within NVBBCC to respond to an emerging disease within bioenergy crops and how it would collaborate and coordinate with USDA and DHS.

09 BIOMASS FUELS↗

Analysis of ground tests of a microwave, earth-occultation, pressure-reference-level system.

A two-satellite, microwave, earth-occultation system can supplement an infrared sensor by providing an accurate altitude reference that will serve to fix, as a function of height, the derived temperature profile of the infrared sounder. The results of ground tests made in Hawaii to estimate the likely effects of scintillation and fading on an occultation system are described. It was found that the microwave signal suffered periods of intense fading; extensive computer analyses of the data were performed in which aircraft-generated refractivity profiles were subjected to ray tracing. Results of these analyses indicate that the probable cause of the observed fading was multipath, a low-altitude phenomenon usually attributed to water vapor inhomogeneities. It is maintained that multipath will therefore have minimal effect on the pressure-reference microwave occultation system, which would operate at a relatively high closest-approach altitude (about 8 km).

Ungar, S. G.↗

Partnership for Edge Physics Simulation

A major goal of our participation in the Edge Physics Simulation project has been to contribute to the understanding of the self-organization of tokamak turbulence fluctuations resulting in the formation of a staircase structure in the ion temperature. A second important goal is to demonstrate how small scale turbulence in plasmas self-organizes with dynamically driven quasi-stationary flow shear. These goals have been accomplished through the analyses of the statistical properties of XGC1 flux driven Gyrokinetic electrostatic ion temperature gradient (ITG) turbulence simulation data in which neutrals are included. The ITG turbulence data, and in particular fluctuation data, were obtained from a massively parallel flux-driven gyrokinetic full-f particle-in-cell simulation of a DIII-D like equilibrium. Below some the findings are summarized. It was observed that the emergence of staircase structure is related to the variations in the normalized temperature gradient length (R/LT) and the poloidal flow shear. Average turbulence intensity is found to be large in the vicinity of minima in R/LTi, where ITG growth is expected to be lower. The distributions of the occurrences of potential fluctuation are found to be Gaussian away from the staircase-step locations, but they are found to be non-Gaussian in the vicinity of staircase-step locations. The results of analytically derived expressions for the distribution of the occurrences of turbulence intensity and intensity flux were compared with the corresponding quantities computed using XGC1 simulation data and good agreement is found. The derived expressions predicts inward and outward propagation of turbulence intensity flux in an intermittent fashion. The outward propagation of turbulence intensity flux occurs at staircase-step locations and is related to the change in poloidal flow velocity shear and to the change in the ion temperature gradient. The standard deviation, skewness and kurtosis for turbulence quantities were computed and found to be large in the vicinity of the staircase-step structures. Large values of skewness and kurtosis can be explained by a temporary opening and closing of the structure which allows turbulence intensity events to propagate. The staircase patterns may reduce the ion heat transport and a manipulation of these patterns may be used to optimize heat transport in tokamaks. An additional objective of the research in support of the Edge Physics Simulation initiative has been to improve the understanding of scrape-off layer thermal transport. In planning experiments and designing future tokamaks, it is important to understand the physical effects that contribute to divertor heat-load fluxes. The research accomplished will contribute to developing new models for the scrape-off layer region. The XGC0 code was used to compute the heat fluxes and the heat-load width in the outer divertor plates of C-Mod and DIII-D tokamaks. It was observed that the width of the XGC0 neoclassical heat-load was approximately inversely proportional to the total plasma current. Anomalous transport in the H-mode pedestal region of five Alcator C-Mod discharges, representing a collisionality scan, was analyzed. The understanding of anomalous transport in the pedestal region is important for the development of a comprehensive model for the H-mode pedestal slope. It was found that the electron thermal anomalous diffusivities at the pedestal top increase with the electron collisionality. This dependence can point to the DRIBM as the modes that drive the anomalous transport in the plasma edge of highly collisional discharges. The effects of plasma shaping on the H-mode pedestal structure was also investigated. The differences in the predicted H-mode pedestal width and height for the DIII-D discharges with different elongation and triangularities were discussed. For the discharges with higher elongation, it was found that the gradients of the plasma profiles in the H-mode pedestal reach semi-steady states. In these simulations, the pedestal slowly continued to evolve to higher pedestal pressures and bootstrap currents until the peeling ballooning stability conditions were satisfied. The discharges with lower elongation do not reach the semi-steady state, and ELM crashes were triggered at earlier times. The plasma elongation was found to have a stronger stabilizing effect than the plasma triangularity. For the discharges with lower elongation and lower triangularity, the ELM frequency was large, and the H-mode pedestal evolves rapidly. It was found that the temperature of neutrals in the scrape-off-layer region can affect the dynamics of the H-mode pedestal buildup. However, the final pedestal profiles were nearly independent of the neutral temperature. The elongation and triangularity affected the pedestal widths of plasma density and electron temperature profiles differently. This study illustrated a new mechanism for controlling the pedestal bootstrap current and the pedestal stability.

97 MATHEMATICS AND COMPUTING↗

2nd Generation QUATARA Flight Computer Project

Single core flight computer boards have been designed, developed, and tested (DD&T) to be flown in small satellites for the last few years. In this project, a prototype flight computer will be designed as a distributed multi-core system containing four microprocessors running code in parallel. This flight computer will be capable of performing multiple computationally intensive tasks such as processing digital and/or analog data, controlling actuator systems, managing cameras, operating robotic manipulators and transmitting/receiving from/to a ground station. In addition, this flight computer will be designed to be fault tolerant by creating both a robust physical hardware connection and by using a software voting scheme to determine the processor's performance. This voting scheme will leverage on the work done for the Space Launch System (SLS) flight software. The prototype flight computer will be constructed with Commercial Off-The-Shelf (COTS) components which are estimated to survive for two years in a low-Earth orbit.

Falker, Jay↗

Lost in space: Onboard star identification using CCD star tracker data without an a priori attitude

There are many algorithms in use today which determine spacecraft attitude by identifying stars in the field of view of a star tracker. Some methods, which date from the early 1960's, compare the angular separation between observed stars with a small catalog. In the last 10 years, several methods have been developed which speed up the process and reduce the amount of memory needed, a key element to onboard attitude determination. However, each of these methods require some a priori knowledge of the spacecraft attitude. Although the Sun and magnetic field generally provide the necessary coarse attitude information, there are occasions when a spacecraft could get lost when it is not prudent to wait for sunlight. Also, the possibility of efficient attitude determination using only the highly accurate CCD star tracker could lead to fully autonomous spacecraft attitude determination. The need for redundant coarse sensors could thus be eliminated at substantial cost reduction. Some groups have extended their algorithms to implement a computation intense full sky scan. Some require large data bases. Both storage and speed are concerns for autonomous onboard systems. Neural network technology is even being explored by some as a possible solution, but because of the limited number of patterns that can be stored and large overhead, nothing concrete has resulted from these efforts. This paper presents an algorithm which, by descretizing the sky and filtering by visual magnitude of the brightness observed star, speeds up the lost in space star identification process while reducing the amount of necessary onboard computer storage compared to existing techniques.

Ketchum, Eleanor A.↗

Scaling Graph Community Detection on the Tilera Many-core Architecture

In an era when power constraints and data movement are proving to be significant barriers for the application of high-end computing, the Tilera many-core architecture offers a low-power platform exhibiting many important characteristics of future systems, including a large number of simple cores, a sophisticated network-on-chip, and fine-grained control over memory and caching policies. While this emerging architecture has been previously studied for structured compute-intensive kernels, benchmarking the platform for data-bound, irregular applications present significant challenges that have remained unexplored. Community detection is an advanced prototypical graph-theoretic operation with applications in numerous scientific domains including life sciences, cyber security, and power systems. In this work, we explore multiple design strategies toward developing a scalable tool for community detection on the Tilera platform. Using several memory layout and work scheduling techniques we demonstrate speedups of up to 46x on 36 cores of the Tilera TileGX36 platform over the best serial implementation, and also show results that have comparable quality and performance to mainstream x86 platforms. To the best of our knowledge this is the first work addressing graph algorithms on the Tilera platform. This study demonstrates that through careful design space exploration, low-power many-core platforms like Tilera can be effectively exploited for graph algorithms that that embody all the essential characteristics of an irregular application.

community detection↗

Implementing direct, spatially isolated problems on transputer networks

Parametric studies were performed on transputer networks of up to 40 processors to determine how to implement and maximize the performance of the solution of problems where no processor-to-processor data transfer is required for the problem solution (spatially isolated). Two types of problems are investigated a computationally intensive problem where the solution required the transmission of 160 bytes of data through the parallel network, and a communication intensive example that required the transmission of 3 Mbytes of data through the network. This data consists of solutions being sent back to the host processor and not intermediate results for another processor to work on. Studies were performed on both integer and floating-point transputers. The latter features an on-chip floating-point math unit and offers approximately an order of magnitude performance increase over the integer transputer on real valued computations. The results indicate that a minimum amount of work is required on each node per communication to achieve high network speedups (efficiencies). The floating-point processor requires approximately an order of magnitude more work per communication than the integer processor because of the floating-point unit's increased computing capacity.

Ellis, Graham K.↗

Parallel IO Libraries for Managing HEP Experimental Data

The computing and storage requirements of the energy and intensity frontiers will grow significantly during the Run 4 & 5 and the HL-LHC era. Similarly, in the intensity frontier, with larger trig ger readouts during supernovae explosions, the Deep Underground Neutrino Experiment (DUNE) will have unique computing challenges that could be addressed by the use of parallel and accelerated dataprocessing capabilities. Most of the requirements of the energy and intensity frontier experiments rely on increasing the role of high performance computing (HPC) in the HEP community. In this presentation, we will describe our ongoing efforts that are focused on using HPC resources for the next generation HEP experiments. The HEPCCE (High Energy Physics-Center for Computational Excellence) IOS (Input/Output and Storage) group has been developing approaches to map HEP data to the HDF5 , an IO library optimized for the HPC platforms to store the intermediate HEP data. The complex HEP data products are serialized using ROOT to allow for experiment independent general mapping approaches of the HEP data to the HDF5 format. The mapping approaches can be optimized for high performance parallel IO. Similarly, simpler data can be directly mapped into the HDF5, which can also be suitable for offloading into the GPUs directly. We will present our works on both complex and simple data model models.

Bashyal, Amit↗

Improving I/O-aware Workflow Scheduling via Data Flow Characterization and trade-off Analysis

The scientific computing paradigm has transitioned from compute-intensive to I/O-intensive and memory-intensive in the past decade, especially when data-driven science has become common practice. Numerous empirical I/O-aware scheduling optimizations have been developed by incorporating I/O capacity and bandwidth as constraints into scheduling. Unfortunately, there is a lack of data flow (I/O) characterization tool and an understanding of trade-offs between concurrency, locality, and I/O bandwidth. To bridge the gap, this work 1) presents a set of descriptors to characterize, organize, and visualize I/O profiles, including flow size, I/O bandwidth, and operation count, which group data flows by I/O types, tasks, and files; 2) proposes an I/O Roofline model-based trade-off analysis to find the optimal trade-off between flow operational intensity, concurrency, and flow performance. The I/O descriptors generate useful insights into complicated I/O behaviors, suggesting distinct concurrency, storage, and scheduling to be used by types, tasks, and files. The proposed trade-off analysis guides scheduling decisions that generate resource assignment with the best flow parallelism. We evaluate our I/O-aware scheduling methodology on a highly I/O-intensive workflow–1000 Genomes. The experimental results demonstrate speedups of up to 2.4× compared to the state-of-the- art methods.

Guo, Luanzheng [BATTELLE (PACIFIC NW LAB)]↗

Experiment in Onboard Synthetic Aperture Radar Data Processing

Single event upsets (SEUs) are a threat to any computing system running on hardware that has not been physically radiation hardened. In addition to mandating the use of performance-limited, hardened heritage equipment, prior techniques for dealing with the SEU problem often involved hardware-based error detection and correction (EDAC). With limited computing resources, software- based EDAC, or any more elaborate recovery methods, were often not feasible. Synthetic aperture radars (SARs), when operated in the space environment, are interesting due to their relevance to NASAs objectives, but problematic in the sense of producing prodigious amounts of raw data. Prior implementations of the SAR data processing algorithm have been too slow, too computationally intensive, and require too much application memory for onboard execution to be a realistic option when using the type of heritage processing technology described above. This standard C-language implementation of SAR data processing is distributed over many cores of a Tilera Multicore Processor, and employs novel Radiation Hardening by Software (RHBS) techniques designed to protect the component processes (one per core) and their shared application memory from the sort of SEUs expected in the space environment. The source code includes calls to Tilera APIs, and a specialized Tilera compiler is required to produce a Tilera executable. The compiled application reads input data describing the position and orientation of a radar platform, as well as its radar-burst data, over time and writes out processed data in a form that is useful for analysis of the radar observations.

Holland, Matthew↗

Cognition at the Point of Sensing

Over the last 15 years, compressive sensing techniques have been developed which have the potential to greatly reduce the amount of data collected by systems while preserving the amount of information obtained. A cost of this efficiency is that a computationally-intensive optimization routine must be used to put the sensed data into a form that a person can interpret. At the same time, machine learning techniques have experienced tremendous growth as well. Machines have demonstrated the ability learn how to effectively perform tasks such as detection and classification at speeds much faster than humanly possible. Our goal in this project was to study the feasibility of using compressive sensing systems "at the edge." That is, how can compressive sensing sensors be deployed such that information is created at the remote sensor rather than sending raw data to a central processing location? Studies were performed to analyze whether machine learning could be done on the compressively sensed data in its raw form. If a machine is performing the task, is it possible to do so without putting the data into a human interpretable form? We show that this is possible for some systems, in particular a compressive sensing snapshot imaging spectrometer. Machine learning tasks were demonstrated to be more effective and more robust to noise when the machine learning algorithm worked on data in its raw form. This system is shown to outperform a traditional spectrometer. Techniques for reducing the complexity of the reconstruction routine were also analyzed. Techniques for such as data regularization, deep neural networks, and matrix completion were studied and shown to have benefits over traditional reconstruction techniques. In this project we showed that compressive sensing sensors are indeed feasible at the edge. As always, sensors and algorithms must be carefully tuned to work in the constrained environment. In this project we developed tools and techniques to enable those analyses.

47 OTHER INSTRUMENTATION↗

Enabling the High Level Synthesis of Data Analytics Accelerators

Conventional High Level Synthesis (HLS) tools mainly tar- get compute intensive kernels typical of digital signal pro- cessing applications. We are developing techniques and ar- chitectural templates to enable HLS of data analytics appli- cations. These applications are memory intensive, present fine-grained, unpredictable data accesses, and irregular, dy- namic task parallelism. We discuss an architectural tem- plate based around a distributed controller to efficiently ex- ploit thread level parallelism. We present a memory in- terface that supports parallel memory subsystems and en- ables implementing atomic memory operations. We intro- duce a dynamic task scheduling approach to efficiently ex- ecute heavily unbalanced workload. The templates are val- idated by synthesizing queries from the Lehigh University Benchmark (LUBM), a well know SPARQL benchmark.

Data Analytics↗

Maximum-Intensity Volumes for Fast Contouring of Lung Tumors Including Respiratory Motion in 4DCT Planning

Purpose: To assess the accuracy of maximum-intensity volumes (MIV) for fast contouring of lung tumors including respiratory motion. Methods and Materials: Four-dimensional computed tomography (4DCT) data of 10 patients were acquired. Maximum-intensity volumes were constructed by assigning the maximum Hounsfield unit in all CT volumes per geometric voxel to a new, synthetic volume. Gross tumor volumes (GTVs) were contoured on all CT volumes, and their union was constructed. The GTV with all its respiratory motion was contoured on the MIV as well. Union GTVs and GTVs including motion were compared visually. Furthermore, planning target volumes (PTVs) were constructed for the union of GTVs and the GTV on MIV. These PTVs were compared by centroid position, volume, geometric extent, and surface distance. Results: Visual comparison of GTVs demonstrated failure of the MIV technique for 5 of 10 patients. For adequate GTV{sub MIV}s, differences between PTVs were <1.0 mm in centroid position, 5% in volume, {+-}5 mm in geometric extent, and {+-}0.5 {+-} 2.0 mm in surface distance. These values represent the uncertainties for successful MIV contouring. Conclusion: Maximum-intensity volumes are a good first estimate for target volume definition including respiratory motion. However, it seems mandatory to validate each individual MIV by overlaying it on a movie loop displaying the 4DCT data and editing it for possible inadequate coverage of GTVs on additional 4DCT motion states.

62 RADIOLOGY AND NUCLEAR MEDICINE↗