Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,459 records · Page 81

Eureka: Enabling Fine-Grained Access and Range Queries on Compressed Scientific Data via Data-Index Co-Compression

Handling large-scale scientific data in high-performance computing (HPC) environments poses significant challenges, including excessive I/O, high storage costs, and slow query performance. Traditional approaches often require full data decompression and scans, making them impractical for real-time or interactive analysis. To address these limitations, we introduce Eureka, a unified data-index co-compression framework that enables fine-grained access and efficient range queries on compressed scientific datasets. Eureka integrates spatial domain decomposition with block-wise error-bounded lossy compression to support selective decompression. It constructs a hierarchical AVL-tree index during compression to capture block-level value ranges, enabling fast pruning during query execution. To reduce metadata overhead, the index itself is also compressed while ensuring recall-preserving results. Experiments on six diverse HPC simulation datasets show that Eureka achieves up to 25x data compression and over 300x index compression, surpassing state-of-the-art compressors such as SZ3 and ZFP in rate-distortion performance. Additionally, Eureka delivers over 30x speedup for low-selectivity range queries, making it a scalable and efficient solution for modern scientific data analysis.

Yan, Ning↗

Integrating ORNL’s HPC and Neutron Facilities with a Performance-Portable CPU/GPU Ecosystem

We explore the development of a performance-portable CPU/GPU ecosystem to integrate two of the US Department of Energy’s (DOE’s) largest scientific instruments, the Oak Ridge Leadership Computing facility and the Spallation Neutron Source (SNS), both of which are housed at Oak Ridge National Laboratory. We select a relevant data reduction workflow use-case to obtain the differential scattering cross-section from data collected by SNS’s CORELLI and TOPAZ instruments. We compare the current CPU-only production implementation using the Garnet Python multiprocess package based on the Mantid C++ framework against our proposed CPU/GPU implementation that uses the LLVM-based, just-in-time Julia scientific language and the JACC.jl performance-portable package. Two proxy apps were developed: (i) an app for extracting relevant Mantid kernels (MDNorm) in C++ and (ii) the Julia MiniVATES.jl miniapp. We present performance results for NVIDIA A100 and AMD MI100 GPUs and AMD EPYC 7513 and 7662 CPUs. The results provide insights for future generations of data reduction software that can embrace performance portability for an integrated research infrastructure across DOE’s experimental and computational facilities.

Hahn, Steven↗

Shape-shifting Elephants: Multi-modal Transport for Integrated Research Infrastructure

Data Acquisition (DAQ) workloads form an important class of scientific network traffic that by its nature (1) flows across different research infrastructure, including remote instruments and supercomputer clusters, (2) has ever-increasing throughput demands, and (3) has ever-increasing integration demands---for example, observations at one instrument could trigger a reconfiguration of another instrument. Today's DAQ transfers rely on UDP and (heavily tuned) TCP, but this is driven by convenience rather than suitability. The mismatch between Internet transport protocols and scientific workloads becomes more stark with the steady increase in link capacities, data generation, and integration across research infrastructure.This position paper argues the importance of developing specialized transport protocols for DAQ workloads. It proposes a new transport feature for this kind of elephant flow: multi-modality involves the network actively configuring the transport protocol to change how DAQ flows are processed across different underlying networks that connect scientific research infrastructure. Multi-modality is a layering violation that is proposed as a pragmatic technique for DAQ transport protocol design. It takes advantage of programmable network hardware that is increasingly being deployed in scientific research infrastructure. The paper presents an initial evaluation through a pilot study that includes a Tofino2 switch and Alveo FPGA cards, and using data from a particle detector.

97 MATHEMATICS AND COMPUTING↗

Big Data Ecosystems Enable Scientific Discovery

Over the past 5 years, advances in experimental, sensor and computational technologies have driven the exponential growth in the volumes, acquisition rates, variety and complexity of scientific data. As noted by Hey et al in their 2009 e-book The Fourth Paradigm, this availability of large-quantities of scientifically meaningful data has given rise to a new scientific methodology - data intensive science. Data intensive science is the ability to formulate and evaluate hypotheses using data and analysis to extend, complement and, at times, replace experimentation, theory, or simulation. This new approach to science no longer requires scientists to interact directly with the objects of their research; instead they can utilize digitally captured, reduced, calibrated, analyzed, synthesized and visualized results - allowing them carry out 'experiments' in data.

99 GENERAL AND MISCELLANEOUS↗

Big PanDa Workflow Management on Titan for High Energy and Nuclear Physics and for Future Extreme Scale Scientific Application

Over a three year period, from 2016-2019, this project demonstrated the scientific benefits of integrating the Titan supercomputer at Oak Ridge Leadership Computing Facility into traditional high throughput grid based distributed computing systems managed by PanDA, the workflow management system used for the execution of all distributed computing applications by the ATLAS experiment at the Large Hadron Collider. PanDA manages millions of batch jobs daily at hundreds of clusters worldwide on request by thousands of physicist users, and processes more than an exabyte of data annually using grid middleware. High levels of operational use of Titan was sustained by PanDA in order to meet the physics goals of ATLAS. The success of this project led to the use of other supercomputers worldwide by ATLAS, and to the adoption of PanDA by other experiments and other scientists. Multiple innovative operational and computer science research goals were achieved supporting the use of supercomputers for scientific domains with large scale distributed data and distributed processing needs.

97 MATHEMATICS AND COMPUTING↗

2HOT: An improved parallel hashed oct-tree N-body algorithm for cosmological simulation

Here we report on improvements made over the past two decades to our adaptive treecode N-body method (HOT). A mathematical and computational approach to the cosmological N-body problem is described, with performance and scalability measured up to 256k (218) processors. We present error analysis and scientific application results from a series of more than ten 69 billion (40963) particle cosmological simulations, accounting for 4×1020 floating point operations. These results include the first simulations using the new constraints on the standard model of cosmology from the Planck satellite. Our simulations set a new standard for accuracy and scientific throughput, while meeting or exceeding the computational efficiency of the latest generation of hybrid TreePM N-body methods.

97 MATHEMATICS AND COMPUTING↗

The parallelism motifs of genomic data analysis

Genomic datasets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share these data with the research community, but some of these genomic data analysis problems require large-scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high-end parallel systems today and place different requirements on programming support, software libraries and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high-performance genomics analysis, including alignment, profiling, clustering and assembly for both single genomes and metagenomes. We identify some of the common computational patterns or ‘motifs’ that help inform parallelization strategies and compare our motifs to some of the established lists, arguing that at least two key patterns, sorting and hashing, are missing. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

59 BASIC BIOLOGICAL SCIENCES↗

2HOT: an improved parallel hashed oct-tree n-body algorithm for cosmological simulation

We report on improvements made over the past two decades to our adaptive treecode N-body method (HOT). A mathematical and computational approach to the cosmological N-body problem is described, with performance and scalability measured up to 256k (2 18 ) processors. We present error analysis and scientific application results from a series of more than ten 69 billion (4096 3 ) particle cosmological simulations, accounting for 4 × 10 20 floating point operations. These results include the first simulations using the new constraints on the standard model of cosmology from the Planck satellite. Our simulations set a new standard for accuracy and scientific throughput, while meeting or exceeding the computational efficiency of the latest generation of hybrid TreePM N-body methods.

Computer Science↗

2HOT: An Improved Parallel Hashed Oct-Tree N-Body Algorithm for Cosmological Simulation

We report on improvements made over the past two decades to our adaptive treecode N-body method (HOT). A mathematical and computational approach to the cosmological N-body problem is described, with performance and scalability measured up to 256k (2 18 ) processors. We present error analysis and scientific application results from a series of more than ten 69 billion (4096 3 ) particle cosmological simulations, accounting for 4×10 20 floating point operations. These results include the first simulations using the new constraints on the standard model of cosmology from the Planck satellite. Our simulations set a new standard for accuracy and scientific throughput, while meeting or exceeding the computational efficiency of the latest generation of hybrid TreePM N-body methods.

Warren, Michael S.↗

PluginPlay: Enabling exascale scientific software one module at a time

For many computational chemistry packages, being able to efficiently and effectively scale across an exascale cluster is a heroic feat. Collective experience from the Department of Energy’s Exascale Computing Project suggests that achieving exascale performance requires far more planning, design, and optimization than scaling to petascale. In many cases, entire rewrites of software are necessary to address fundamental algorithmic bottlenecks. This in turn requires a tremendous amount of resources and development time, resources that cannot reasonably be afforded by every computational science project. It thus becomes imperative that computational science transition to a more sustainable paradigm. Key to such a paradigm is modular software. While the importance of modular software is widely recognized, what is perhaps not so widely appreciated is the effort still required to leverage modular software in a sustainable manner. The present manuscript introduces PluginPlay, https://github.com/NWChemEx-Project/PluginPlay, an inversion-of-control framework designed to facilitate developing, maintaining, and sustaining modular scientific software packages. Here this manuscript focuses on the design aspects of PluginPlay and how they specifically influence the performance of the resulting package. Although, PluginPlay serves as the framework for the NWChemEx package, PluginPlay is not tied to NWChemEx or even computational chemistry. We thus anticipate PluginPlay to prove to be a generally useful tool for a number of computational science packages looking to transition to the exascale.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

On-demand data analytics in HPC environments at leadership computing facilities: Challenges and experiences

Scaling-up scientific data analysis and machine learning algorithms for data-driven discovery is a grand challenge that we face today. Despite the growing need for analysis from science domains that are generating `Big Data' from instruments and simulations, building high-performance analytical workflows of data-intensive algorithms have been daunting because: (i) the `Big Data' hardware and software architecture landscape is constantly evolving, (ii) newer architectures impose new programming models, and (iii) data-parallel kernels of analysis algorithms and their performance facets on different architectures are poorly understood. To address these problems, we have: (i) identified scalable data-parallel kernels of popular data analysis algorithms, (ii) implemented `Mini-Apps' of those kernels using different programming models (e.g. Map Reduce, MPI, etc.), (iii) benchmarked and validated the performance of the kernels in diverse architectures. In this paper, we discuss two of those Mini-Apps and show the execution of principal component analysis built as a workflow of the Mini-Apps. We show that Mini-Apps enable scientists to (i) write domain-specific data analysis code that scales on most HPC hardware and (ii) and offers the ability (most times with over a 10x speed-up) to analyze data sizes 100 times the size of what off-the-shelf desktop/workstations of today can handle.

Harney, John↗

Site Environmental Report for 2016

Lawrence Berkeley National Laboratory (LBNL, Berkeley Lab) is a multi-program scientific facility operated by the University of California (UC) for the U.S. Department of Energy (DOE). Berkeley Lab’s research is focused on the physical, biological, environmental, and computational sciences with the objective of delivering scientific knowledge and discoveries pertinent to DOE’s mission. This annual report describes environmental protection activities and potential impacts resulting from LBNL operations conducted in 2016. The format and content of this report satisfy the requirements of both DOE Order 231.1B Admin Chg 1, Environment, Safety, and Health Reporting, and the operating contract between UC and DOE (DOE Contract No. DE-AC02-05CH11231, also known as Contract 31). Activities are planned and conducted with full regard to protecting employees, the public, and the environment, as well as complying with all applicable environmental, safety, and health laws and regulations. Berkeley Lab implements an Environmental Management System (EMS) to oversee these compliance activities and improve overall environmental performance while maintaining operational capability and sustaining its overall mission. The effectiveness of the EMS and environmental programs is reviewed annually as part of the operating contract’s performance evaluation process. For fiscal year (FY) 2016, which began October 1, 2015, and ended September 30, 2016, the EMS was given a performance rating of A minus for its management of environmental activities (on a scale from A plus as the highest grade to F as the lowest). The measurement and rating system was developed jointly by Berkeley Lab, UC, and DOE. The FY 2016 rating was based on how Berkeley Lab met the objective in DOE’s FY 2017 Performance Evaluation and Measurement Plan (Appendix B in Contract 31, Section J) of providing an efficient and effective EMS. To support this rating, Berkeley Lab prepared a list of projects completed during the performance period that achieved reduced environmental impacts. The EMS was also graded through the federal Office of Management and Budget’s EMS Annual Report Data, in which elements of the International Organization for Standardization (ISO) 14001 standard are rated and the degree of integration between the EMS and Berkeley Lab’s sustainable practices is measured. Overall scores fall into one of three categories: green (highest), yellow (middle), or red (lowest). For FY 2016, Berkeley Lab received a score of green, as described in more detail in Chapter 2. An overview of environmental protection and restoration programs is provided (Chapter 3), including information about compliance activities, operating permits, and regulatory agency inspections that occurred during 2016. Six minor violations issued during these inspections are discussed, one of which was rescinded in 2017. This report also includes information on environmental monitoring performed in 2016 (Chapter 4). The results of these monitoring activities confirmed that groundwater cleanup actions continue to show improving conditions, and all emissions and discharges from LBNL operations were within environmental compliance release limits with the exception of some stormwater discharges. While most stormwater discharges measured throughout the LBNL site fall within acceptable levels established by the state's stormwater permit, iron and aluminum exceeded permitted levels. Berkeley Lab is aggressively implementing controls to reduce iron and aluminum discharge levels. The controls include construction of asphaltic berms to restrain and filter runoff to storm drains, and installation of filtration units in storm drain basins to absorb metal contaminants. The radiological dose assessments (Chapter 5) performed in 2016 concluded that the maximum dose to a hypothetical resident from Berkeley Lab’s airborne radionuclide releases was approximately 0.12% of the DOE and U.S. Environmental Protection Agency annual limit of 10 millirem per year (mrem/year); the dose from all radiation sources at Berkeley Lab was approximately 0.1% of the average natural background radiation dose of 310 mrem/yr in the United States, and about 0.4% of the DOE annual limit of 100 mrem/yr from all sources.

54 ENVIRONMENTAL SCIENCES↗

Site Environmental Report for 2017

Lawrence Berkeley National Laboratory (LBNL, Berkeley Lab) is a multi-program scientific facility operated by the University of California (UC) for the U.S. Department of Energy (DOE). Berkeley Lab’s research is focused on the physical, biological, environmental, and computational sciences with the objective of delivering scientific knowledge and discoveries pertinent to DOE’s mission. This annual report describes environmental protection activities and potential impacts resulting from LBNL operations conducted in 2017. The format and content of this report satisfy the requirements of both DOE Order 231.1B, Administrative Change 1 (Environment, Safety, and Health Reporting) and the operating contract between UC and DOE (DOE Contract No. DE-AC02-05CH11231, also known as Contract 31). Activities are planned and conducted with full regard to protecting employees, the public, and the environment, as well as complying with all applicable environmental, safety, and health laws and regulations. Berkeley Lab implements an Environmental Management System (EMS) to oversee these compliance activities and continually improve overall environmental performance while maintaining operational capability and sustaining its overall mission. The effectiveness of the EMS and environmental programs is reviewed annually as part of the performance evaluation process of Contract 31. For fiscal year (FY) 2017, which began October 1, 2016, and ended September 30, 2017, the EMS was given a performance rating of B plus for its management of environmental activities (on a scale from A plus as the highest grade to F as the lowest). The measurement and rating system was developed jointly by Berkeley Lab, UC, and DOE. The FY 2017 rating was based on how Berkeley Lab met the objective in DOE’s FY 2017 Performance Evaluation and Measurement Plan (Appendix B in Contract 31, Section J) of providing an efficient and effective EMS. Six significant accomplishments, which ranged from effective teaming on projects and with regulatory agencies to improved program assessment approaches, were factors in the rating. The EMS was also graded through the federal Office of Management and Budget’s annual EMS performance metrics, in which a reporting scorecard rates elements of the International Organization for Standardization (ISO) 14001 standard and the degree of integration between the EMS and Berkeley Lab’s sustainable practices. Overall scores fall into one of three categories: green (highest), yellow (middle), or red (lowest). For FY 2017, Berkeley Lab received a score of green, as described in more detail in Chapter 2. An overview of environmental protection and restoration programs is provided, including information about compliance activities, operating permits, and regulatory agency inspections that occurred during 2017. Thirteen minor violations issued during City of Berkeley inspections of above and underground storage tanks, treatment units, and hazardous waste storage areas are discussed in this chapter. This report also includes information on environmental monitoring performed in 2017. The results of these monitoring activities confirmed that groundwater cleanup actions continue to show improving conditions, and all emissions and discharges from LBNL operations were within environmental compliance release limits, with the exception of some stormwater discharges. Most stormwater discharges measured throughout the LBNL site fall within acceptable levels established by the state’s stormwater permit; however, iron and aluminum exceeded Numeric Action Levels. To reduce iron and aluminum discharge levels, Berkeley Lab is aggressively implementing controls such as construction of asphaltic berms and check dams to restrain and filter runoff to storm drains, and installation of filtration units in storm drain basins to collect sediment and absorb metal contaminants. The radiological dose assessments performed in 2017 concluded that the maximum potential dose to a hypothetical resident from Berkeley Lab’s airborne radionuclide releases was approximately 0.1% of the DOE and U.S. Environmental Protection Agency annual limit of 10 millirem per year (mrem/yr); the potential dose from all radiation sources at Berkeley Lab was approximately 0.06% of the average natural background radiation dose of 310 mrem/yr in the United States, and about 0.2% of the DOE annual limit of 100 mrem/yr from all sources.

54 ENVIRONMENTAL SCIENCES↗

Site Environmental Report for 2018

Lawrence Berkeley National Laboratory (LBNL, Berkeley Lab) is a multi-program scientific facility operated by the University of California (UC) for the U.S. Department of Energy (DOE). Berkeley Lab’s research is focused on the physical, biological, environmental, and computational sciences, with the objective of delivering scientific knowledge and discoveries pertinent to DOE’s mission. This annual report describes environmental protection activities and potential impacts resulting from LBNL operations conducted in 2018. The format and content of this report satisfy the requirements of both DOE Order 231.1B, Administrative Change 1 (Environment, Safety, and Health Reporting) and the operating contract between UC and DOE (DOE Contract No. DE-AC02-05CH11231, also known as Contract 31). LBNL activities are planned and conducted with full regard to protecting employees, the public, and the environment, as well as complying with all applicable environmental, safety, and health laws and regulations. As presented in Chapter 2, Berkeley Lab implements an Environmental Management System (EMS) to oversee environmental compliance activities and continually improve overall environmental performance while maintaining operational capability and sustaining its overall mission. The effectiveness of the EMS and environmental programs is reviewed annually as part of the performance evaluation process of Contract 31. For fiscal year (FY) 2018, which began October 1, 2017, and ended September 30, 2018, the EMS was given a performance rating of B plus for its management of environmental activities (on a scale from A plus as the highest grade, to F as the lowest). The measurement and rating system was developed jointly by Berkeley Lab, UC, and DOE. The FY 2018 rating was based on how Berkeley Lab met the objective in DOE’s FY 2018 Performance Evaluation and Measurement Plan (Appendix B in Contract 31, Section J) of providing an efficient and effective EMS. Six significant accomplishments, which ranged from effective teaming on projects and with regulatory agencies to improved program assessment approaches, were factors in the rating. The EMS was also graded through the federal Office of Management and Budget’s annual EMS performance metrics, in which a reporting scorecard rates elements of the International Organization for Standardization (ISO) 14001 standard and the degree of integration between the EMS and Berkeley Lab’s sustainable practices. Overall scores fall into one of three categories: green (highest), yellow (middle), or red (lowest). For FY 2018, Berkeley Lab received a score of green. A week-long formal conformity audit of the EMS was conducted at the end of July by a qualified external party, as is required every three years. The audit team found that Berkeley Lab’s EMS continues to conform to the ISO standard with no major nonconformance. An overview of environmental protection and restoration programs is provided in Chapter 3, including information about compliance activities, operating permits, and regulatory agency inspections and audits that occurred during 2018. Twenty-one minor violations and two Class II violations were issued during agency inspections of programs governing aboveground and underground storage tanks, hazardous waste treatment units, hazardous waste storage areas, hazardous materials management, and wastewater, stormwater, and air emission sources. This report also includes information on environmental monitoring performed in 2018 (Chapter 4). The results of these monitoring activities confirmed that groundwater cleanup actions have been effective in reducing concentrations of volatile organic compounds (VOCs) in the groundwater, although concentrations appear to be reaching asymptotic levels in some areas. Site groundwater plumes are stable or are attenuating, and VOCs are not migrating off site. All emissions and discharges from LBNL operations were within environmental compliance release limits except oxides of nitrogen and carbon monoxide from boilers (under 5 million BTUs) from Buildings 2, 66, and 88. The radiological dose assessments (Chapter 5) performed in 2018 concluded that the maximum potential dose to a hypothetical resident from Berkeley Lab’s airborne radionuclide releases was approximately 0.04% of the DOE and U.S. Environmental Protection Agency annual limit of 10 millirem per year (mrem/yr); the potential dose from all radiation sources at Berkeley Lab was approximately 0.15% of the average natural background radiation dose of 310 mrem/yr in the United States, and about 0.5% of the DOE annual limit of 100 mrem/yr from all sources.

54 ENVIRONMENTAL SCIENCES↗

Investigating Scientific Workload Acceleration using BlueField SmartNICs [Slides]

Modern computing platforms whose workloads generate large amounts of network traffic, such as cloud and HPC systems, often suffer from performance bottlenecks associated with the network interface. In order to alleviate the effects of this obstacle, a new generation of accelerators known as ‘SmartNICs’, which are designed to offload low level networking tasks from the processor into the NIC, have emerged.

42 ENGINEERING↗

2021 Site Environmental Report (SER) for the Ernest Orlando Lawrence Berkeley National Laboratory (LBNL)

Lawrence Berkeley National Laboratory (LBNL, Berkeley Lab) is a multiprogram scientific facility operated by the University of California (UC) for the U.S. Department of Energy (DOE). Berkeley Lab’s research is focused on the physical, biological, environmental, and computational sciences, with the objective of delivering scientific knowledge and discoveries pertinent to DOE’s mission. This annual report describes environmental protection activities and potential impacts resulting from operations conducted in calendar year 2021, unless otherwise indicated. The format and content of this report satisfy the requirements of both DOE Order 231.1B, Administrative Change 1 (Environment, Safety, and Health Reporting) (DOE, 2012) and the operating contract between UC and DOE (DOE Contract No. DE-AC02-05CH11231, also known as Contract 31).

54 ENVIRONMENTAL SCIENCES↗

Design of Mega-Voltage X-ray Digital Radiography and Computed Tomography Performance Phantoms

A number of fundamental scientific questions have arisen concerning the operation of high-energy DR and CT systems. Some of these questions include: (1) How deeply can such systems penetrate thickly shielded objects? (2) How well can such systems distinguish between dense and relatively high Z materials such as lead, tungsten and depleted uranium and lower Z materials such as steel, copper and tin? (3) How well will such systems operate for a uranium material which is an intermediate case between low density yellowcake and high density depleted uranium metal? These questions have led us to develop a set of phantoms to help answer these questions, but do not have any direct bearing on any smuggling concern. These new phantoms are designed to allow a systemic exploration of these questions by gradually varying their compositions and thicknesses. These phantoms are also good probes of the blurring behavior of radiography and tomography systems. These phantoms are composed of steel ({rho} assumed to be 7.8 g/cc), lead ({rho} assumed to be 11.4 g/cc), tungsten ({rho} assumed to be 19.25 g/cc), uranium oxide (UO{sub 3}) ({rho} assumed to be 4.6 g/cc), and depleted uranium (DU) ({rho} assumed to be 18.9 g/cc). There are five designed phantoms described in this report: (1) Cylindrical shells of Tungsten and Steel; (2) Depleted Uranium Inside Tungsten Hemi-cube Shells; (3) Nested Spherical Shells; (4) UO{sub 3} Cylinder; and (5) Shielded DU Sphere.

42 ENGINEERING↗