Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Experiences with a Flexible User Research Process to Build Data Change Tools

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects.In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

ASC Data Science Infrastructure Project June 2023 [Slides]

Data Science Infrastructure (DSI) Project offers a standardized but flexible way of storing data and associated metadata to support AI/ML and data intensive scientific workflows: Establish requirements via use case interviews; Develop DSI framework to support flexible, secure, and accessible data stores tailored to researcher needs; Augment existing institutional frameworks for code management, performance testing and file indexing via automated DSI interfaces; Deploy DSI framework to support key use cases in collaboration with domain scientists.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Enabling modern data discovery for atmospheric measurements

The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Aluminum Electrolytic Capacitor Vulnerability Evaluation in DC Power Supplies at the Spallation Neutron Source

The service life of electrolytic capacitors is a concern for long-term reliable operation of power supplies. The Power Conversion Group at the Spallation Neutron Source (SNS) facility of Oak Ridge National Laboratory is responsible for more than 500 power supplies with over 1000 electrolytic capacitors installed in these power supplies. Most of the electrolytic capacitors have been in operation since 2007 and are either well over or close to the manufacturer provided lifetime. This paper addresses this vulnerability to ensure continued reliable operation of the SNS. The power supply circuit is simulated in a MATLAB/Simulink environment to quantify the capacitor ripple current and operating voltage. Combined with manufacturer data, this information is used to estimate capacitor lifetime. Utilizing the simulation results and lifetime projections of the capacitors, capacitor replacements for the power supplies is prioritized. To assess end-of-life, the ambient temperature and ripple current of the capacitor is used. Based on the manufacturer's data and scientific literature review, capacitor end-of-life is established. The results of the simulations and analysis is presented. Data on the highest priority supplies will also be presented, and statistics on the capacitor performance, reliability and remaining lifetime will be shown.

Harave, Sudarshan↗
DOE OSTI2024

Developing time-resolved x-ray diffraction diagnostics at the National Ignition Facility (invited)

As part of a program to measure phase transition timescales in materials under dynamic compression, we have designed new x-ray imaging diagnostics to record multiple x-ray diffraction measurements during a single laser-driven experiment. Our design places several ns-gated hybrid CMOS (hCMOS) sensors within a few cm of a laser-driven target. The sensors must be protected from an extremely harsh environment, including debris, electromagnetic pulses, and unconverted laser light. Another key challenge is reducing the x-ray background relative to the faint diffraction signal. Building on the success of our predecessor (Target Diffraction In Situ), we implemented a staged approach to platform development. First, we built a demonstration diagnostic (Gated Diffraction Development Diagnostic) with two hCMOS sensors to confirm we could adequately protect them from the harsh environment and also acquire acceptable diffraction data. This allowed the team to quickly assess the risks and address the most significant challenges. Here, we also collected scientifically useful data during development. Leveraging what we learned, we recently developed a much more ambitious instrument (Flexible Imaging Diffraction Diagnostic for Laser Experiments) that can field up to eight hCMOS sensors in a flexible geometry and participate in back-to-back shots at the National Ignition Facility (NIF). The design also allows for future iterations, such as faster hCMOS sensors and an embedded x-ray streak camera. The enhanced capabilities of the new instrument required a much more complex design, and the unexpected issues encountered on the first few shots at NIF remind us that complexity has consequences. Our progress in addressing these challenges is described herein, as is our current focus on improving data quality by reducing x-ray background and quantifying the uncertainties of our diffraction measurements.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Surrogate multi-fidelity data and model fusion for scientific discovery and uncertainty quantification in Earth System Models

This whitepaper addresses the Earth and Environmental Systems Sciences Division (EESSD)’s predictability challenges in modeling the integrated water cycle and data-model integration. Specifically, it focuses on reducing and characterizing the uncertainty in the representation of process models for unresolved physics, either due to model resolution or limited by the physical under standing or computational efficiency, and the use of observational data for in-situ process parameter optimization within ESM. The described methods may also be used to determine the nature of responses (e.g. strength and direction), and hence to identify critical processes that drive the overall ESM responses to perturbation in the forcing

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

First Report of the Nuclear Data Subcommittee of the Nuclear Science Advisory Committee

Accurate, reliable nuclear data is essential for the success of Federal missions such as nonproliferation, nuclear forensics, homeland security, national defense, space exploration, clean energy generation, and scientific research. Data access is also key to innovative commercial developments such as new medicines, automated industrial controls, energy exploration, energy security, nuclear reactor design, and isotope production. The United States Nuclear Data Program (USNDP) is the domestic custodian of nuclear data. In its April 2022 meeting, the DOE/NSF Nuclear Science Advisory Committee was charged with preparing two reports on nuclear data. In this first report, we review recent accomplishments of the USNDP and discuss complementary and collaborative international efforts. Detailed descriptions of nuclear data needs for basic science, nonproliferation, national security, nuclear energy together with medical and space applications are also presented. Lastly, a set of specific cross-cutting nuclear data needs with relevance for multiple applications areas are also identified for further discussion in a follow-on report planned for release at the end of January 2023.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Cross-facility science with the Superfacility Project at LBNL

As data sets from DOE user science facilities grow in both size and complexity there is an urgent need for new capabilities to transfer, analyze and manage the data underlying scientific discoveries. LBNL's Superfacility project brings together experimental and observational research instruments with computational and network facilities at the National Energy Research Scientific Computing Center (NERSC) and the Energy Sciences Network (ESnet) with the goal of enabling user science. Here, we report on recent innovations in the Superfacility project, including advanced data management, API-based automation, real-Time interactive user interfaces, and supported infrastructure for 'edge' services.

Enders, B↗
DOE OSTI2023

Online and Scalable Data Compression Pipeline with Guarantees on Quantities of Interest

Data compression is becoming critical for data-intensive scientific applications. Scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Prior work has shown that a pipeline can be built to guarantee error on the primary data (PD) within user-defined bounds and achieve near-floating point QoI errors. In this paper, we present novel computational approaches for accelerating the pipeline and demonstrate results that enable concurrent execution of compression in parallel with the simulation nodes. This allows compression, including the writing of the required compression data, for the previous time step to be completed while the simulation proceeds with the current time step. Overall, the approach presented in this paper results in a 6–8 times improvement in computational overhead compared to previous work. These results were obtained using data generated by a large-scale fusion code called XGC, which produces hundreds of terabytes of data in a single day.

Banerjee, Tania↗
DOE OSTI2026

Enhancing ZFP: A Statistical Approach to Understanding and Reducing Error Bias in a Lossy Floating-Point Compression Algorithm

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression; but, lossless data compression is typically ineffective when applied to floating-point data. Thus, users tend to apply a lossy data compressor, which allows for small deviations from the original data. It is essential to understand how the error from lossy compression impacts the accuracy of the data analytics. Thus, we must analyze not only the compression properties but the error as well. In this paper, we provide a statistical analysis of the error caused by ZFP compression, a state-of-the-art, lossy compression algorithm explicitly designed for floating-point data. We show that the error is indeed biased and propose simple modifications to the algorithm to neutralize the bias and further reduce the resulting error.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Coastal Typologies: Methods for Improving Representation of Arctic Coastal Environments, Starting with Alaska's Northern Slope [Slides]

The objectives of the project included: Broaden QGIS (mapping) and Python (data analysis) skill set; Contribute to overall development of coastal typologies by conducting data mining and analysis; Expand scientific literacy; Narrow data sources to those most useful for project; Convert chosen data sets into image formats for analysis; and, Utilize python to analyze images.

58 GEOSCIENCES↗
DOE OSTI2024

Studying CPU and memory utilization of applications on Fujitsu A64FX and Nvidia Grace Superchip

ARM-based manycore CPU architectures are well-positioned to provide the rising memory throughput requirements of modern data intensive scientific applications in High Performance Computing (HPC). The Fujitsu A64FX CPU platform is based on the ARM v8.2A architecture, and is the processor of the flagship Japanese supercomputer - "Fugaku", which was previously ranked as the #1 supercomputer in the world according to the Top500 list. The Nvidia Grace superchip features 144 Neoverse V2 cores based on the ARMv9 architecture with 4x128b SVE2, providing exceptional computational power. The chip supports up to 480GB of memory, making it ideal for AI, machine learning, and scientific computing workloads. In this paper, we conduct a thorough performance exploration of a variety of parallel bandwidth-sensitive benchmarks and applications compiled with the native Fujitsu compiler on a Fugaku A64FX compute node and ARM (LLVM) Compiler on an NVIDIA Grace superchip compute node, engaging all the computational cores per cluster using OpenMP multithreading (assuming the cores can drive the available bandwidth). Our ultimate goals are to study the resource utilization of scientific applications and benchmarks on A64FX and Grace superchip, considering graph application scenarios ( GAP Benchmark suite) and eleven appli- cation proxies from the Rodinia heterogeneous benchmark suite (considering domains such as Data Mining, Bioinformatics, Fluid Dynamics, Pattern Recognition, etc.). Through exhaustive performance monitoring, we quantify the resource utilization of diverse OpenMP-based HPC applications on both the Fujitsu A64FX and the Nvidia Grace Superchip platforms.

benchmarking, Performance Analysis, High performan↗
DOE OSTI2022

Learning to simulate high energy particle collisions from unlabeled data

In many scientific fields which rely on statistical inference, simulations are often used to map from theoretical models to experimental data, allowing scientists to test model predictions against experimental results. Experimental data is often reconstructed from indirect measurements causing the aggregate transformation from theoretical models to experimental data to be poorly-described analytically. Instead, numerical simulations are used at great computational cost. We introduce Optimal-Transport-based Unfolding and Simulation (OTUS), a fast simulator based on unsupervised machine-learning that is capable of predicting experimental data from theoretical models. Without the aid of current simulation information, OTUS trains a probabilistic autoencoder to transform directly between theoretical models and experimental data. Identifying the probabilistic autoencoder’s latent space with the space of theoretical models causes the decoder network to become a fast, predictive simulator with the potential to replace current, computationally-costly simulators. Here, we provide proof-of-principle results on two particle physics examples, Z-boson and top-quark decays, but stress that OTUS can be widely applied to other fields.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

DeepPatent2: A Large-Scale Benchmarking Corpus for Technical Drawing Understanding

Abstract Recent advances in computer vision (CV) and natural language processing have been driven by exploiting big data on practical applications. However, these research fields are still limited by the sheer volume, versatility, and diversity of the available datasets. CV tasks, such as image captioning, which has primarily been carried out on natural images, still struggle to produce accurate and meaningful captions on sketched images often included in scientific and technical documents. The advancement of other tasks such as 3D reconstruction from 2D images requires larger datasets with multiple viewpoints. We introduce DeepPatent2, a large-scale dataset, providing more than 2.7 million technical drawings with 132,890 object names and 22,394 viewpoints extracted from 14 years of US design patent documents. We demonstrate the usefulness of DeepPatent2 with conceptual captioning. We further provide the potential usefulness of our dataset to facilitate other research areas such as 3D image reconstruction and image retrieval.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Secure Collaborative Environment for Seamless Sharing of Scientific Knowledge

In a secure collaborative environment, tera-bytes of data generated from powerful scientific instruments are used to train secure machine learning (ML) models on exascale computing systems, which are then securely shared with internal or external collaborators as cloud-based services. Devising such a secure platform is necessary for seamless scientific knowledge sharing without compromising individual, or institute-level, intellectual property and privacy details. By enabling new computing opportunities with sensitive data, we envision a secure collaborative environment that will play a significant role in accelerating scientific discovery. Several recent technological advancements have made it possible to realize these capabilities. In this paper, we present our efforts at ORNL toward developing a secure computation platform. We present a use case where scientific data generated from complex instruments, like those at the Spallation Neutron Source (SNS), are used to train a differential privacy enabled deep learning (DL) network on Summit, which is then hosted as a secure multi-party computation (MPC) service on ORNL’s Compute and Data Environment for Science (CADES) cloud computing platform for third-party inference. In this feasibility study, we discuss the challenges involved, elaborate on leveraged technologies, analyze relevant performance results and present the future vision of our work to establish secure collaboration capabilities within and outside of ORNL.

Yoginath, Srikanth↗
DOE OSTI2024

Extending deterministic transport capabilities for very-high and ultra-high energy electron beams

Focused Very-High Energy Electron (VHEE, 50–300 MeV) and Ultra-High Energy Electron (UHEE, > 300 MeV) beams can accurately target both large and deeply seated human tumors with high sparing properties, while avoiding the spatial requirements and cost of proton and heavy ion facilities. Advanced testing phases are underway at the CLEAR facilities at CERN (Switzerland), NLCTA at Stanford (USA), and SPARC at INFN (Italy), aiming to accelerate the transition to clinical application. Currently, Monte Carlo (MC) transport is the sole paradigm supporting preclinical trials and imminent clinical deployment. In this paper, we propose an alternative: the first extension of the nuclear-reactor deterministic chain Njoy-Dragon for VHEE and UHEE applications. We have extended the Boltzmann-Fokker-Planck (BFP) multigroup formalism and validated it using standard radio-oncology benchmarks, complex assemblies with a wide range of atomic numbers, and comprehensive irradiation of the entire periodic table. We report that 99% of water voxels exhibit a BFP-MC deviation below 2% for electron energies under 1.5 GeV. Additionally, we demonstrate that at least 97% of voxels of bone, lung, adipose tissue, muscle, soft tissue, tumor, steel, and aluminum meet the same criterion between 50 MeV and 1.5 GeV. For water, the thorax, and the breast intra-operative benchmark, typical average BFP-MC deviations of 0.3% and 0.4% were observed at 300 MeV and 1 GeV, respectively. By irradiating the entire periodic table, we observed similar performance between lithium (Z = 3) and cerium (Z = 58). Deficiencies observed between praseodymium (Z = 59) and einsteinium (Z = 99) have been reported, analyzed, and quantified, offering critical insights for the ongoing development of the Evaluated Nuclear Data File mode in NJOY.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2021

Evaluating MPI resource usage summary statistics

The Message Passing Interface (MPI) remains the dominant programming model for scientific applications running on today’s high-performance computing (HPC) systems. This dominance stems from MPI’s powerful semantics for inter-process communication that has enabled scientists to write applications for simulating important physical phenomena. MPI does not, however, specify how messages and synchronization should be carried out. Those details are typically dependent on low-level architecture details and the message characteristics of the application. Therefore, analyzing an application’s MPI resource usage is critical to tuning MPI’s performance on a particular platform. The result of this analysis is typically a discussion of the mean message sizes, queue search lengths and message arrival times for a workload or set of workloads. While a discussion of the arithmetic mean in MPI resource usage might be the most intuitive summary statistic, it is not always the most accurate in terms of representing the underlying data. In this paper, we analyze MPI resource usage for a number of key MPI workloads using an existing MPI trace collector and discrete-event simulator. Our analysis demonstrates that the average, while easy and efficient to calculate, is a useful metric for characterizing latency and bandwidth measurements, but may not be a good representation of application message sizes, match list search depths, or MPI inter-operation times. Additionally, we show that the median and mode are superior choices in many cases. We also observe that the arithmetic mean is not the best representation of central tendency for data that are drawn from distributions that are multi-modal or have heavy tails. Furthermore, the results and analysis of our work provide valuable guidance on how we, as a community, should discuss and analyze MPI resource usage data for scientific applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

End-to-end online performance data capture and analysis for scientific workflows

With the increased prevalence of employing workflows for scientific computing and a push towards exascale computing, it has become paramount that we are able to analyze characteristics of scientific applications to better understand their impact on the underlying infrastructure and vice-versa. Such analysis can help drive the design, development, and optimization of these next generation systems and solutions. Here, we present the architecture, integrated with existing well-established and newly developed tools, to collect online performance statistics of workflow executions from various, heterogeneous sources and publish them in a distributed database (Elasticsearch). Using this architecture, we are able to correlate online workflow performance data, with data from the underlying infrastructure, and present them in a useful and intuitive way via an online dashboard. We have validated our approach by executing two classes of real-world workflows, both under normal and anomalous conditions. The first is an I/O-intensive genome analysis workflow; the second, a CPU- and memory-intensive material science workflow. Based on the data collected in Elasticsearch, we are able to demonstrate that we can correctly identify anomalies that we injected. The resulting end-to-end data collection of workflow performance data is an important resource of training data for automated machine learning analysis.

97 MATHEMATICS AND COMPUTING↗