Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

Visualization Techniques in Space and Atmospheric Sciences

Unprecedented volumes of data will be generated by research programs that investigate the Earth as a system and the origin of the universe, which will in turn require analysis and interpretation that will lead to meaningful scientific insight. Providing a widely distributed research community with the ability to access, manipulate, analyze, and visualize these complex, multidimensional data sets depends on a wide range of computer science and technology topics. Data storage and compression, data base management, computational methods and algorithms, artificial intelligence, telecommunications, and high-resolution display are just a few of the topics addressed. A unifying theme throughout the papers with regards to advanced data handling and visualization is the need for interactivity, speed, user-friendliness, and extensibility.

Szuszczewicz, E. P.↗

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗

The Scientific Data Management Center: Available Technologies and Highlights

Managing scientific data has been identified by the scientific community as one of the most important emerging needs because of the sheer volume and increasing complexity of data being collected. Effectively generating, managing, and analyzing this information requires a comprehensive, end-to-end approach to data management that encompasses all of the stages from the initial data acquisition to the final analysis of the data. Based on community input, we have identified three significant requirements. First, more efficient access to storage systems is needed. In particular, parallel file system and I/O system improvements are needed to write and read large volumes of data without slowing a simulation. Second, scientists require technologies to facilitate better understanding of their data, in particular the ability to effectively perform complex data analysis and searches over extremely large data sets. Furthermore, exploratory analysis requires techniques for efficiently selecting subsets of the data. Third, generating the data, collecting and storing the results, keeping track of data provenance, data post-processing, and analysis of results is a tedious, fragmented process. Tools for automation of this process in a robust, tractable, and recoverable fashion are required to enhance scientific exploration.

99 GENERAL AND MISCELLANEOUS↗

Top Research Challenges and Opportunities for Near Real-Time Extreme-Scale Visualization of Scientific Data

The rapid advancement in scientific simulations and experimental facilities has resulted in the generation of vast amounts of data at unprecedented scales. The analysis and visualization of large amounts of data is a challenge in and of itself, but the requirements for timeliness significantly magnify these difficulties. Near real-time visualization is critical to monitor and analyze the data produced by these large facilities, but current production tools are not well-suited to these requirements. In this position paper, we share our perspective on some of the challenges, and thus, opportunities for research that stand in the way of near-real-time visualization of large scientific data.

Pugmire, Dave↗

M2-SCREAM: A Stratospheric Composition Reanalysis of Aura MLS Data With MERRA-2 Transport

MERRA-2 Stratospheric Composition Reanalysis of Aura Microwave Limb Sounder (M2-SCREAM) is a new reanalysis of stratospheric ozone, water vapor, hydrogen chloride (HCl), nitric acid (HNO 3 ) and nitrous oxide (N 2 O) between 2004 and the present (with a latency of several months). The assimilated fields are provided at a 50-km horizontal resolution and at a three-hourly frequency. M2-SCREAM assimilates version 4.2 Microwave Limb Sounder (MLS) profiles of the five constituents alongside total ozone column from the Ozone Monitoring Instrument. Dynamics and tropospheric water vapor are constrained by the MERRA-2 reanalysis. The assimilated species are in excellent agreement with the MLS observations, except for HNO 3 in polar night, where data are not assimilated. Comparisons against independent observations show that the reanalysis realistically captures the spatial and temporal variability of all the assimilated constituents. In particular, the standard deviations of the differences between M2-SCREAM and constituent mixing ratio data from The Atmospheric Chemistry Experiment Fourier Transform Spectrometer are much smaller than the standard deviations of the measured constituents. Evaluation of the reanalysis against aircraft data and balloon-borne frost point hygrometers indicates a faithful representation of small-scale structures in the assimilated water vapor, HNO 3 and ozone fields near the tropopause. Comparisons with independent observations and a process-based analysis of the consistency of the assimilated constituent fields with the MERRA-2 dynamics and with large-scale stratospheric processes demonstrate the utility of M2-SCREAM for scientific studies of chemical and transport variability on time scales ranging from hours to decades. Analysis uncertainties and guidelines for data usage are provided.

MERRA-2↗

Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system

A case study on climate models intercomparison data analysis addressing several classes of multi-model experiments is being implemented in the context of the EU H2020 INDIGO-DataCloud project. Such experiments require the availability of large amount of data (multi-terabyte order) related to the output of several climate models simulations as well as the exploitation of scientific data management tools for large-scale data analytics. More specifically, the paper discusses in detail a use case on precipitation trend analysis in terms of requirements, architectural design solution, and infrastructural implementation. The experiment has been tested and validated on CMIP5 datasets, in the context of a large scale distributed testbed across EU and US involving three ESGF sites (LLNL, ORNL, and CMCC) and one central orchestrator site (PSNC).

Fiore, S.↗

Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM). Final Report

Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM) is a proposed system that will enable data from an infinite number of diverse sources to be organized and accessed from anywhere using any handheld or other computer device. The approach offers a powerful roadmap for the creation and integration of a unified knowledge base of an entire ecosystem, including its many geophysical, geographical, social, political, agricultural, energy, transportation, and cyber aspects. The resulting aggregation of data has the potential to generate an informational universe of unprecedented size that has never before been possible due to the prohibitive costs, managerial complexity, and technical barriers associated with ever-changing exponential-growth data flows. We envision that DREAM will accelerate discovery by enabling climate researchers, among other types of researchers, to manage, analyze, and visualize data from earth-scale measurements and simulations. DREAM’s success will be built on proven components that leverage existing services and resources. A key building block for DREAM will be the ESGF, chaired by Dean N. Williams. Expanding on the existing ESGF, the project will ensure that the access, storage, movement, and analysis of the large quantities of data that are processed and produced by diverse science projects can be dynamically distributed with proper resource management. Much of the Office of Science data is currently generated by multiple stand-alone facilities. DREAM can collect data accumulated from these facilities and incorporate it into a fully integrated network accessible from anywhere in the world. The result is a completely new paradigm shift for data management, analysis, and visualization enabling researchers to: Manage their calculations, data, tools, and research results; Ensure that all data are sharable, reproducible and (re)usable—accompanied by appropriate metadata describing its provenance, syntax, and semantics at creation; Advance application performance by selectively adapting APIs and services in response to scientific requirements and architectural complexities; and Provide scalable interactive resource management—navigate data and metadata at multiple levels, provide architecture-aware data integration, analysis and visualization tools. We will engage closely with DOE, NASA, and NOAA science groups working at the leading edge of computing. These engagements—in domains such as biology, climate, and hydrology—will allow us to advance disciplinary science goals and inform our development of technologies that can accelerate discovery across DOE more broadly. We will advertise and promote our technologies via dedicated workshops, tutorials, and sessions at conferences, stand-alone events with broad inter-disciplinary invitation, and engagements with leadership facilities.

54 ENVIRONMENTAL SCIENCES↗

Big Data, Big Solutions

Data—lots of data—generated in seconds and piling up on the internet, streaming and stored in countless databases. Big data is important for commerce, society and our nation’s security. Yet the volume, velocity, variety and veracity of data is simply too great for any single analyst to make sense of alone. It requires advanced, data-intensive computing. Simply put, data-intensive computing is the use of sophisticated computers to sort through mounds of information and present analysts with solutions in the form of graphics, scenarios, formulas, new hypotheses and more. This scientific capability is foundational to PNNL’s energy, environment and security missions. Senior Scientist and Division Director Bill Pike and his team are developing analytic tools that are used to solve important national challenges, including cyber systems defense, power grid control systems, intelligence analysis, climate change and scientific exploration.

97 MATHEMATICS AND COMPUTING↗

Lunar Science for Landed Missions Sites Visualized with NASA's Moon Trek Portal

The Lunar Science for Landed Missions Workshop, held in January 2018 at NASA Ames, asked the important question, “Where should we explore next on the Moon?” Representatives from NASA, other international space agencies, commercial exploration companies, and the international lunar science and exploration communities gathered to present and debate the scientific and exploration relevance of a range of potential future lunar landing sites. From this workshop, a list of representative high-value sites was reported. In this presentation, we will present the sites and summarize their individual merits as described in the workshop report. We will augment this with detailed visualizations of each of the sites through the capabilities of NASA’s Moon Trek (https://trek.nasa.gov/moon) data visualization and analysis portal with the goal of highlighting each site’s outstanding geomorphological features along with of the scientific and exploration contexts of each site.

Day, Brian↗

LBNL DDoS Detection on Science Networks v0.1

This software contains a set of analysis tools to identify certain types of DDoS attacks and disambiguate those attacks from large scientific data transfers over networks as well as ordinary Internet network traffic. It also contains a piece of "driver" code that allows individual scripts that detect certain types of attacks to selectively be run.

Peisert, SeanP.↗

Western Regional Center of the National Institute for Climatic Change Research

The major goal of this project was fostering, integrating, synthesizing, and disseminating experimental, observational, and modeling research on predicted climate change in the western region of the U.S. and the impacts of that change on the structure, productivity, and climatic interactions of the region's natural and managed ecological systems. This was accomplished through administering a competitive grants program developed in collaboration with the other four regional centers of the NICCR. The activities supported included efforts to synthesize research on climate change in the western U.S. through meta-analysis studies, model comparisons, and data synthesis workshops. Results from this work were disseminated to the scientific and public media. This project also supported the development of the NICCR web site, hosted at NAU, which was used as the means to accept pre-proposal and proposal submissions for each funding cycle, and served as a clearing house for public outreach for results from NICCR-funded research

54 ENVIRONMENTAL SCIENCES↗

Assessment of Proton Deflectometry for Exploding Wire Experiments

This project provides the first demonstration of the application of proton deflectometry for the diagnosis of electromagnetic field topology and current-carrying regions in Z-pinch plasma experiments. Over the course of this project several milestones were achieved. High-energy proton beam generation was demonstrated on the short-pulse high-intensity Leopard laser, (10 Joules in ~350 femtoseconds, and the proton beam generation was shown to be reproducible. Next, protons were used to probe the electromagnetic field structure of short circuit loads in order to benchmark the two numerical codes, the resistive-magnetohydrodynamics (MHD) code, Gorgon, and the hybrid particle-in-cell code, LSP for the interpretation of results. Lastly, the proton deflectometry technique was used to map the magnetic field structure of pulsed-power-driven plasma loads including wires and supersonic jets formed with metallic foils. Good agreement between the modeling and experiments has been obtained. The demonstrated technique holds great promise to significantly improve the understanding of current flow and electromagnetic field topology in pulsed power driven high energy density plasmas. Proton probing with a high intensity laser was for the first time implemented in the presence of the harsh debris and x-ray producing z-pinch environment driven by a mega-ampere-scale pulsed-power machine. The intellectual merit of the program was that it investigated strongly driven MHD systems and the influence of magnetic field topology on plasma evolution in pulsed power driven plasmas. The experimental program involved intense field-matter interaction in the generation of the proton probe, as well as the generation of plasma subjected to 1 MegaGauss scale magnetic fields. The computational aspect included two well-documented codes, in combination for the first time to provide accurate interpretation of the experimental results. The broader impact included the support of 2 graduate students, one at UCSD and one at NTF, who were exposed to both the experimental physics work, the MHD and PIC modeling of the system. A first generation college undergraduate student was employed to assist in experiments and data analysis throughout the project. Data resulting from the research program were broadly disseminated by publication in scientific journals, and presentation at international and national conferences and workshops.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Nuclear Physics of Compact Binaries with GADGET

The research goal of this project was to experimentally constrain the thermonuclear rate of the 15 O(α, Υ) 19 Ne reaction, whose uncertainty has a strong influence on the modeling of astrophysical Type-I X-ray bursts on accreting neutron stars. To this end, Experiment 21072 was run in November 2022 at the Facility for Rare Isotope Beams using the GADGET II detection system. The limited scope and duration of this award included work in the year following the experiment: drafting a technical paper on the time projection chamber of the GADGET II detection system, and analysis of data from Experiment 21072. The award contributed to the education and development of the scientific and technical workforce by partially supporting one postdoctoral Research Associate who did this work.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Scalable Performance Measurement and Analysis

Concurrency levels in large-scale, distributed-memory supercomputers are rising exponentially. Modern machines may contain 100,000 or more microprocessor cores, and the largest of these, IBM's Blue Gene/L, contains over 200,000 cores. Future systems are expected to support millions of concurrent tasks. In this dissertation, we focus on efficient techniques for measuring and analyzing the performance of applications running on very large parallel machines. Tuning the performance of large-scale applications can be a subtle and time-consuming task because application developers must measure and interpret data from many independent processes. While the volume of the raw data scales linearly with the number of tasks in the running system, the number of tasks is growing exponentially, and data for even small systems quickly becomes unmanageable. Transporting performance data from so many processes over a network can perturb application performance and make measurements inaccurate, and storing such data would require a prohibitive amount of space. Moreover, even if it were stored, analyzing the data would be extremely time-consuming. In this dissertation, we present novel methods for reducing performance data volume. The first draws on multi-scale wavelet techniques from signal processing to compress systemwide, time-varying load-balance data. The second uses statistical sampling to select a small subset of running processes to generate low-volume traces. A third approach combines sampling and wavelet compression to stratify performance data adaptively at run-time and to reduce further the cost of sampled tracing. We have integrated these approaches into Libra, a toolset for scalable load-balance analysis. We present Libra and show how it can be used to analyze data from large scientific applications scalably.

99 GENERAL AND MISCELLANEOUS↗

MODFLOW6 models used to evaluate potential stresses and hydrologic conditions driving water-level fluctuations in well ER-5-3-2, Frenchman Flat, Southern Nevada

The hydrograph for well ER-5-3-2 in Frenchman Flat, southern Nevada, has previously unexplained water-level fluctuations. Four, three-dimensional, groundwater models (MODFLOW 6) were developed to evaluate potential stresses and hydrologic conditions affecting the well ER-5-3-2 hydrograph. Four model scenarios were developed that simulated: (1) wellbore leakage without recharge, (2) wellbore leakage with recharge, (3) shallow (low transmissivity) and deep (high transmissivity) carbonate rocks, and (4) lateral heterogeneity of carbonate rocks. Input and output files for the four model scenarios are in the model and output directories, respectively. Hydraulic conductivity, specific storage, and wellbore-leakage rates (when simulated) were estimated with parameter estimation (PEST) by minimizing a weighted composite, sum-of-squares objective function. The objective function was informed by measurement and Tikhonov regularization observations. Measurement observations included drawdowns from the constant-rate aquifer test and water-level altitudes measured in well ER-5-3-2 from 2001-2021. Tikhonov regularization informed hydraulic conductivity and specific storage parameters that were insensitive to measurement observations, where homogeneity was the preferred relation. Batch files, executables, and MODFLOW 6, PEST, and post-processing utilities are in the ancillary directory. Supplementary data also are included in the ancillary directory, including site information, high-frequency water-level and aquifer-test data, transmissivity estimates, water-chemistry data, and water-temperature analyses. This USGS data release contains data, analyses, and model files for the simulations and analysis results described in U.S. Geological Survey Scientific Investigations Report (https://doi.org/10.3133/sir20225132).

54 ENVIRONMENTAL SCIENCES↗

Attitude and translation control of a low-altitude Gravsat

Dynamic simulation of the Gravsat's attitude and translation control system is used to provide an upper bound for the fuel supply and test the feasibility of the preliminary design. A preliminary design is made for the disturbance compensation system (DISCOS) sensor, the thruster control laws, reaction wheel control laws, and the onboard state estimators. The sensor analysis and noise measurements show no problems in scaling the Triad navigation satellite sensor design up to meet the Gravsat requirements, except for proof mass center-of-mass offset. A promising technique is proposed to measure and eliminate this error. The covariance analysis confirms that a sophisticated post-flight data fit will be necessary to reconstruct a scientifically useful proof mass state. The DISCOS sensor will have to be continuously calibrated from the inflight data to achieve this reconstruction.

Ray, J. C.↗

The handling of data in solar-terrestrial and in planetary physics

The evolution of data handling systems for solar-terrestrial and planetary physics applications is discussed. The main elements of data management systems for scientific satellites are described, including: command and control; data transmission; prioritization of data users; and data reduction and analysis. The design approaches incorporated into the data management systems of the Interplanetary Monitoring Platform (IMP), International Sun Earth Explorer (ISEE), and the Orbiting Geophysical Observatory, (OGO) spacecraft, are discussed. Emphasis is given to classical, centralized, and hybrid system designs. The report of the Committee on Data Management and Computation (CODMAC) concerning concepts for new data management systems is briefly summarized.

Vette, J. I.↗

NASA/MSFC FY92 Earth Science and Applications Program Research Review

A large amount of attention has recently been given to global issues such as the ozone hole, tropospheric temperature variability, etc. A scientific challenge is to better understand atmospheric processes on a variety of spatial and temporal scales in order to predict environmental changes. Measurement of geophysical parameters such as wind, temperature, and moisture are needed to validate theories, provide analyzed data sets, and initialize or constrain numerical models. One of NASA's initiatives is the Mission to Planet Earth Program comprised of an Earth Observation System (EOS) and the scientific strategy to analyze these data. This work describes these efforts in the context of satellite data analysis and fundamental studies of atmospheric dynamics which examine selected processes important to the global circulation.

Arnold, James E.↗