Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

2012 Joint Research Target (JRT) Final Report

The report summarizes: [1] Diagnostic upgrades and analysis improvements that support the JRT; [2] Descriptions of new experiments and a brief summary of findings; [3] New analysis of previous experiments; [4] Plans for further data analysis and recommendations for further work; [5] Scientific publications that are derived from or contributed directly to the JRT. The report is organized as follows: Section 1: Describes inter-machine comparisons that were facilitated by the JRT focus. Similarities and differences found on the different facilities are outlined, and plans for additional analysis of data and supporting simulations are described.Section 2: Reviews diagnostic development, experiments and results from C-Mod. C-Mod dedicated 13.1 run days to the JRT divided among 10 experimental proposals. These were organized into three general thrust areas that represent distinct experimental approaches to realizing the regimes called for in the JRT description. Initial comparisons with linear and nonlinear simulation have been carried out for these experiments, and plans for an extensive campaign of analysis have been outlined. Section 3: Reviews results from DIII-D including new experiments, detailed comparisons of simulations to previously collected data and connections between the new work and past research. Four experimental days were dedicated to the JRT in 2012. These included studies of L-modes, H-modes and QH-modes. Section 4: Summarizes new analysis of data collected before the NSTX shutdown. The work focuses on the roles of low and high-k turbulence; collisionality scans and impurity particle transport.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Environmental analysis using integrated GIS and remotely sensed data - Some research needs and priorities

This paper discusses some basic scientific issues and research needs in the joint processing of remotely sensed and GIS data for environmental analysis. Two general topics are treated in detail: (1) scale dependence of geographic data and the analysis of multiscale remotely sensed and GIS data, and (2) data transformations and information flow during data processing. The discussion of scale dependence focuses on the theory and applications of spatial autocorrelation, geostatistics, and fractals for characterizing and modeling spatial variation. Data transformations during processing are described within the larger framework of geographical analysis, encompassing sampling, cartography, remote sensing, and GIS. Development of better user interfaces between image processing, GIS, database management, and statistical software is needed to expedite research on these and other impediments to integrated analysis of remotely sensed and GIS data.

Davis, Frank W.↗

Analysis of data from satellite experiments

Identical ultraviolet photometer experiments were carried on the eccentric orbiting OGO-5 spacecraft and on the polar orbiting OGO-6 spacecraft. The experiments operated flawlessly and, due to sophisticated electronic design, allowed signals to be processed accurately over a large dynamic range. Scientific results gained from the post-mission data analysis efforts are summarized.

Thomas, G. E.↗

Multi-resolution enhancement for full-spectrum neural representations

Scientific data acquisition continues to outpace storage and analysis capabilities, making voxel-basedrepresentations increasingly intractable. Implicit neural representations (INRs) offer a promising solutionby encoding signals through coordinate-based neural networks, serving as surrogates of data, withcomputational and storage requirements scaling with network complexity rather than data dimensionality.However, smaller INRs struggle to faithfully represent multiscale structures, high-frequency informationand fine textures that constitute a large proportion of scientific measurements. We propose WIEN-INR, atheoretically guided hierarchical INR framework that distributes modelling across resolution scales andenables improved representation capacity through a novel enhancement network to recover subtle details.This multiscale architecture allows smaller networks to retain the full spatial-frequency content of thesignal as well as preserve training efficiency and lower storage cost. Evaluated on distinct raw experimentalmeasurements across scales and complexities, WIEN-INR represents a practical step towards a broaderadoption of neural representations in scientific workflows, delivering compact, robust and high-fidelityrepresentations.

Ni, Yuan [SLAC National Accelerator Laboratory (SL↗

Enabling discovery data science through cross-facility workflows

Experimental and observational instruments for scientific research (such as light sources, genome sequencers, accelerators, telescopes and electron microscopes) increasingly require High Performance Computing (HPC) scale capabilities for data analysis and workflow processing. Next-generation instruments are being deployed with higher resolutions and faster data capture rates, creating a big data crunch that cannot be handled by modest institutional computing resources. Often these big data analysis pipelines also require near real-time computing and have higher resilience requirements than the simulation and modeling workloads more traditionally seen at HPC centers. While some facilities have enabled workflows to run at a single HPC facility, there is a growing need to integrate capabilities across HPC facilities to enable cross-facility workflows, either to provide resilience to an experiment, increase analysis throughput capabilities, or to better match a workflow to a particular architecture. In this paper we describe the barriers to executing complex data analysis workflows across HPC facilities and propose an architectural design pattern for enabling scientific discovery using cross-facility workflows that includes orchestration services, application programming interfaces (APIs), data access and co-scheduling.

Antypas, Katerina B.↗

Data Analysis and Theoretical Studies of the Upper Mesosphere and Lower Thermosphere

Three separate tasks were proposed under this award. The first involved extending our continuing study of electrodynamical feedback between the thermosphere/ionosphere and the magnetosphere. The second was a model-experiment comparison study of global dynamics and the third was a 'spectral energetics' analysis of tidal dissipation and energy exchange mechanisms. The Earth's mesosphere and lower-thermosphere/ionosphere (MLTI), between approximately 60 and 180 km altitude, is the most poorly understood region of the Earth's atmosphere, primarily because of its relative inaccessibility. This lack of knowledge has been widely recognized and has provided important scientific rationale for the upcoming NASA TIMED mission. While the data gathered during the TIMED era will revolutionize our understanding of the MLTI region, much work can be done prior to the mission, both to develop data-analysis and modeling techniques and to study the more limited relevant experimental data from previous missions. The grant reported on here continues and extends an existing successful program of scientific research into the energetics, dynamics and electrodynamics of the MLTI, using available theoretical and data analysis tools.

Burns, Alan↗

Scientific Data Management Center for Enabling Technologies

Managing scientific data has been identified by the scientific community as one of the most important emerging needs because of the sheer volume and increasing complexity of data being collected. Effectively generating, managing, and analyzing this information requires a comprehensive, end-to-end approach to data management that encompasses all of the stages from the initial data acquisition to the final analysis of the data. Fortunately, the data management problems encountered by most scientific domains are common enough to be addressed through shared technology solutions. Based on community input, we have identified three significant requirements. First, more efficient access to storage systems is needed. In particular, parallel file system and I/O system improvements are needed to write and read large volumes of data without slowing a simulation, analysis, or visualization engine.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Science Autonomy in Robotic Exploration

Historical mission operations have involved: (1) return of scientific data; (2) evaluation of these data by scientists; (3) recommendations for future mission activity by scientists; (4) commands for these transmitted to the craft; and (5) the activity being undertaken. This cycle is repeated throughout the mission with command opportunities once or twice per day. For a rover, this historical cycle is not amenable to rapid long range traverses or rapid response to any novel or unexpected situations. In addition to real-time response issues, imaging and/or spectroscopic devices can produce tremendous data volumes during a traverse. However, such data volumes can rapidly exceed on-board memory capabilities prior to the ability to transmit it to Earth. Additionally, the necessary communication band-widths are restrictive enough so that only a small portion of these data can actually be returned to Earth. Such scenarios suggest enabling some science decisions to be made on-board the robots. These decisions involve automating various aspects of scientific discovery instead of the electromechanical control, health, and navigation issues associated with robotic operations. The robot retains access to the full data fidelity obtained by its scientific sensors, and is in the best position to implement actions based upon these data. Such an approach would eventually enable the robot to alter observations and assure only the highest quality data is obtained for analysis. Additionally, the robot can begin to understand what is scientifically interesting and implement alternative observing sequences, because the observed data deviate from expectations based upon current theories/models of planetary processes. Such interesting data and/or conclusions can then be prioritized and selectively transmitted to Earth; reducing memory and communications demands. Results of Ames' current work in this area will be presented.

Roush, Ted L.↗

Robotic Exploration: The Role of Science Autonomy

Historical mission operations have involved: (1) return of scientific data; (2) evaluation of these data by scientists; (3) recommendations for future mission activity by scientists; (4) commands for these transmitted to the craft; and (5) the activity being, undertaken. This cycle is repeated throughout the mission with command opportunities once or twice per day. For a rover, this historical cycle is not amenable to rapid long range traverses or rapid response to any novel or unexpected situations. In addition to real-time response issues, imaging and/or spectroscopic devices can produce tremendous data volumes during a traverse. However, such data volumes can rapidly exceed on-board memory capabilities prior to the ability to transmit it to Earth. Additionally, the necessary communication band-widths are restrictive enough so that only a small portion of these data can actually be returned to Earth. Such scenarios suggest enabling some science decisions to be made on-board the robots. These decisions involve automating various aspects of scientific discovery instead of the electromechanical control, health, and navigation issues associated with robotic operations. The robot retains access to the full data fidelity obtained by its scientific sensors, and is in the best position to implement actions based upon these data. Such an approach would eventually enable the robot to alter observations and assure only the highest quality data is obtained for analysis. Additionally, the robot can begin to understand what is scientifically interesting and implement alternative observing sequences, because the observed data deviate from expectations based upon current theories/models of planetary processes. Such interesting data and/or conclusions can then be prioritized and selectively transmitted to Earth; reducing memory and communications demands. Results of Ames' current work in this area will be presented.

Roush, Ted L.↗

Heterogeneous Visualization: A Call for Empirically Driven Technology Advancement and Adoption; Preprint

Our ability to generate data from simulations and experimental platforms is generally outpacing our ability to analyze those data efficiently and effectively. There are a growing number and variety of possible visualization display modalities, from commodity head-mounted displays to large-scale immersive environments, from desktop and touch displays to high-resolution display walls. Many of these visualization technologies and visual environments are rapidly evolving and could revolutionize analysis for certain classes of complex scientific and engineering data. However, the choice of what technology or combination of technologies to use has historically been largely ad hoc and lacks a carefully validated understanding of how to best utilize different visualization technologies to advance the science. To maximize the impact of our data and analyses, it is imperative that we use the most effective visualization environments for the analysis task at hand.

heterogeneous visualization↗

PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Here we report that computational science is well established as the third pillar of scientific discovery and is on par with experimentation and theory. However, as we move closer toward the ability to execute exascale calculations and process the ensuing extreme-scale amounts of data produced by both experiments and computations alike, the complexity of managing the compute and data analysis tasks has grown beyond the capabilities of domain scientists. Therefore, workflow management systems are absolutely necessary to ensure current and future scientific discoveries. A key research question for these workflow management systems concerns the performance optimization of complex calculation and data analysis tasks. The central contribution of this article is a description of the PANORAMA approach for modeling and diagnosing the run-time performance of complex scientific workflows. This approach integrates extreme-scale systems testbed experimentation, structured analytical modeling, and parallel systems simulation into a comprehensive workflow framework called Pegasus for understanding and improving the overall performance of complex scientific workflows.

97 MATHEMATICS AND COMPUTING↗

Characterizing Space Environments with Long-Term Space Plasma Archive Resources

A significant scientific benefit of establishing and maintaining long-term space plasma data archives is the ready access the archives afford to resources required for characterizing spacecraft design environments. Space systems must be capable of operating in the mean environments driven by climatology as well as the extremes that occur during individual space weather events. Long- term time series are necessary to obtain quantitative information on environment variability and extremes that characterize the mean and worst case environments that may be encountered during a mission. In addition, analysis of large data sets are important to scientific studies of flux limiting processes that provide a basis for establishing upper limits to environment specifications used in radiation or charging analyses. We present applications using data from existing archives and highlight their contributions to space environment models developed at Marshall Space Flight Center including the Chandra Radiation Model, ionospheric plasma variability models, and plasma models of the L2 space environment.

Minow, Joseph I.↗

FIVE-YEAR WILKINSON MICROWAVE ANISOTROPY PROBE OBSERVATIONS: DATA PROCESSING, SKY MAPS, AND BASIC RESULTS

We present new full-sky temperature and polarization maps in five frequency bands from 23 to 94 GHz, based on data from the first five years of the Wilkinson Microwave Anisotropy Probe (WMAP) sky survey. The new maps are consistent with previous maps and are more sensitive. The five-year maps incorporate several improvements in data processing made possible by the additional years of data and by a more complete analysis of the instrument calibration and in-flight beam response. We present several new tests for systematic errors in the polarization data and conclude that W-band polarization data is not yet suitable for cosmological studies, but we suggest directions for further study. We do find that Ka-band data is suitable for use; in conjunction with the additional years of data, the addition of Ka band to the previously used Q- and V-band channels significantly reduces the uncertainty in the optical depth parameter, {tau}. Further scientific results from the five-year data analysis are presented in six companion papers and are summarized in Section 7 of this paper. With the five-year WMAP data, we detect no convincing deviations from the minimal six-parameter {lambda}CDM model: a flat universe dominated by a cosmological constant, with adiabatic and nearly scale-invariant Gaussian fluctuations. Using WMAP data combined with measurements of Type Ia supernovae and Baryon Acoustic Oscillations in the galaxy distribution, we find (68% CL uncertainties): {omega} {sub b} h {sup 2} = 0.02267{sup +0.00058} {sub -0.00059}, {omega} {sub c} h {sup 2} = 0.1131 {+-} 0.0034, {omega}{sub {lambda}} = 0.726 {+-} 0.015, n{sub s} = 0.960 {+-} 0.013, {tau} = 0.084 {+-} 0.016, and {delta}{sub R}{sup 2} = (2.445{+-}0.096)x10{sup -9} at k = 0.002 Mpc{sup -1}. From these we derive {sigma}{sub 8} = 0.812 {+-} 0.026, H {sub 0} = 70.5 {+-} 1.3 km s{sup -1} Mpc{sup -1}, {omega} {sub b} = 0.0456 {+-} 0.0015, {omega} {sub c} = 0.228 {+-} 0.013, {omega} {sub m} h {sup 2} = 0.1358{sup +0.0037} {sub -0.0036}, z {sub reion} = 10.9 {+-} 1.4, and t {sub 0} = 13.72 {+-} 0.12 Gyr. The new limit on the tensor-to-scalar ratio is r < 0.22(95%CL), while the evidence for a running spectral index is insignificant, dn{sub s} /dln k = -0.028 {+-} 0.020 (68% CL). We obtain tight, simultaneous limits on the (constant) dark energy equation of state and the spatial curvature of the universe: -0.14 < 1 + w < 0.12(95%CL) and -0.0179 < {omega} {sub k} < 0.0081(95%CL). The number of relativistic degrees of freedom, expressed in units of the effective number of neutrino species, is found to be N {sub eff} = 4.4 {+-} 1.5 (68% CL), consistent with the standard value of 3.04. Models with N {sub eff} = 0 are disfavored at >99.5% confidence. Finally, new limits on physically motivated primordial non-Gaussianity parameters are -9 < f {sup local} {sub NL} < 111 (95% CL) and -151 < f {sup equil} {sub NL} < 253 (95% CL) for the local and equilateral models, respe0013ctive.

79 ASTRONOMY AND ASTROPHYSICS↗

Five-Year Wilkinson Microwave Anisotropy Probe Observations: Data Processing, Sky Maps, and Basic Results

We present new full-sky temperature and polarization maps in five frequency bands from 23 to 94 GHz, based on data from the first five years of the Wilkinson Microwave Anisotropy Probe (WMAP) sky survey. The new maps are consistent with previous maps and are more sensitive. The five-year maps incorporate several improvements in data processing made possible by the additional years of data and by a more complete analysis of the instrument calibration and in-flight beam response. We present several new tests for systematic errors in the polarization data and conclude that W-band polarization data is not yet suitable for cosmological studies, but we suggest directions for further study. We do find that Ka-band data is suitable for use; in conjunction with the additional years of data, the addition of Ka band to the previously used Q- and V-band channels significantly reduces the uncertainty in the optical depth parameter, tau. Further scientific results from the five-year data analysis are presented in six companion papers and are summarized in Section 7 of this paper. With the five-year WMAP data, we detect no convincing deviations from the minimal six-parameter ACDM model: a flat universe dominated by a cosmological constant, with adiabatic and nearly scale-invariant Gaussian fluctuations. Using WMAP data combined with measurements of Type Ia supernovae and Baryon Acoustic Oscillations in the galaxy distribution, we find (68% CL uncertainties): OMEGA(sub b)h(sup 2) = 0.02267(sup +0.00058)(sub -0.00059), OMEGA(sub c)h(sup 2) = 0.1131 plus or minus 0.0034, OMEGA(sub logical and) = 0.726 plus or minus 0.015, ns = .960 plus or minus 0.013, tau = 0.84 plus or minus 0.016, and DELTA(sup 2)(sub R) = (22.445 plus or minus 0.096) x 10(exp -9) at k = 0.002 Mpc(exp -1). From these we derive sigma(sub 8) = 0.812 plus or minus 0.026, H(sub 0) = 70.5 plus or minus 1.3 kilometers per second Mpc(exp -1), OMEGA(sub b) = 0.0456 plus or minus 0.0015, OMEGA(sub c) = .228 plus or minus 0.013, OMEGA(sub m)h(sup 2) = 0.1358(sup +0.0037)(sub -0.0036), z reion = 10.9 plus or minus 1.4, and t(sub 0) = 13.72 plus or minus 0.12 Gyr. The new limit on the tensor-to-scalar ration is r less than 0.22 (95% CL), while the evidence for a running spectral index is insignificant, dn(sub s)/d ln k = -0.028 plus or minus 0.020 (68% CL). We obtain tight, simultaneous limits on the (constant) dark energy equation of state and the spatial curvature of the universe: -0.14 less than 1 + w less than 0.12 (95% CL) and -0.0179 less than OMEGA(sub k) less than 0.0081 (95% CL). The number of relativistic degrees of freedom, expressed in units of the effective number of neutrino species, is found to be N(sub eff) = 4.4 plus or minus 1.5 (69% CL), consistent with the standard value of 3.04. Models with N(sub eff) = 0 are disfavored at greater than 99% confidence. Finally, new limits on physically motivated primordial non-Gaussianity parameters are -9 less than f(sup local)(sub NL) less than 111 (95% CL) and -151 less than f(sup equal)(sub NL) less than 253 (95% CL) for the local and equilateral models, respectively.

Hinshaw, G.↗

Coordinative Entities: Forms of Organizing in Data Intensive Science

Scientific collaboration is a long-standing subject of CSCW scholarship that typically focuses on the development and use of computing systems to facilitate research. The research presented in this article investigates the sociality of science by identifying and describing particular, common forms of organizing that researchers in four different scientific realms employ to conduct work in both local contexts and as part of distributed, global projects. This paper introduces five prototypical forms of organizing we categorize as coordinative entities: the Principal Group, Intermittent Exchange, Sustained Aggregation, Federation, and Facility Organization. Coordinative entities as a categorization help specify, articulate, compare, and trace overlapping and evolving arrangements scientists use to facilitate data intensive research. We use this typology to unpack complexities of data intensive scientific collaboration in four cases, showing how scientists invoke different coordinative entities across three types of research activities: data collection, processing, and analysis. Finally, our contribution scrutinizes the sociality of scientific work to illustrate how these actors engage in relational work within and among diverse, dispersed forms of organizing across project, funding, and disciplinary boundaries.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Enabling Command-and-Control in Advanced In Situ Workflows

Scientific discovery is progressing towards autonomous science with the combination of scientific instruments, high-performance computing, and artificial intelligence in complex workflows. This evolution introduces new requirements for managing scientific workflows, including feedback loops, near real-time constraints, and the ability to dynamically control workflow execution. In situ workflows that analyze and visualize data as it is generated are well-suited to satisfy stringent time constraints and their iterative nature offers greater opportunities for command-and-control. However, only a few of the many workflow management systems available have been specifically designed to manage in situ workflows and often lack support for automated feedback loops that allow analysis and visualization components to interact with the main scientific data producer. To address this need, we present in this paper how to add command-and-control capabilities to a workflow management system. We identify the functional design requirements of such a command-and-control system, detail its architecture, interface, and core mechanisms, and illustrate how advanced in situ workflows can leverage command-and-control in three use cases: graceful termination with checkpoint, dynamic and adaptive data reduction, and event-triggered analysis.

Mehta, Kshitij [ORNL] (ORCID:0000000297149981)↗

Finding Tropical Cyclones on a Cloud Computing Cluster: Using Parallel Virtualization for Large-Scale Climate Simulation Analysis

Extensive computing power has been used to tackle issues such as climate changes, fusion energy, and other pressing scientific challenges. These computations produce a tremendous amount of data; however, many of the data analysis programs currently only run a single processor. In this work, we explore the possibility of using the emerging cloud computing platform to parallelize such sequential data analysis tasks. As a proof of concept, we wrap a program for analyzing trends of tropical cyclones in a set of virtual machines (VMs). This approach allows the user to keep their familiar data analysis environment in the VMs, while we provide the coordination and data transfer services to ensure the necessary input and output are directed to the desired locations. This work extensively exercises the networking capability of the cloud computing systems and has revealed a number of weaknesses in the current cloud system software. In our tests, we are able to scale the parallel data analysis job to a modest number of VMs and achieve a speedup that is comparable to running the same analysis task using MPI. However, compared to MPI based parallelization, the cloud-based approach has a number of advantages. The cloud-based approach is more flexible because the VMs can capture arbitrary software dependencies without requiring the user to rewrite their programs. The cloud-based approach is also more resilient to failure; as long as a single VM is running, it can make progress while as soon as one MPI node fails the whole analysis job fails. In short, this initial work demonstrates that a cloud computing system is a viable platform for distributed scientific data analyses traditionally conducted on dedicated supercomputing systems.

97 MATHEMATICS AND COMPUTING↗

ECP Software Technology Capability Assessment Report

The Exascale Computing Project Software Technology (ECP ST) focus area represents the key bridge between Exascale systems and the scientists developing applications that will run on those platforms. ECP offers a unique opportunity to build a coherent set of software (often referred to as the "software stack") that will allow application developers to maximize their ability to write highly parallel applications, targeting multiple Exascale architectures with runtime environments that will provide high performance and resilience. But applications are only useful if they can provide scientific insight, and the unprecedented data produced by these applications require a complete analysis work ow that includes new technology to scalably collect, reduce, organize, curate, and analyze the data into actionable decisions. This requires approaching scientific computing in a holistic manner, encompassing the entire user workflow - from conception of a problem, setting up the problem with validated inputs, performing high-fidelity simulations, to the application of uncertainty quantification to the final analysis. The software stack plan defined here aims to address all of these needs by extending current technologies to Exascale where possible, by performing the research required to conceive of new approaches necessary to address unique problems where current approaches will not suffice, and by deploying high-quality and robust software products on the platforms developed in the Exascale systems project. The ECP ST portfolio has established a set of interdependent projects that will allow for the research, development, and delivery of a comprehensive software stack,

97 MATHEMATICS AND COMPUTING↗