Research directed toward defining the variability of the atmosphere between 30 and 200 kilometers Interim scientific report
Reduction and analysis of sounding rocket data for models of upper atmosphere
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Reduction and analysis of sounding rocket data for models of upper atmosphere
The Compton Observatory Science Workshop was held in Annapolis, Maryland on September 23-25, 1991. The primary purpose of the workshop was to provide a forum for the exchange of ideas and information among scientists with interests in various areas of high energy astrophysics, with emphasis on the scientific capabilities of the Compton Observatory. Early scientific results, as well as reports on in-flight instrument performance and calibrations are presented. Guest investigator data products, analysis techniques, and associated software were discussed. Scientific topics covered included active galaxies, cosmic gamma ray bursts, solar physics, pulsars, novae, supernovae, galactic binary sources, and diffuse galactic and extragalactic emission.
The main goal (long term) of this work is to perform computational dynamics analysis and quantify uncertainty from vector fields computed directly from measured data. Global analysis based on observed spatiotemporal evolution is performed by objective function based on expected physics and informed scientific priors, variational optimization to compute vector fields from measured data, and transport analysis proceeding with observations and priors. A mathematical formulation for computing flow fields is set up for computing the minimizer for the problem. An application to oceanic flow based on sea surface temperature is presented.
The primary research objective is to initiate long-term monitoring of stratospheric ozone with a ground-based millimeter-wave spectrometer, the first of several such instruments projected to be part of the Network for the Detection of Stratospheric Change. The ultimate goal of this monitoring is twofold. First, to detect any secular trend in stratospheric ozone abundance, whether of natural or anthropogenic origin and, second, to provide ground-truth validation for existing and future satellite measurements of ozone. With this goal in mind, a more immediate objective is to validate the millimeter-wave measurements by tests of the instrument, internal consistency tests on the data, and most importantly, by intercomparison with all other available ozone measurements. The validation process is expected to lead to refinements in the instrument and its operating procedures and in the data analysis. The final objective is to perform short-term scientific studies with the data, including studies of the ozone diurnal and seasonal variations, and comparison of ozone variations with changes in other geophysical parameters, notably temperature and water vapor. Routine observations are now ongoing; these will allow continuing intercomparisons with the Stratospheric Aerosol and Gas Experiment (SAGE II) and one of the lidars, which is permanently on site. The experience gained during the Stratospheric Ozone Intercomparison (STOIC) caused us to refine our calibration procedures and identify the need for internal shielding of the millimeter receiver from radio frequency interference. Installation of this shielding is planned for the near future and should allow improvements in the instrument calibration and a higher signal-to-noise ratio, both of which will result in improved measurement precision.
The Data group provides data analysis and visualization support to its customers. This consists primarily of the development and support of VisIt, a data analysis and visualization tool. Support ranges from answering questions about the tool, providing classes on how to use the tool, and performing data analysis and visualization for customers. The Information Management and Graphics Group supports and develops tools that enhance our ability to access, display, and understand large, complex data sets. Activities include applying visualization software for large scale data exploration; running video production labs on two networks; supporting graphics libraries and tools for end users; maintaining PowerWalls and assorted other displays; and developing software for searching and managing scientific data. Researchers in the Center for Applied Scientific Computing (CASC) work on various projects including the development of visualization techniques for large scale data exploration that are funded by the ASC program, among others. The researchers also have LDRD projects and collaborations with other lab researchers, academia, and industry. The IMG group is located in the Terascale Simulation Facility, home to Dawn, Atlas, BGL, and others, which includes both classified and unclassified visualization theaters, a visualization computer floor and deployment workshop, and video production labs. We continued to provide the traditional graphics group consulting and video production support. We maintained five PowerWalls and many other displays. We deployed a 576-node Opteron/IB cluster with 72 TB of memory providing a visualization production server on our classified network. We continue to support a 128-node Opteron/IB cluster providing a visualization production server for our unclassified systems and an older 256-node Opteron/IB cluster for the classified systems, as well as several smaller clusters to drive the PowerWalls. The visualization production systems includes NFS servers to provide dedicated storage for data analysis and visualization. The ASC projects have delivered new versions of visualization and scientific data management tools to end users and continue to refine them. VisIt had 4 releases during the past year, ending with VisIt 2.0. We released version 2.4 of Hopper, a Java application for managing and transferring files. This release included a graphical disk usage view which works on all types of connections and an aggregated copy feature for quickly transferring massive datasets quickly and efficiently to HPSS. We continue to use and develop Blockbuster and Telepath. Both the VisIt and IMG teams were engaged in a variety of movie production efforts during the past year in addition to the development tasks.
The Web provides new Methods for accessing Information world-wide, but the current text-and-pictures approach neither utilizes all the Web's possibilities not provides for its limitations. While the inclusion of pictures and animations in a paper communicates more effectively than text alone, It Is essentially an extension of the concept of "publication." Also, as use of the Web increases putting images and animations online will quickly load even the "Information Superhighway." We need to find forms of communication that take advantage of the special nature of the Web. This paper presents one approach: the use of the Internet and the Mosaic interface for data sharing and collaborative analysis. We will describe (and In the presentation, demonstrate) our approach: using FAST (Flow Analysis Software Toolkit), a scientific visualization package, as a data viewer and interactive tool called from MOSAIC. Our intent is to stimulate the development of other tools that utilize the unique nature of electronic communication.
This white paper investigates several key aspects of the trust that a user can give to the results of numerical simulations and scientific data analytics. In this document, the notion of trust is related to the integrity of numerical simulations and data analytics applications. This white paper complements the DOE ASCR report on Cybersecurity for Scientific Computing Integrity by (1) exploring the sources of trust loss; (2) reviewing the definitions of trust in several areas; (3) providing numerous cases of result alteration, some of them leading to catastrophic failures; (4) examining the current notion of trust in numerical simulation and scientific data analytics; (5) providing a gap analysis; and (6) suggesting two important research directions and their respective research topics. To simplify the presentation without loss of generality, we consider that trust in results can be lost (or the results’ integrity impaired) because of any form of corruption happening during the execution of the numerical simulation or the data analytics application. In general, the sources of such corruption are threefold: errors, bugs, and attacks. Current applications are already using techniques to deal with different types of corruption. However, not all potential corruptions are covered by these techniques. We firmly believe that the current level of trust that a user has in the results is at least partially founded on ignorance of this issue or the hope that no undetected corruptions will occur during the execution. This white paper explores the notion of trust and suggests recommendations for developing a more scientifically grounded notion of trust in numerical simulation and scientific data analytics. We first formulate the problem and show that it goes beyond previous questions regarding the quality of results such as V&V, uncertainly quantification, and data assimilation. We then explore the complexity of this difficult problem, and we sketch complementary general approaches to address it. This paper does not focus on the trust that the execution will actually complete. The product of simulation or of data analytic executions is the final element of a potentially long chain of transformations, where each stage has the potential to introduce harmful corruptions. These corruptions may produce results that deviate from the user-expected accuracy without notifying the user of this deviation. There are many potential sources of corruption before and during the execution; consequently, in this white paper we do not focus on the protection of the end result after the execution.
Version 2 of the Global Ocean Data Analysis Project (GLODAPv2) data product is composed of data from 724 scientific cruises covering the global ocean. It includes data assembled during the previous efforts GLODAPv1.1 (Global Ocean Data Analysis Project version 1.1) in 2004, CARINA (CARbon IN the Atlantic) in 2009/2010, and PACIFICA (PACIFic ocean Interior CArbon) in 2013, as well as data from an additional 168 cruises. Data for 12 core variables (salinity, oxygen, nitrate, silicate, phosphate, dissolved inorganic carbon, total alkalinity, pH, CFC-11, CFC-12, CFC-113, and CCl 4 ) have been subjected to extensive quality control, including systematic evaluation of bias. The data are available in two formats: (i) as submitted but updated to WOCE exchange format and (ii) as a merged and internally consistent data product. In the latter, adjustments have been applied to remove significant biases, respecting occurrences of any known or likely time trends or variations. Adjustments applied by previous efforts were re-evaluated. Hence, GLODAPv2 is not a simple merging of previous products with some new data added but a unique, internally consistent data product. In conclusion, this compiled and adjusted data product is believed to be consistent to better than 0.005 in salinity, 1 % in oxygen, 2 % in nitrate, 2 % in silicate, 2 % in phosphate, 4 µmol kg -1 in dissolved inorganic carbon, 6 µmol kg -1 in total alkalinity, 0.005 in pH, and 5 % for the halogenated transient tracers.
Modeling large datasets through Multivariate Functional Approximations (MFA) provide an elegant way to handle many visualization and scientific analysis workflows. The process necessitates scalable data partitioning methods to compute MFA representations efficiently without compromising the accuracy or continuity of the reconstructed solution. We propose a domain -decomposed method for computing the MFA with B -spline bases, which reduces the total work per task and uses a restricted Additive Schwarz (RAS) method to converge the control point data degrees -of -freedom along subdomain boundaries. We provide an in-depth analysis of the parallel approach with domain decomposition solvers, aiming to minimize local subdomain error residuals and recover high -order continuity at subdomain interfaces with appropriate choices of knot overlaps. The communication cost, determined by the overlap regions in the RAS implementation, is optimized to recover the numerical error profile of the single subdomain case. Our proposed method stands in contrast to previous methods, which typically only recover either C 0 or at best C 1 continuity for arbitrary B -spline degree expansions, or those that require post -processing to blend discontinuities in the reconstructed data. We demonstrate the effectiveness of our approach using analytical and real -world datasets in 1D, 2D, and 3D through both strong and weak scaling studies. The performance results indicate that the overall cost of computing the approximation is directly proportional to the underlying nearest -neighbor communication implementation, and is only weakly dependent on the overlap region size that determines the size of the messages. This finding underscores the efficiency and scalability of our proposed method, making it a promising solution for handling large datasets in scientific workflows.
We observe the emergence of a new generation of scientific workflows that process data produced at a sustained rate by scientific instruments and large scale numerical simulations. This data is consumed by multiple analysis, visualization, or Machine Learning components not only to enable inference and justify the scientific program, but also to monitor and steer the evolution of these experiments. In such workflows, moving intermediate data efficiently is key to performance, more than efficiently scheduling computational tasks. However, most traditional workflow management systems focus on optimizing task scheduling and then deal with data management, assuming a “move little, compute for long” model, which makes them unfit to the efficient management of this new generation of workflows. Therefore, we advocate for a new way to manage scientific workflows. We propose to consider an efficiently and independently managed data plane that can store and stream data. Workflows compute components, in the application plane can then interact with the data plane, abstracted from complexities of data management. Then, the role of a workflow management system would become that of a control plane that allows users to connect services together to execute the workflow and manages connections between the application and data planes. In this position paper, we characterize several next-generation workflow motifs and describe how their interaction with the data plane is a challenge to traditional workflow management systems. Then, we express a set of requirements that a workflow management system should meet to efficiently manage next-generation workflows at different scales. Based on these requirements, we expose our vision of driving next-generation workflows from the data plane and list remaining open challenges.
A principal tenant of the scientific method is that experiments must be repeatable and relies on ceteris paribus (i.e., all other things being equal). As a scientific community, involved in data sciences, we must investigate ways to establish an environment where experiments can be repeated. We can no longer allude to where the data comes from, we must add rigor to the data collection and management process from which our analysis is conducted. This paper describes a computing environment to support repeatable scientific big data experimentation of world-wide scientific literature, and recommends a system that is housed at the Oak Ridge National Laboratory in order to provide value to investigators from government agencies, academic institutions, and industry entities. The described computing environment also adheres to the recently instituted digital data management plan mandated by multiple US government agencies, which involves all stages of the digital data life cycle including capture, analysis, sharing, and preservation. It particularly focuses on the sharing and preservation of digital research data. The details of this computing environment are explained within the context of cloud services by the three layer classification of Software as a Service , Platform as a Service , and Infrastructure as a Service .
The RAVEN code is becoming a comprehensive tool to perform probabilistic risk assessment, uncertainty quantification, and verification and validation. The RAVEN code is being developed to support many programs and to provide a set of methodologies and algorithms for advanced analysis. Scientific computer codes can generate enormous amounts of data. To post-process and analyze such data might, in some cases, take longer than the initial software runtime. Data mining algorithms/methods help in recognizing and understanding patterns in the data, and thus discover knowledge in databases. The methodologies used in the dynamic probabilistic risk assessment or in uncertainty and error quantification analysis couple system/physics codes with simulation controller codes, such as RAVEN. RAVEN introduces both deterministic and stochastic elements into the simulation while the system/physics code model the dynamics deterministically. A typical analysis is performed by sampling values of a set of parameter values. A major challenge in using dynamic probabilistic risk assessment or uncertainty and error quantification analysis for a complex system is to analyze the large number of scenarios generated. Data mining techniques are typically used to better organize and understand data, i.e. recognizing patterns in the data. This report focuses on development and implementation of Application Programming Interfaces (APIs) for different data mining algorithms, and the application of these algorithms to different databases.
Many scientific disciplines rely on computational methods for data analysis, model generation, and prediction. Implementing these methods is often accomplished by researchers with domain expertise but without formal training in software engineering or computer science. This arrangement has led to underappreciation of sustainability and maintainability of scientific software tools developed in academic environments. Some software tools have avoided this fate, including the scientific library Rosetta. We use this software and its community as a case study to show how modern software development can be accomplished successfully, irrespective of subject area. Rosetta is one of the largest software suites for macromolecular modeling, with 3.1 million lines of code and many state-of-the-art applications. Since the mid 1990s, the software has been developed collaboratively by the RosettaCommons, a community of academics from over 60 institutions worldwide with diverse backgrounds including chemistry, biology, physiology, physics, engineering, mathematics, and computer science. Developing this software suite has provided us with more than two decades of experience in how to effectively develop advanced scientific software in a global community with hundreds of contributors. Here we illustrate the functioning of this development community by addressing technical aspects (like version control, testing, and maintenance), community-building strategies, diversity efforts, software dissemination, and user support. We demonstrate how modern computational research can thrive in a distributed collaborative community. The practices described here are independent of subject area and can be readily adopted by other software development communities
Because of the importance of surfaces and interfaces in many scientific and technological areas, the use of x-ray photoelectron spectroscopy (XPS) has been growing exponentially. Although XPS is being used to obtain useful information about the surface composition of samples, much more information about materials and their properties can be extracted from XPS data than commonly obtained. This paper describes some of the areas where alternative analysis methods or experimental design can obtain information about the near-surface region of a sample, often information not available in other ways. Experienced XPS analysts are familiar with many of these methods, but they may not be known to new or casual XPS users, and sometimes, they have not been used because of an inappropriately assumed complexity. The information available includes optical, electronic, and electrical properties; nanostructure; expanded chemical information; and enhanced analysis of biological materials and solid/liquid interfaces. Many of these analyses can be conducted on standard laboratory XPS systems, with either no or relatively minor system alterations. Topics discussed include (1) considerations beyond the “traditional” uniform surface layer composition calculation, (2) using the Auger parameter to determine a sample property, (3) use of the D parameter to identify sp 2 and sp 3 carbon information, (4) information from the XPS valence band, (5) using cryocooling to expand range of samples that can be analyzed and minimize damage, and (6) using electrical potential effects on XPS signals to extract chemically resolved electrical measurements including band alignment and electrical property information.
For the x-ray observatory ROSAT, data from survey and pointed mission phases taken with different focal plane instruments and according to a complex mission timeline have to be handled. Data analysis therefore puts high demands on appropriate software tools. With EXSAS - the Extended Scientific Analysis System developed with an effort of 20 man years by the German ROSAT Scientific Data Center - a comfortable system for the reduction of data from the ROSAT x-ray and XUV instruments has been made available. EXSAS comprises a large collection of application modules as typically required in analyzing data of this wavelength regime and runs as a specific context in the wide-spread ESO-MIDAS environment. EXSAS, completely written in FORTRAN 77, takes full advantage of all the standards used in MIDAS and therefore, reflects the same portability (different UNIX installations and VMS). If required, the FORTRAN code also enables users to adapt the software in an easy way to their specific needs. To maintain independence from the specifics of different operating systems also on the data input side, all ROSAT data redistributed in the widely accepted FITS format. Although EXSAS has been developed specifically for data analysis of the ROSAT instruments, its structural design is sufficiently general to serve equally well also data from other X-ray and XUV instruments. EXSAS analysis modules are grouped into 4 application packages dealing with Data Preparation and Instrument Correction, Spatial Analysis, Spectral Analysis and Timing Analysis. A special EXSAS header, read and updated by each application, maintains the general information transfer on the origin, the history and the parameter space of the data stored in tables and images. About 100 genuine commands (most of which offer several additional options) allow to interactively explore the functionality of the system. Up to now 40 institutes all over the world have requested the EXSAS software. Maintenance and regular updates of the software and the comprehensive documentation are provided by the ROSAT Scientific Data Center at Garching.
This dissertation presents research on addressing some of the contemporary challenges in the analysis of vector fields—an important type of scientific data useful for representing a multitude of physical phenomena, such as wind flow and ocean currents. In particular, new theories and computational frameworks to enable consistent feature extraction from vector fields are presented. One of the most fundamental challenges in the analysis of vector fields is that their features are defined with respect to reference frames. Unfortunately, there is no single “correct” reference frame for analysis, and an unsuitable frame may cause features of interest to remain undetected, thus creating serious physical consequences. This work develops new reference frames that enable extraction of localized features that other techniques and frames fail to detect. As a result, these reference frames objectify the notion of “correctness” of features for certain goals by revealing the phenomena of importance from the underlying data. An important consequence of using these local frames is that the analysis of unsteady (time-varying) vector fields can be reduced to the analysis of sequences of steady (timeindependent) vector fields, which can be performed using simpler and scalable techniques that allow better data management by accessing the data on a per-time-step basis. Nevertheless, the state-of-the-art analysis of steady vector fields is not robust, as most techniques are numerical in nature. The residing numerical errors can violate consistency with the underlying theory by breaching important fundamental laws, which may lead to serious physical consequences. This dissertation considers consistency as the most fundamental characteristic of computational analysis that must always be preserved, and presents a new discrete theory that uses combinatorial representations and algorithms to provide consistency guarantees during vector field analysis along with the uncertainty visualization of unavoidable discretization errors. Together, the two main contributions of this dissertation address two important concerns regarding feature extraction from scientific data: correctness and precision. The work presented here also opens new avenues for further research by exploring more-general reference frames and more-sophisticated domain discretizations.
Fermilab has been running a Ceph cluster in production for several years to support high-throughput scientific computing. Our primary use case is CephFS, which serves interactive data analysis workloads, with growing interest in using RGW for scalable object storage of scientific datasets. In this talk, we'll share lessons learned from successfully deploying and maintaining our Ceph cluster with cephadm, including challenges faced, performance tuning, and operational practices. We'll also present custom tools we've developed to streamline monitoring and management and discuss how Ceph fits into our broader storage architecture for large-scale scientific research.
Analysis of the Skylab S-193 Altimeter Experiment data has produced many significant results. The purpose of this paper is to present some of these results which illustrate the programmatic altimeter technology gained and show the scientific potential of altimetry for various applications, including correlation with existing topographic features, geoids and physical models. It will be shown that there exists a very strong correlation between subsurface topography in the broad ocean area and the shape of the ocean surface as determined from the altimeter. Modeled scattering waveforms will be correlated with Skylab Altimeter waveforms. The agreement of a variety of existing geoid models with the altimeter data will be shown. In addition the calibration, accuracy and stability of the S-193 altimeter instrumentation will be discussed.