Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software: data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Astro-H Data Analysis, Processing and Archive

Astro-H (Hitomi) is an X-ray Gamma-ray mission led by Japan with international participation, launched on February 17, 2016. The payload consists of four different instruments (SXS, SXI, HXI and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data along with the plan for the archive and user support.These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and USA.

Archive↗

AXEAP : a software package for X-ray emission data analysis using unsupervised machine learning

The Argonne X-ray Emission Analysis Package ( AXEAP ) has been developed to calibrate and process X-ray emission spectroscopy (XES) data collected with a two-dimensional (2D) position-sensitive detector. AXEAP is designed to convert a 2D XES image into an XES spectrum in real time using both calculations and unsupervised machine learning. AXEAP is capable of making this transformation at a rate similar to data collection, allowing real-time comparisons during data collection, reducing the amount of data stored from gigabyte-sized image files to kilobyte-sized text files. With a user-friendly interface, AXEAP includes data processing for non-resonant and resonant XES images from multiple edges and elements. AXEAP is written in MATLAB and can run on common operating systems, including Linux, Windows, and MacOS.

97 MATHEMATICS AND COMPUTING↗

Multivariate statistical analysis software technologies for astrophysical research involving large data bases

The existing and forthcoming data bases from NASA missions contain an abundance of information whose complexity cannot be efficiently tapped with simple statistical techniques. Powerful multivariate statistical methods already exist which can be used to harness much of the richness of these data. Automatic classification techniques have been developed to solve the problem of identifying known types of objects in multi parameter data sets, in addition to leading to the discovery of new physical phenomena and classes of objects. We propose an exploratory study and integration of promising techniques in the development of a general and modular classification/analysis system for very large data bases, which would enhance and optimize data management and the use of human research resources.

Djorgovski, Stanislav↗

Multivariate statistical analysis software technologies for astrophysical research involving large data bases

The existing and forthcoming data bases from NASA missions contain an abundance of information whose complexity cannot be efficiently tapped with simple statistical techniques. Powerful multivariate statistical methods already exist which can be used to harness much of the richness of these data. Automatic classification techniques have been developed to solve the problem of identifying known types of objects in multiparameter data sets, in addition to leading to the discovery of new physical phenomena and classes of objects. We propose an exploratory study and integration of promising techniques in the development of a general and modular classification/analysis system for very large data bases, which would enhance and optimize data management and the use of human research resource.

Djorgovski, George↗

Open Reproducible Electron Microscopy Data Analysis

Electron microscopy (EM) is a cornerstone technique in the materials and biological sciences capable of imaging structures at nano- to atomic-scale resolution. Advances in technologies mean that one acquires datasets at increasing data rates and sizes. These advancements present enormous opportunities for researchers to understand complex systems. However, processing the resulting large-scale, complex data in a reproducible and shareable way is a real challenge for researchers. The building, managing, and maintaining complex workflows in a reproducible manner requires extensive knowledge in several areas outside the researchers’ core skill sets, such as software engineering, data science, and high-performance computing (HPC). Our work demonstrates an innovative approach to solving these problems, enabling reproducible EM data analysis through container encapsulated pipelines. Using modern container technologies, we encapsulate processing elements and connect them using shared memory. We expose user-friendly, advanced algorithms and tools to allow end users to utilize without expert programming skills. The platform enables reproducible, scalable, shareable pipelines for the analysis and visualization of EM data. Focusing on interoperability, we leverage the DOE and other agencies’ existing investments to provide a powerful software platform for EM data analysis.

Harris, Christopher↗

The Analysis Description Language Ecosystem: Latest developments and physics applications

We present latest developments in Analysis Description Language (ADL), a declarative domain-specific language describing the physics algorithm of a HEP data analysis decoupled from software frameworks. Analyses written in ADL can be integrated into any framework for various tasks. ADL is a multipurpose construct with uses ranging from analysis design to preservation, reinterpretation, queries, visualisation, combination, etc. The most advanced infrastructure to execute ADL on events is the CutLang runtime interpreter. Recent technical developments include an automated interface with different data types, generation of the abstract syntax tree, a visualization tool that that auto-converts analysis flows to graphs, incorporation of trained machine learning models and a Jupyter-based plotting tool. We also report physics implications including a large scale LHC analysis implementation and validation effort for beyond the standard model reinterpretation purposes and studies with ATLAS and CMS open data.

Sekmen, Sezen [Kyungpook National Univ., Daegu (Ko↗

LV software support for supersonic flow analysis

The software for configuring a Laser Velocimeter (LV) counter processor system was developed using structured design. The LV system includes up to three counter processors and a rotary encoder. The software for configuring and testing the LV system was developed, tested, and included in an overall software package for data acquisition, analysis, and reduction. Error handling routines respond to both operator and instrument errors which often arise in the course of measuring complex, high-speed flows. The use of networking capabilities greatly facilitates the software development process by allowing software development and testing from a remote site. In addition, high-speed transfers allow graphics files or commands to provide viewing of the data from a remote site. Further advances in data analysis require corresponding advances in procedures for statistical and time series analysis of nonuniformly sampled data.

Bell, William A.↗

LV software support for supersonic flow analysis

The software for configuring an LV counter processor system has been developed using structured design. The LV system includes up to three counter processors and a rotary encoder. The software for configuring and testing the LV system has been developed, tested, and included in an overall software package for data acquisition, analysis, and reduction. Error handling routines respond to both operator and instrument errors which often arise in the course of measuring complex, high-speed flows. The use of networking capabilities greatly facilitates the software development process by allowing software development and testing from a remote site. In addition, high-speed transfers allow graphics files or commands to provide viewing of the data from a remote site. Further advances in data analysis require corresponding advances in procedures for statistical and time series analysis of nonuniformly sampled data.

Bell, W. A.↗

MAPPRAISER: A massively parallel map-making framework for multi-kilo pixel CMB experiments

Forthcoming cosmic microwave background (CMB) polarized anisotropy experiments have the potential to revolutionize our understanding of the Universe and fundamental physics. The sought-after, tale-telling signatures will be however distributed over voluminous data sets which these experiments will collect. These data sets will need to be efficiently processed and unwanted contributions due to astrophysical, environmental, and instrumental effects characterized and efficiently mitigated in order to uncover the signatures. This poses a significant challenge to data analysis methods, techniques, and software tools which will not only have to be able to cope with huge volumes of data but to do so with unprecedented precision driven by the demanding science goals posed for the new experiments. A keystone of efficient CMB data analysis is solvers of very large linear systems of equations. Such systems appear in very diverse contexts throughout CMB data analysis pipelines, however they typically display similar algebraic structures and can therefore be solved using similar numerical techniques. Linear systems arising in the so-called map-making problem are one of the most prominent and common ones. In this work we present a massively parallel, flexible and extensible framework, comprised of a numerical library, MIDAPACK, and a high level code, MAPPRAISER, which provide tools for solving efficiently such systems. Here, the framework implements iterative solvers based on conjugate gradient techniques: enlarged and preconditioned using different preconditioners. We demonstrate the framework on simulated examples reflecting basic characteristics of the forthcoming data sets issued by ground-based and satellite-borne instruments, executing it on as many as 16,384 compute cores. The software is developed as an open source project freely available to the community at: https://github.com/B3Dcmb/midapack.

79 ASTRONOMY AND ASTROPHYSICS↗

Interpreting forest and grassland biome productivity utilizing nested scales of image resolution and biogeographical analysis

Several hardware, software, and data collection problems encountered were conquered. The Geographic Information System (GIS) data from other systems were converted to ERDAS format for incorporation with the image data. Statistical analysis of the relationship between spectral values and productivity is being pursued. Several project sites, including Jackson, Pope, Boulder, Smokies, and Huntington Forest are evolving as the most intensively studied areas, primarily due to availability of data and time. Progress with data acquisition and quality checking, more details on experimental sites, and brief summarizations of research results and future plans are discussed. Material on personnel, collaborators, facilities, site background, and meetings and publications of the investigators are included.

Iverson, L. R.↗

Earth resources data analysis system

The Earth Resources Laboratory Data Analysis System provides a valuable research tool in the evaluation of multispectral, radiometric, photographic, and passive microwave imagery data. The system has been in operational use for several months and has performed quite well. Emphasis on versatility of both the hardware and software design facilitates system expansion to many specialized data analysis tasks. The many independent software handlers provide building blocks for major processing programs.

Source record↗

Simulation of a Real-Time Local Data Integration System over East-Central Florida

The Applied Meteorology Unit (AMU) simulated a real-time configuration of a Local Data Integration System (LDIS) using data from 15-28 February 1999. The objectives were to assess the utility of a simulated real-time LDIS, evaluate and extrapolate system performance to identify the hardware necessary to run a real-time LDIS, and determine the sensitivities of LDIS. The ultimate goal for running LDIS is to generate analysis products that enhance short-range (less than 6 h) weather forecasts issued in support of the 45th Weather Squadron, Spaceflight Meteorology Group, and Melbourne National Weather Service operational requirements. The simulation used the Advanced Regional Prediction System (ARPS) Data Analysis System (ADAS) software on an IBM RS/6000 workstation with a 67-MHz processor. This configuration ran in real-time, but not sufficiently fast for operational requirements. Thus, the AMU recommends a workstation with a 200-MHz processor and 512 megabytes of memory to run the AMU's configuration of LDIS in real-time. This report presents results from two case studies and several data sensitivity experiments. ADAS demonstrates utility through its ability to depict high-resolution cloud and wind features in a variety of weather situations. The sensitivity experiments illustrate the influence of disparate data on the resulting ADAS analyses.

Case, Jonathan↗

Structure, composition and thermal state of the crust in Brazil

Aeromagnetic survey data of the Minas Gerais areas were analyzed and compared with MAGSAT anomaly data. Initially the data set was used to determine the thickness of the magnetized crust. The value obtained is in agreement with crustal models derived from deep seismic sounding and gravity data, indicating that the bottom of the magnetized layer occurs close to the Moho transition. An analysis of the long wavelength pattern indicates that the anomalies form a series of lineations along the magnetic E-W. These are shown to be the result of attenuation. On the MAGSAT total intensity map localized anomalies are superimposed onto an anomaly background formed by alternating positive and negative patches. Implementation and testing of analysis software for MAGSAT data are also discussed.

Pacca, I. I. G.↗

Automated Processing of ISIS Topside Ionograms into Electron Density Profiles

Modeling of the topside ionosphere has for the most part relied on just a few years of data from topside sounder satellites. The widely used Bent et al. (1972) model, for example, is based on only 50,000 Alouette 1 profiles. The International Reference Ionosphere (IRI) (Bilitza, 1990, 2001) uses an analytical description of the graphs and tables provided by Bent et al. (1972). The Alouette 1, 2 and ISIS 1, 2 topside sounder satellites of the sixties and seventies were ahead of their times in terms of the sheer volume of data obtained and in terms of the computer and software requirements for data analysis. As a result, only a small percentage of the collected topside ionograms was converted into electron density profiles. Recently, a NASA-funded data restoration project has undertaken and is continuing the process of digitizing the Alouette/ISIS ionograms from the analog 7-track tapes. Our project involves the automated processing of these digital ionograms into electron density profiles. The project accomplished a set of important goals that will have a major impact on understanding and modeling of the topside ionosphere: (1) The TOPside Ionogram Scaling and True height inversion (TOPIST) software was developed for the automated scaling and inversion of topside ionograms. (2) The TOPIST software was applied to the over 300,000 ISIS-2 topside ionograms that had been digitized in the fkamework of a separate AISRP project (PI: R.F. Benson). (3) The new TOPIST-produced database of global electron density profiles for the topside ionosphere were made publicly available through NASA s National Space Science Data Center (NSSDC) ftp archive at . (4) Earlier Alouette 1,2 and ISIS 1, 2 data sets of electron density profiles from manual scaling of selected sets of ionograms were converted fiom a highly-compressed binary format into a user-friendly ASCII format and made publicly available through nssdcftp.gsfc.nasa.gov. The new database for the topside ionosphere established as a result of this project, has stimulated a multitude of new studies directed towards a better description and prediction of the topside ionosphere. Marinov et al. (2004) developed a new model for the upper ion transition height (Oxygen to Hydrogen and Helium) and Bilitza (2004) deduced a correction term for the I N topside electron density model. Kutiev et al. (2005) used this data to develop a new model for the topside ionosphere scale height (TISH) as a function of month, local time, latitude, longitude and solar flux F10.7. Comparisons by Belehaki et al. (2005) show that TISH is in general agreement with scale heights deduced from ground ionosondes but the model predicts post-midnight and afternoon maxima whereas the ionosonde data show a noon maximum. Webb and Benson (2005) reported on their effort to deduce changes in the plasma temperature and ion composition from changes in the topside electron density profile as recorded by topside sounders. Limitations and possible improvements of the IRI topside model were discussed by Coisson et al. (2005) including also the possible use of the NeQuick model, Our project progressed in close collaboration and coordination with the GSFC team involved in the ISIS digitization effort. The digitization project was highly successful producing a large amount of digital topside ionograms. Several no-cost extensions of the TOPIST project were necessary to keep up with the pace and volume of the digitization effort.

Reinisch, bodo W.↗

Software manual for operating particle displacement tracking data acquisition and reduction system

The software manual is presented. The necessary steps required to record, analyze, and reduce Particle Image Velocimetry (PIV) data using the Particle Displacement Tracking (PDT) technique are described. The new PDT system is an all electronic technique employing a CCD video camera and a large memory buffer frame-grabber board to record low velocity (less than or equal to 20 cm/s) flows. Using a simple encoding scheme, a time sequence of single exposure images are time coded into a single image and then processed to track particle displacements and determine 2-D velocity vectors. All the PDT data acquisition, analysis, and data reduction software is written to run on an 80386 PC.

Wernet, Mark P.↗

Developing an Automated Uncertainty Quantification Tool to Improve Watershed-Scale Predictions of Water and Nutrient Cycling

Managing the flow of water, nutrients, and contaminants in watersheds is vital to addressing pressing issues related to water scarcity, access to clean drinking water, energy production, resilience to natural and anthropogenic perturbations, and ecological restoration. Decisions about the management of watersheds critically depend on the accuracy with which the flow of water and chemicals through the watershed can be predicted by computer models. Prediction uncertainty can be reduced by matching the model to data, which are collected in the field at great expense. The contribution of watershed characterization data to reducing uncertainty of relevant model predictions can be evaluated in a so-called data-worth analysis, which provides transparent, quantitative metrics about a data set’s value for the support of relevant watershed management objectives. To achieve this goal, we developed a software package that implements the data-worth analysis approach for use with state-of-the-art watershed models. The purpose of the proposed data-worth analysis is to help decision-makers allocate resources for watershed characterization such that the uncertainty in model predictions can be significantly reduced, which leads to better, more effective management decisions. At the same time, watershed characterization costs can be reduced. The specific technical objectives of this SBIR/STTR Phase II project were to develop a framework and associated software toolsets that implement the uncertainty quantification and data-worth analysis approach for use with state-of-the-art watershed models. This goal was achieved by (A) developing a user-friendly, robust software package that is accessible to a wide audience, including watershed managers, policy-makers, and public stakeholders; (B) by demonstrating application of the prototype on several use cases that are representative of complex watershed management challenges spanning a range of scales and that consider different open-source, DOE-based codes and other modeling platforms; and (C) by gathering information about the needs and requirements from potential users to help guide future developments, ensuring that the final product will be commercially viable. The developed software consists of a graphical user interface that guides the user through a sequence of analysis steps, supported by toolsets that leverage state-of-the-art computational simulation-optimization capabilities. A prototype of the software runs on multiple platforms (PC, Mac, multi-processor Linux environment), is linked to diverse watershed simulators (e.g., ECOSYS, TOUGH2, TOUGHREACT, Amanzi-ATS), performs multiple analysis tasks (predictive simulations, sensitivity analysis, uncertainty analysis, automatic parameter estimation, and data-worth analysis, multicomponent geothermometry), and is readily extensible to include external simulators and analysis tools. The software is being commercialized and will be continually updated to address user needs.

58 GEOSCIENCES↗

Mapping structural heterogeneity at the nanoscale with scanning nano-structure electron microscopy (SNEM)

Here, in this work, we explore the use of scanning electron diffraction (also known as 4D-STEM) coupled with electron atomic pair distribution function analysis (ePDF) to understand the local order (structure and chemistry) as a function of position in a complex multicomponent system, a hot rolled, Ni-encapsulated, Zr 65 Cu 17.5 Ni 10 Al 7.5 bulk metallic glass (BMG), with a spatial resolution of 3 nm. We show that it is possible to gain insight into the chemistry and chemical clustering/ordering tendency in different regions of the sample, including in the vicinity of nano-scale crystallites that are identified from virtual dark field images and in heavily deformed regions at the edge of the BMG. In addition to simpler analysis, unsupervised machine learning was used to extract partial PDFs from the material, modeled as a quasi-binary alloy, and map them in space. These maps allowed key insights not only into the local average composition, as validated by EELS, but also a unique insight into chemical short-range ordering tendencies in different regions of the sample during formation. The experiments are straightforward and rapid and, unlike spectroscopic measurements, don’t require energy filters on the instrument. We spatially map different quantities of interest (QoI’s), defined as scalars that can be computed directly from positions and widths of ePDF peaks or parameters refined from fits to the patterns. We developed a flexible and rapid data reduction and analysis software framework that allows experimenters to rapidly explore images of the sample on the basis of different QoI’s. The power and flexibility of this approach are explored and described in detail. Because of the fact that we are getting spatially resolved images of the nanoscale structure obtained from ePDFs we call this approach scanning nano-structure electron microscopy (SNEM), and we believe that it will be powerful and useful extension of current 4D-STEM methods.

36 MATERIALS SCIENCE↗

DRAGraces: A Pipeline for the GRACES High-resolution Spectrograph at Gemini

This paper describes the software Data Reduction and Analysis for GRACES (DRAGraces), which is a pipeline reducing spectra from Gemini Remote Access to the CFHT ESPaDOnS Spectrograph (GRACES) at the Gemini North Telescope. The code is written in the IDL language. It is designed to find all the GRACES frames in a given directory, automatically determine the list of bias, flat, arc, and science frames, and perform the whole reduction and extraction within a few minutes. We compare the output from DRAGraces with that of the Open source Pipeline for ESPaDOnS Reduction and Analysis (OPERA), a pipeline developed at the Canada–France–Hawaii Telescope (CFHT) that also can extract GRACES spectra. Both pipelines were developed completely independently, yet they give very similar extracted spectra. They both have their advantages and disadvantages. For instance, DRAGraces is more straightforward and easy to use and is less likely to be derailed by a parameter that needs to be tweaked, while OPERA offers a more careful extraction that can be significantly superior when the highest resolution is required and when the signal-to-noise ratio is low. One should compare both before deciding which one to use for their science. Yet, both pipelines deliver a fairly comparable resolution power (R ~ 52.8k and 36.6k for DRAGraces and R ~ 58k and 40k for OPERA in high- and low-resolution spectral modes, respectively), wavelength solution, and signal-to-noise ratio per resolution element.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗