Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS2012

Space Shuttle Orbiter Digital Outer Mold Line Scanning

The Space Shuttle Orbiters Discovery and Endeavor have been digitally scanned to produce post-flight configuration outer mold line surfaces. Very detailed scans of the windward side of these vehicles provide resolution of the detailed tile step and gap geometry, as well as the reinforced carbon carbon nose cap and leading edges. Lower resolution scans of the upper surface provide definition of the crew cabin windows, wing upper surfaces, payload bay doors, orbital maneuvering system pods and the vertical tail. The process for acquisition of these digital scans as well as post-processing of the very large data set will be described.

Campbell, Charles H.↗
NASA NTRS2013

Tool for Rapid Analysis of Monte Carlo Simulations

Designing a spacecraft, or any other complex engineering system, requires extensive simulation and analysis work. Oftentimes, the large amounts of simulation data generated are very difficult and time consuming to analyze, with the added risk of overlooking potentially critical problems in the design. The authors have developed a generic data analysis tool that can quickly sort through large data sets and point an analyst to the areas in the data set that cause specific types of failures. The first version of this tool was a serial code and the current version is a parallel code, which has greatly increased the analysis capabilities. This paper describes the new implementation of this analysis tool on a graphical processing unit, and presents analysis results for NASA's Orion Monte Carlo data to demonstrate its capabilities.

Restrepo, Carolina↗
NASA NTRS2012

QuakeSim 2.0

QuakeSim 2.0 improves understanding of earthquake processes by providing modeling tools and integrating model applications and various heterogeneous data sources within a Web services environment. QuakeSim is a multisource, synergistic, data-intensive environment for modeling the behavior of earthquake faults individually, and as part of complex interacting systems. Remotely sensed geodetic data products may be explored, compared with faults and landscape features, mined by pattern analysis applications, and integrated with models and pattern analysis applications in a rich Web-based and visualization environment. Integration of heterogeneous data products with pattern informatics tools enables efficient development of models. Federated database components and visualization tools allow rapid exploration of large datasets, while pattern informatics enables identification of subtle, but important, features in large data sets. QuakeSim is valuable for earthquake investigations and modeling in its current state, and also serves as a prototype and nucleus for broader systems under development. The framework provides access to physics-based simulation tools that model the earthquake cycle and related crustal deformation. Spaceborne GPS and Inter ferometric Synthetic Aperture (InSAR) data provide information on near-term crustal deformation, while paleoseismic geologic data provide longerterm information on earthquake fault processes. These data sources are integrated into QuakeSim's QuakeTables database system, and are accessible by users or various model applications. UAVSAR repeat pass interferometry data products are added to the QuakeTables database, and are available through a browseable map interface or Representational State Transfer (REST) interfaces. Model applications can retrieve data from Quake Tables, or from third-party GPS velocity data services; alternatively, users can manually input parameters into the models. Pattern analysis of GPS and seismicity data has proved useful for mid-term forecasting of earthquakes, and for detecting subtle changes in crustal deformation. The GPS time series analysis has also proved useful as a data-quality tool, enabling the discovery of station anomalies and data processing and distribution errors. Improved visualization tools enable more efficient data exploration and understanding. Tools provide flexibility to science users for exploring data in new ways through download links, but also facilitate standard, intuitive, and routine uses for science users and end users such as emergency responders.

Donnellan, Andrea↗
NASA NTRS2012

Improving the Automatic Inversion of Digital Alouette/ISIS Ionogram Reflection Traces into Topside Electron Density Profiles

The topside sounders of the International Satellites for Ionospheric Studies (ISIS) program were designed as analog systems. The resulting ionograms were displayed on 35 mm film for analysis by visual inspection. Each of these satellites, launched between 1962 and 1971, produced data for 10 to 20 years. A number of the original telemetry tapes from this large data set have been converted directly into digital records. Software, known as the Topside Ionogram Scalar With True-Height (TOPIST) algorithm, has been produced and used for the automatic inversion of the ionogram reflection traces on more than 100,000 ISIS-2 digital topside ionograms into topside vertical electron density profiles Ne(h). Here we present some topside ionospheric solar cycle variations deduced from the TOPIST database to illustrate the scientific benefit of improving and expanding the topside ionospheric Ne(h) database. The profile improvements will be based on improvements in the TOPIST software motivated by direct comparisons between TOPIST profiles and profiles produced by manual scaling in the early days of the ISIS program. The database expansion will be based on new software designed to overcome limitations in the original digital topside ionogram database caused by difficulties encountered during the analog-to-digital conversion process in the detection of the ionogram frame sync pulse and/or the frequency markers. This improved and expanded TOPIST topside Ne(h) database will greatly enhance investigations into both short- and long-term ionospheric changes, e.g., the observed topside ionospheric responses to magnetic storms, induced by interplanetary magnetic clouds, and solar cycle variations, respectively.

Benson, Robert F.↗
NASA NTRS2013

Shape-induced Gravitational Sorting of Saharan Dust During Transatlantic Voyage: Evidence from CALIOP Lidar Depolarization Measurements

Motivated by the physical picture of shape-dependent air resistance and, consequently, shape-induced differential sedimentation of dust particles, we searched for and found evidence of dust particle asphericity affecting the evolution and distribution of dust-scattered light depolarization ratio (delta). Specifically, we examined a large data set of Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) observations of Saharan dust from June to August 2007. Observing along a typical transatlantic dust track, we find that (1) median delta is uniformly distributed between 2 and 5 km altitudes as the elevated dust leaves the west coast of Africa, thereby indicating uniformly random mixing of particle shapes with height; (2) vertical homogeneity of median delta breaks down during the westward transport: between 2 and 5 km delta increases with altitude and this increase becomes more pronounced with westward progress; (3) delta tends to increase at higher altitude (greater than 4 km) and decrease at lower altitude (less than 4 km) during the westward transport. All these features are captured qualitatively by a minimal model (two shapes only), suggesting that shape-induced differential settling and consequent sorting indeed contribute significantly to the observed temporal evolution and vertical stratification of dust properties. By implicating particle shape as a likely cause of gravitational sorting, these results will affect the estimates of radiative transfer through Saharan dust layers.

CALIPSO↗
NASA NTRS2015

Kinematic and Microphysical Control of Lightning Flash Rate over Northern Alabama

The Deep Convective Clouds and Chemistry (DC3) experiment seeks to examine the relationship between deep convection and the production of nitrogen oxides (NO (sub x)) via lightning (LNO (sub x)). A critical step in estimating LNO (sub x) production in a cloud-resolving model (CRM) without explicit lightning is to estimate the flash rate from available model parameters that are statistically and physically correlated. As such, the objective of this study is to develop, improve and evaluate lightning flash rate parameterizations in a variety of meteorological environments and storm types using radar and lightning mapping array (LMA) observations taken over Northern Alabama from 2005-2012, including during DC3. UAH's Advanced Radar for Meteorological and Operational Research (ARMOR) and the Weather Surveillance Radar - 1988 Doppler (WSR 88D) located at Hytop (KHTX) comprises the dual-Doppler and polarimetric radar network, which has been in operation since 2004. The northern Alabama LMA (NA LMA) in conjunction with Vaisala's National Lightning Detection Network (NLDN) allow for a detailed depiction of total lightning during this period. This study will integrate ARMOR-KHTX dual Doppler/polarimetric radar and NA LMA lightning observations from past and ongoing studies, including the more recent DC3 results, over northern Alabama to form a large data set of 15-20 case days and over 20 individual storms, including both ordinary multicell and supercell convection. Several flash rate parameterizations will be developed and tested, including those based on 1) graupel/small hail volume; 2) graupel/small hail mass, and 3) convective updraft volume. Sensitivity of the flash rate parameterizations to storm intensity, storm morphology and environmental conditions will be explored.

Carey, Lawrence D.↗
NASA NTRS2015

Data Mining and Analysis

The Data Mining project seeks to bring the capability of data visualization to NASA anomaly and problem reporting systems for the purpose of improving data trending, evaluations, and analyses. Currently NASA systems are tailored to meet the specific needs of its organizations. This tailoring has led to a variety of nomenclatures and levels of annotation for procedures, parts, and anomalies making difficult the realization of the common causes for anomalies. Making significant observations and realizing the connection between these causes without a common way to view large data sets is difficult to impossible. In the first phase of the Data Mining project a portal was created to present a common visualization of normalized sensitive data to customers with the appropriate security access. The tool of the visualization itself was also developed and fine-tuned. In the second phase of the project we took on the difficult task of searching and analyzing the target data set for common causes between anomalies. In the final part of the second phase we have learned more about how much of the analysis work will be the job of the Data Mining team, how to perform that work, and how that work may be used by different customers in different ways. In this paper I detail how our perspective has changed after gaining more insight into how the customers wish to interact with the output and how that has changed the product.

iss↗
NASA NTRS2013

Downscaling Satellite Precipitation with Emphasis on Extremes: A Variational 1-Norm Regularization in the Derivative Domain

The increasing availability of precipitation observations from space, e.g., from the Tropical Rainfall Measuring Mission (TRMM) and the forthcoming Global Precipitation Measuring (GPM) Mission, has fueled renewed interest in developing frameworks for downscaling and multi-sensor data fusion that can handle large data sets in computationally efficient ways while optimally reproducing desired properties of the underlying rainfall fields. Of special interest is the reproduction of extreme precipitation intensities and gradients, as these are directly relevant to hazard prediction. In this paper, we present a new formalism for downscaling satellite precipitation observations, which explicitly allows for the preservation of some key geometrical and statistical properties of spatial precipitation. These include sharp intensity gradients (due to high-intensity regions embedded within lower-intensity areas), coherent spatial structures (due to regions of slowly varying rainfall),and thicker-than-Gaussian tails of precipitation gradients and intensities. Specifically, we pose the downscaling problem as a discrete inverse problem and solve it via a regularized variational approach (variational downscaling) where the regularization term is selected to impose the desired smoothness in the solution while allowing for some steep gradients(called 1-norm or total variation regularization). We demonstrate the duality between this geometrically inspired solution and its Bayesian statistical interpretation, which is equivalent to assuming a Laplace prior distribution for the precipitation intensities in the derivative (wavelet) space. When the observation operator is not known, we discuss the effect of its misspecification and explore a previously proposed dictionary-based sparse inverse downscaling methodology to indirectly learn the observation operator from a database of coincidental high- and low-resolution observations. The proposed method and ideas are illustrated in case studies featuring the downscaling of a hurricane precipitation field.

Hurricanes↗
NASA NTRS2013

Hadoop for High-Performance Climate Analytics: Use Cases and Lessons Learned

Scientific data services are a critical aspect of the NASA Center for Climate Simulations mission (NCCS). Hadoop, via MapReduce, provides an approach to high-performance analytics that is proving to be useful to data intensive problems in climate research. It offers an analysis paradigm that uses clusters of computers and combines distributed storage of large data sets with parallel computation. The NCCS is particularly interested in the potential of Hadoop to speed up basic operations common to a wide range of analyses. In order to evaluate this potential, we prototyped a series of canonical MapReduce operations over a test suite of observational and climate simulation datasets. The initial focus was on averaging operations over arbitrary spatial and temporal extents within Modern Era Retrospective- Analysis for Research and Applications (MERRA) data. After preliminary results suggested that this approach improves efficiencies within data intensive analytic workflows, we invested in building a cyber infrastructure resource for developing a new generation of climate data analysis capabilities using Hadoop. This resource is focused on reducing the time spent in the preparation of reanalysis data used in data-model inter-comparison, a long sought goal of the climate community. This paper summarizes the related use cases and lessons learned.

analytics↗
NASA NTRS2017

Application of Machine Learning to Rotorcraft Health Monitoring

Machine learning is a powerful tool for data exploration and model building with large data sets. This project aimed to use machine learning techniques to explore the inherent structure of data from rotorcraft gear tests, relationships between features and damage states, and to build a system for predicting gear health for future rotorcraft transmission applications. Classical machine learning techniques are difficult, if not irresponsible to apply to time series data because many make the assumption of independence between samples. To overcome this, Hidden Markov Models were used to create a binary classifier for identifying scuffing transitions and Recurrent Neural Networks were used to leverage long distance relationships in predicting discrete damage states. When combined in a workflow, where the binary classifier acted as a filter for the fatigue monitor, the system was able to demonstrate accuracy in damage state prediction and scuffing identification. The time dependent nature of the data restricted data exploration to collecting and analyzing data from the model selection process. The limited amount of available data was unable to give useful information, and the division of training and testing sets tended to heavily influence the scores of the models across combinations of features and hyper-parameters. This work built a framework for tracking scuffing and fatigue on streaming data and demonstrates that machine learning has much to offer rotorcraft health monitoring by using Bayesian learning and deep learning methods to capture the time dependent nature of the data. Suggested future work is to implement the framework developed in this project using a larger variety of data sets to test the generalization capabilities of the models and allow for data exploration.

machine learning↗
NASA NTRS2017

Ask-the-expert: Active Learning Based Knowledge Discovery Using the Expert

Often the manual review of large data sets, either for purposes of labeling unlabeled instances or for classifying meaningful results from uninteresting (but statistically significant) ones is extremely resource intensive, especially in terms of subject matter expert (SME) time. Use of active learning has been shown to diminish this review time significantly. However, since active learning is an iterative process of learning a classifier based on a small number of SME-provided labels at each iteration, the lack of an enabling tool can hinder the process of adoption of these technologies in real-life, in spite of their labor-saving potential. In this demo we present ASK-the-Expert, an interactive tool that allows SMEs to review instances from a data set and provide labels within a single framework. ASK-the-Expert is powered by an active learning algorithm for training a classifier in the backend. We demonstrate this system in the context of an aviation safety application, but the tool can be adopted to work as a simple review and labeling tool as well, without the use of active learning.

software↗
NASA NTRS2017

Ask-the-Expert: Active Learning Based Knowledge Discovery Using the Expert

Often the manual review of large data sets, either for purposes of labeling unlabeled instances or for classifying meaningful results from uninteresting (but statistically significant) ones is extremely resource intensive, especially in terms of subject matter expert (SME) time. Use of active learning has been shown to diminish this review time significantly. However, since active learning is an iterative process of learning a classifier based on a small number of SME-provided labels at each iteration, the lack of an enabling tool can hinder the process of adoption of these technologies in real-life, in spite of their labor-saving potential. In this demo we present ASK-the-Expert, an interactive tool that allows SMEs to review instances from a data set and provide labels within a single framework. ASK-the-Expert is powered by an active learning algorithm for training a classifier in the back end. We demonstrate this system in the context of an aviation safety application, but the tool can be adopted to work as a simple review and labeling tool as well, without the use of active learning.

GUI↗
NASA NTRS2018

Using ADOPT Algorithm and Operational Data to Discover Precursors to Aviation Adverse Events

The US National Airspace System (NAS) is making its transition to the NextGen system and assuring safety is one of the top priorities in NextGen. At present, safety is managed reactively (correct after occurrence of an unsafe event). While this strategy works for current operations, it may soon become ineffective for future airspace designs and high density operations. There is a need for proactive management of safety risks by identifying hidden and "unknown" risks and evaluating the impacts on future operations. To this end, NASA Ames has developed data mining algorithms that finds anomalies and precursors (high-risk states) to safety issues in the NAS. In this paper, we describe a recently developed algorithm called ADOPT that analyzes large volumes of data and automatically identifies precursors from real world data. Precursors help in detecting safety risks early so that the operator can mitigate the risk in time. In addition, precursors also help identify causal factors and help predict the safety incident. The ADOPT algorithm scales well to large data sets and to multidimensional time series, reduce analyst time significantly, quantify multiple safety risks giving a holistic view of safety among other benefits. This paper details the algorithm and includes several case studies to demonstrate its application to discover the "known" and "unknown" safety precursors in aviation operation.

aviation safet↗
NASA NTRS2019

Ask-The-Expert: Minimizing Human Review for Big Data Analytics Through Active Learning

In this CIF project, we worked toward semi-automating knowledge discovery from anomaly detection algorithms through the use of active learning. Active learning is an area of research within machine learning that uses an "expert in the loop" to learn from large data sets that have very few annotations or labels available, and where providing such labels is expensive. In our case, the task can be defined as the identification of safety events from flight operational data. Since traditional anomaly detection algorithms cannot differentiate between operationally relevant and irrelevant statistical anomalies, Subject Matter Experts (SMEs) have a lengthy and expensive burden of investigating every example identified by the detection algorithm, classifying and labeling them as relevant or irrelevant. Active learningidentifies the unlabeled example for which a label would most improve the classifier, asks the domain expert for a label, and repeats this process until there are no more resources (time, budget) available for labeling or a minimum required performance is reached. A positive label indicates an operationally significant safety event whereas a negative label indicates otherwise. Based on these few labels we propose to build an active learning system that utilizes the SME's time in the most effective manner by iteratively asking for labels for as few informative instances as possible. Our work was proposed to be a stepping stone toward implementation and deployment of the system with user interface to be pursued by the Aviation Operations and Safety Program (AOSP) given its interest in safety monitoring and discovery of safety incidents.

aviation safety↗
NASA NTRSDate not supplied

Detecting And Characterizing Archetypes of Unintended Consequences in Engineered Systems

When designing engineered systems, the potential for unintended consequences of design policies or design decisions exists despite best intentions. Conditions that might cause the formation of unintended consequences are often known only in hindsight. However, since these conditions are associated with a single event, it is difficult to uncover the general patterns of conditions leading to unintended consequences. In this research, patterns of conditions associated with unintended consequences are learned from historical data and represented in the form of archetypes. While previous work using systems theoretic modeling has identified high-level archetypes, this work leverages a self-organizing map to learn archetypes of unintended consequences from human-tagged risk factors in a large data set of lessons learned from adverse events at NASA. The sixty-six identified archetypes contain patterns of conditions such as complexity and human-machine interaction associated with the formation of unintended consequences. To validate the archetypes, a sample of the archetypes is represented using system dynamics in order to illustrate that the identified archetypes are specialized versions of known high-level archetypes of unintended consequences. While the research is based upon a specific dataset, the archetypes apply to any engineered system and the pattern of leading indicators open a new path to manage unintended consequences and mitigate the magnitude of potentially adverse outcomes.

Hannah S Walsh↗
NASA NTRSDate not supplied

The Ejectable Data Recorder: A Lean, Risk-Informed Approach for Hardware Development

NASA is developing the Orion spacecraft to transport crew from the Earth to the Moon as part of the Artemis series of missions. To provide a crew escape capability from pre-launch through ascent, the Orion vehicle is equipped with a Launch Abort System (LAS), built by Lockheed Martin, which pulls the capsule away from the launch vehicle in the event of an abort scenario. The Ascent Abort 2 (AA-2) test flight occurred on July 2, 2019,and tested a production version of the LAS to ensure that it can operate as intended, and to collect a large data set from hundreds of sensors on the vehicle to support Orion flight certification. In the original AA-2 architecture, a single-string set of communications antennas on the LAS would downlink all of the in-flight test data to ground stations. However, that communications architecture was predicted to have data dropouts during abort and jettison of the LAS, and would not support data transmission at all after LAS jettison. As a result, a comprehensive trade study was completed, yielding the addition of antennas on the crew module (CM), a buffer/rebroadcast capability for key portions of the flight, and an ejectable data recorder (EDR) subsystem. This EDR subsystem would serve as a backup to the radio frequency (RF) communications system, and would be non-flight critical, providing a unique capability that enabled management to take a different approach with the hardware and software development. The Crew Module and Separation Ring were developed as “Class 1”Flight Hardware, albeit with some tailoring approaches to enable efficiencies. The Class 1 designation requires full rigor for flight hardware and software, documenting everything that happens to a piece of hardware from procurement through disposal, requiring a full spectrum of acceptance tests, and the highest rigor of quality assurance processes. At the other end of the spectrum, Class 3hardware is controlled, but not intended for flight, and leaves the level of rigor up to the project manager. This classification is often used for research and development projects. Similarly,Class-1E has been recently defined at NASA for ISS payloads and technology development projects that are not flight critical and do not need the full rigor of Class 1 to be successful. The EDR subsystem was challenged at commencement to adopt a skunkworks and agile-like approach to hardware development, allowing for a different risk posture than the rest of the AA-2 hardware. After initially pursuing Class 1 processes, the EDR subsystem design evolved to incorporating numerous commercial components, leading to re-designation as a Class-1E subsystem. The resulting EDR subsystem was fully successful in meeting all flight system requirements, and achieved 100% retrieval of flight test data. This paper will discuss the risk posture of the EDR subsystem and the subsequent tailoring that was enacted as part of its Class-1E status.

EDR↗
NASA NTRS2019

Machine Learning for the Zwicky Transient Facility

The Zwicky Transient Facility is a large optical survey in multiple filters producing hundreds of thousands of transient alerts per night. We describe here various machine learning (ML) implementations and plans to make the maximal use of the large data set by taking advantage of the temporal nature of the data, and further combining it with other data sets. We start with the initial steps of separating bogus candidates from real ones, separating stars and galaxies, and go on to the classification of real objects into various classes. Besides the usual methods (e.g., based on features extracted from light curves) we also describe early plans for alternate methods including the use of domain adaptation, and deep learning. In a similar fashion we describe efforts to detect fast moving asteroids. We also describe the use of the Zooniverse platform for helping with classifications through the creation of training samples, and active learning. Finally we mention the synergistic aspects of ZTF and LSST from the ML perspective.

Ashish Mahabal↗
NASA NTRSDate not supplied

A Robust Schema for Storing and Managing Machine Learning Data and Models

- Machine Learning (ML) has enabled models that can improve efficiency and decrease computational cost - ML models are crucial in enabling Integrated Computational Materials Engineering (ICME) - Large data sets require robust means of storing ML data and models

Brandon L. Hearley↗