Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54

The Porous Microstructure Analysis (PuMA) software

The open-source Porous Microstructure Analysis (PuMA) software was implemented to offer an efficient framework for determining material characteristics from 3D microstructures. Its development was inspired by progress in X-ray microtomography, an imaging technology that captures the internal structure of materials in 3D, and even in a 4D temporal context. Over recent years, this method has transformed the domain of materials science due to its capability to non-destructively examine material microstructures while presenting digital data about their geometrical details. It has provided insights into materials relevant to several NASA missions, including heatshields, parachute fabrics, meteorites, and other advanced composites. PuMA, in its current version 3, delivers an array of features, spanning from basic geometric insights of a microstructure to intricate anisotropic thermo-elastic and chemical behavior. Specifically, the software evaluates morphological attributes (specific surface area, volume fractions, mean intercept lengths, orientation) and physical characteristics (conductivity, elasticity, permeability, and tortuosity). Additionally, it can model material degradation processes, such as oxidation and surface chemistry interactions. The software can generate synthetic microstructures, from straightforward geometrical designs to intricate woven and non-woven geometries. Coupling material generation and characterization enable parametric studies and sensitivity analysis to optimize the microstructural performance and inform design decisions and reliability assessment based on uncertainty quantification. A recent addition to PuMA includes the TomoSAM plugin, devised to incorporate the cutting-edge Segment Anything Model (SAM). SAM is a promptable deep learning model that can identify objects and create image masks in a zero-shot manner, based only on a few user clicks. The synergy between these tools aids in the segmentation of complex 3D datasets from tomography or other imaging techniques, which would otherwise require a laborious manual segmentation process.

Tomography↗

The Porous Microstructure Analysis (PuMA) software

The open-source Porous Microstructure Analysis (PuMA) software was created to offer an efficient framework for determining material properties from 3D microstructures. Its development was inspired by progress in X-ray microtomography, an imaging technology that captures the internal structure of materials in 3D, and even in a 4D temporal context. Over recent years, this method has transformed the domain of materials science due to its capability to non-destructively examine material microstructures while presenting digital data about their geometrical details. It has provided insights into materials relevant to several NASA missions, including heatshields, parachute fabrics, meteorites, and other advanced composites. PuMA, in its current version 3, delivers an array of features, spanning from basic geometric insights of a microstructure to intricate anisotropic thermo-elastic and chemical behavior. Specifically, the software evaluates morphological attributes (specific surface area, volume fractions, mean intercept lengths, orientation) and physical characteristics (conductivity, elasticity, permeability, and tortuosity). Additionally, it can model material degradation processes, such as oxidation and surface chemistry interactions. The software can generate synthetic microstructures, from straightforward geometrical designs to intricate woven and non-woven geometries. Coupling material generation and characterization enables parametric studies and sensitivity analysis to optimize the microstructural performance and inform design decisions and reliability assessment based on uncertainty quantification. A recent addition to PuMA includes the TomoSAM plugin, devised to incorporate the cutting-edge Segment Anything Model (SAM) into our image segmentation workflow. SAM is a promptable deep learning model that can identify objects and create image masks in a zero-shot manner, based only on a few user clicks. The synergy between these tools aids in the segmentation of complex 3D datasets from tomography and other imaging techniques, which would otherwise require a laborious manual segmentation process.

Tomography↗

NASA Systems Engineering Research Consortium: Defining the Path to Elegance in Systems

The NASA Systems Engineering Research Consortium was formed at the end of 2010 to study the approaches to producing elegant systems on a consistent basis. This has been a transformative study looking at the engineering and organizational basis of systems engineering. The consortium has engaged in a variety of research topics to determine the path to elegant systems. In the second year of the consortium, a systems engineering framework emerged which structured the approach to systems engineering and guided our research. This led in the third year to set of systems engineering postulates that the consortium is continuing to refine. The consortium has conducted several research projects that have contributed significantly to the understanding of systems engineering. The consortium has surveyed the application of the NASA 17 systems engineering processes, explored the physics and statistics of systems integration, and considered organizational aspects of systems engineering discipline integration. The systems integration methods have included system exergy analysis, Akaike Information Criteria (AIC), State Variable Analysis, Multidisciplinary Coupling Analysis (MCA), Multidisciplinary Design Optimization (MDO), System Cost Modelling, System Robustness, and Value Modelling. Organizational studies have included the variability of processes in change evaluations, margin management within the organization, information theory of board structures, social categorization of unintended consequences, and initial looks at applying cognitive science to systems engineering. Consortium members have also studied the bidirectional influence of policy and law with systems engineering.

Watson, Michael D.↗

NASA Systems Engineering Research Consortium: Defining the Path to Elegance in Systems

The NASA Systems Engineering Research Consortium was formed at the end of 2010 to study the approaches to producing elegant systems on a consistent basis. This has been a transformative study looking at the engineering and organizational basis of systems engineering. The consortium has engaged in a variety of research topics to determine the path to elegant systems. In the second year of the consortium, a systems engineering framework emerged which structured the approach to systems engineering and guided our research. This led in the third year to set of systems engineering postulates that the consortium is continuing to refine. The consortium has conducted several research projects that have contributed significantly to the understanding of systems engineering. The consortium has surveyed the application of the NASA 17 systems engineering processes, explored the physics and statistics of systems integration, and considered organizational aspects of systems engineering discipline integration. The systems integration methods have included system energy analysis, Akaike Information Criteria (AIC), State Variable Analysis, Multidisciplinary Coupling Analysis (MCA), Multidisciplinary Design Optimization (MDO), System Cost Modeling, System Robustness, and Value Modeling. Organizational studies have included the variability of processes in change evaluations, margin management within the organization, information theory of board structures, social categorization of unintended consequences, and initial looks at applying cognitive science to systems engineering. Consortium members have also studied the bidirectional influence of policy and law with systems engineering.

Watson, Michael D.↗

A reliable algorithm for optimal control synthesis

In recent years, powerful design tools for linear time-invariant multivariable control systems have been developed based on direct parameter optimization. In this report, an algorithm for reliable optimal control synthesis using parameter optimization is presented. Specifically, a robust numerical algorithm is developed for the evaluation of the H(sup 2)-like cost functional and its gradients with respect to the controller design parameters. The method is specifically designed to handle defective degenerate systems and is based on the well-known Pade series approximation of the matrix exponential. Numerical test problems in control synthesis for simple mechanical systems and for a flexible structure with densely packed modes illustrate positively the reliability of this method when compared to a method based on diagonalization. Several types of cost functions have been considered: a cost function for robust control consisting of a linear combination of quadratic objectives for deterministic and random disturbances, and one representing an upper bound on the quadratic objective for worst case initial conditions. Finally, a framework for multivariable control synthesis has been developed combining the concept of closed-loop transfer recovery with numerical parameter optimization. The procedure enables designers to synthesize not only observer-based controllers but also controllers of arbitrary order and structure. Numerical design solutions rely heavily on the robust algorithm due to the high order of the synthesis model and the presence of near-overlapping modes. The design approach is successfully applied to the design of a high-bandwidth control system for a rotorcraft.

Vansteenwyk, Brett↗

Approach for Autonomous Control of Unmanned Aerial Vehicle Using Intelligent Agents for Knowledge Creation

This paper describes the development of a planned approach for Autonomous operation of an Unmanned Aerial Vehicle (UAV). A Hybrid approach will seek to provide Knowledge Generation thru the application of Artificial Intelligence (AI) and Intelligent Agents (IA) for UAV control. The application of many different types of AI techniques for flight will be explored during this research effort. The research concentration will be directed to the application of different AI methods within the UAV arena. By evaluating AI approaches, which will include Expert Systems, Neural Networks, Intelligent Agents, Fuzzy Logic, and Complex Adaptive Systems, a new insight may be gained into the benefits of AI techniques applied to achieving true autonomous operation of these systems thus providing new intellectual merit to this research field. The major area of discussion will be limited to the UAV. The systems of interest include small aircraft, insects, and miniature aircraft. Although flight systems will be explored, the benefits should apply to many Unmanned Vehicles such as: Rovers, Ocean Explorers, Robots, and autonomous operation systems. The flight system will be broken down into control agents that will represent the intelligent agent approach used in AI. After the completion of a successful approach, a framework of applying a Security Overseer will be added in an attempt to address errors, emergencies, failures, damage, or over dynamic environment. The chosen control problem was the landing phase of UAV operation. The initial results from simulation in FlightGear are presented.

Dufrene, Warren R., Jr.↗

Control of Future Air Traffic Systems via Complexity Bound Management

The complexity of the present system for managing air traffic has led to "discreteness" in approaches to creating new concepts: new concepts are created as point designs, based on experience, expertise, and creativity of the proposer. Discrete point designs may be highly successful but they are difficult to substantiate in the face of equally strong substantiation of competing concepts, as well as the state of the art in concept evaluation via simulations. Hybrid concepts may present a compromise - the golden middle. Yet a hybrid of sometimes in principle incompatible concepts forms another point design that faces the challenge of substantiation and validation. We are faced with the need to re-design the air transportation system ab initio. This is a daunting task, especially considering the problem of transitioning from the present system to any fundamentally new system. However, design from scratch is also an opportunity to reconsider approaches to new concept development. In this position paper we propose an approach, Optimized Parametric Functional Design, for systematic development of concepts for management and control of airspace systems, based on optimization formulations in terms of required system functions and states. This reasoning framework, realizable in the context of ab initio system design, offers an approach to deriving substantiated airspace management and control concepts. With growing computational power, we hope that the approach will also yield a methodology for actual dynamic control of airspace

Alexandrov, Natalia↗

Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2

Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.

Sergii Skakun↗

A New Approach to Using a Cloud-Resolving Model to Study the Interactions between Clouds, Precipitation and Aerosols

Recently Grabowski and Khairoutdinov and Randall have proposed and demonstrated, the feasibility of using 2D CRMs (these did not include detailed land processes and used periodic lateral boundary conditions) as 'super-parameterizations or a multi-scale modeling framework MMF' for cloud processes within atmospheric generation circulation models (GCMs). In the MMF, a fine-resolution 2D CRM takes the place of the single-column parameterization that is used in conventional GCMs. An overview of this approach is given by Randall et al.. Since a CRM can explicitly simulate cloud processes at the natural space and time scales of clouddynamical processes, it's possible to compute statistics of cloud fields, including radiances and radar backscatters, directly simulated/produced by the CRM. A seed fund is available at NASA Goddard to build a MMF based on a 2D GCE model and the Goddard fvGCM. A prototype MMF will be developed at end of 2004. The major objectives are to test and evaluate its performance using the GCE model as the super parameterization and compare its results with other super parameterizations, conventional GCMs and observations. Recently, a detailed spectral-bin microphysical scheme was implemented into the Goddard Cumulus Ensemble (WE) model. The GCE model is a CRM and it has been improved and applied to study the precipitation processes in the past two decades. Atmospheric aerosols are also described using number density size-distribution functions. The model is tested by studying the evolution of deep tropical clouds in the west Pacific warm pool region and summertime convection over a mid-latitude continent with different concentrations of CCN: a low 'clean' concentration and a high 'dirty' concentration. The impact of atmospheric aerosol concentration on cloud and precipitation will be investigated. In addition, the similarities and differences between bulk- and spectral-bin microphysics will be presented,

Tao, Wei-Kuo↗

Anisotropic Solution Adaptive Unstructured Grid Generation Using AFLR

An existing volume grid generation procedure, AFLR3, was successfully modified to generate anisotropic tetrahedral elements using a directional metric transformation defined at source nodes. The procedure can be coupled with a solver and an error estimator as part of an overall anisotropic solution adaptation methodology. It is suitable for use with an error estimator based on an adjoint, optimization, sensitivity derivative, or related approach. This offers many advantages, including more efficient point placement along with robust and efficient error estimation. It also serves as a framework for true grid optimization wherein error estimation and computational resources can be used as cost functions to determine the optimal point distribution. Within AFLR3 the metric transformation is implemented using a set of transformation vectors and associated aspect ratios. The modified overall procedure is presented along with details of the anisotropic transformation implementation. Multiple two-and three-dimensional examples are also presented that demonstrate the capability of the modified AFLR procedure to generate anisotropic elements using a set of source nodes with anisotropic transformation metrics. The example cases presented use moderate levels of anisotropy and result in usable element quality. Future testing with various flow solvers and methods for obtaining transformation metric information is needed to determine practical limits and evaluate the efficacy of the overall approach.

Marcum, David L.↗

NASA System Safety Handbook. Volume 2: System Safety Concepts, Guidelines, and Implementation Examples

This is the second of two volumes that collectively comprise the NASA System Safety Handbook. Volume 1 (NASASP-210-580) was prepared for the purpose of presenting the overall framework for System Safety and for providing the general concepts needed to implement the framework. Volume 2 provides guidance for implementing these concepts as an integral part of systems engineering and risk management. This guidance addresses the following functional areas: 1.The development of objectives that collectively define adequate safety for a system, and the safety requirements derived from these objectives that are levied on the system. 2.The conduct of system safety activities, performed to meet the safety requirements, with specific emphasis on the conduct of integrated safety analysis (ISA) as a fundamental means by which systems engineering and risk management decisions are risk-informed. 3.The development of a risk-informed safety case (RISC) at major milestone reviews to argue that the systems safety objectives are satisfied (and therefore that the system is adequately safe). 4.The evaluation of the RISC (including supporting evidence) using a defined set of evaluation criteria, to assess the veracity of the claims made therein in order to support risk acceptance decisions.

Safety Case↗

Global Landslide Hazard Assessment for Situational Awareness (LHASA) Version 2: New Activities and Future Plans

A remote sensing-based system has been developed to characterize the potential for rainfall-triggered landslides across the globe in near real-time. The Landslide Hazard Assessment for Situational Awareness (LHASA) model uses a decision tree framework to combine a static susceptibility map derived from information on slope, rock characteristics, forest loss, distance to fault zones and distance to road networks with satellite precipitation estimates from the Global Precipitation Measurement (GPM) mission. Since 2016, the LHASA model has been providing near real-time and retrospective estimates of potential landslide activity. Results of this work are available at https://landslides.nasa.gov. In order to advance LHASA’s capabilities to characterize landslide hazards and impacts dynamically, we have implemented a new approach that leverages machine learning, new parameters, and new inventories. LHASA 2.0 uses the XGBoost machine learning model to bring in dynamic variables as well as additional static variables to better represent landslide hazard globally. Global rainfall forecasts are also being evaluated to provide a 1-3 day forecast of potential landslide activity. Additional factors such as recent seismicity and burned areas are also being considered to represent the preconditioning or changing interactions with subsequent rainfall over affected areas. A series of parameters are being tested within this structure using NASA’s Global Landslide Catalog as well as many other event-based and multi-temporal inventories mapped by the project team or provided by project partners. In addition to estimates of landslide hazard, LHASA Version 2 will incorporate dynamic estimates of exposure including population, roads and infrastructure to highlight the potential impacts that rainfall-triggered landslides. The ultimate goal of LHASA Version 2.0 is to approximate the relative probabilities of landslide hazard and exposure across different space and time scales to inform hazard assessment retrospectively over the past 20 years, in near real-time, and in the future. In addition to the hazard. This presentation will outline the new activities for LHASA Version 2.0 and present some next steps for this system.

Dalia Kirschbaum↗

GCAP 2.0: A Global 3-D Chemical-Transport Model Framework for Past, Present, and Future Climate Scenarios

This paper describes version 2.0 of the Global Change and Air Pollution (GCAP 2.0) model framework, a one-way offline coupling between version E2.1 of the NASA Goddard Institute for Space Studies (GISS) general circulation model (GCM) and the GEOS-Chem global 3-D chemical-transport model (CTM). Meteorology for driving GEOS-Chem has been archived from the E2.1 contributions to phase 6 of the Coupled Model Intercomparison Project (CMIP6) for the pre-industrial era and the recent past. In addition, meteorology is available for the near future and end of the century for seven future scenarios ranging from extreme mitigation to extreme warming. Emissions and boundary conditions have been prepared for input to GEOS-Chem that are consistent with the CMIP6 experimental design. The model meteorology, emissions, transport, and chemistry are evaluated in the recent past and found to be largely consistent with GEOS-Chem driven by the Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2) product and with observational constraints.

Global Change and Air Pollution (GCAP 2.0) model↗

Non-GEO mobile satellite systems: A risk assessment

Since 1991, The MITRE Corporation has performed several independent evaluations of proposed mobile satellite service (MSS) systems that would operate from low Earth orbit (LEO) or medium Earth orbit (MEO), also known as intermediate circular orbit (ICO). This paper introduces a top level Risk Taxonomy tailored to summarize the technical and programmatic risks that MITRE has identified. In general, as risks are identified and addressed, a system's technical characteristics, cost and schedule are affected. This paper traces changes in three key parameters - satellite launch mass, system cost, and system schedule - for each of the five original non-GEO MSS systems for which license applications were made to the U.S. Federal Communications Commission (FCC) from November 1990 until June 1991. Finally, specific risk areas are identified using the Risk Taxonomy as a framework for discussion.

Gaffney, Leah M.↗

Figures of Merit for Control Verification

This paper proposes a methodology for evaluating a controller's ability to satisfy a set of closed-loop specifications when the plant has an arbitrary functional dependency on uncertain parameters. Control verification metrics applicable to deterministic and probabilistic uncertainty models are proposed. These metrics, which result from sizing the largest uncertainty set of a given class for which the specifications are satisfied, enable systematic assessment of competing control alternatives regardless of the methods used to derive them. A particularly attractive feature of the tools derived is that their efficiency and accuracy do not depend on the robustness of the controller. This is in sharp contrast to Monte Carlo based methods where the number of simulations required to accurately approximate the failure probability grows exponentially with its closeness to zero. This framework allows for the integration of complex, high-fidelity simulations of the integrated system and only requires standard optimization algorithms for its implementation.

Crespo, Luis G.↗

Development of Thermal Protection Materials for Future Mars Entry, Descent and Landing Systems

Entry Systems will play a crucial role as NASA develops the technologies required for Human Mars Exploration. The Exploration Technology Development Program Office established the Entry, Descent and Landing (EDL) Technology Development Project to develop Thermal Protection System (TPS) materials for insertion into future Mars Entry Systems. An assessment of current entry system technologies identified significant opportunity to improve the current state of the art in thermal protection materials in order to enable landing of heavy mass (40 mT) payloads. To accomplish this goal, the EDL Project has outlined a framework to define, develop and model the thermal protection system material concepts required to allow for the human exploration of Mars via aerocapture followed by entry. Two primary classes of ablative materials are being developed: rigid and flexible. The rigid ablatives will be applied to the acreage of a 10x30 m rigid mid L/D Aeroshell to endure the dual pulse heating (peak approx.500 W/sq cm). Likewise, flexible ablative materials are being developed for 20-30 m diameter deployable aerodynamic decelerator entry systems that could endure dual pulse heating (peak aprrox.120 W/sq cm). A technology Roadmap is presented that will be used for facilitating the maturation of both the rigid and flexible ablative materials through application of decision metrics (requirements, key performance parameters, TRL definitions, and evaluation criteria) used to assess and advance the various candidate TPS material technologies.

Cassell, Alan M.↗

Relative Contributions of Mean-State Shifts and ENSO-Driven Variability to Precipitation Changes in a Warming Climate

The El Nino-Southern Oscillation (ENSO) is an important driver of regional hydroclimate variability through far-reaching teleconnections. This study uses simulations performed with Coupled General Circulation Models (CGCMs) to investigate how regional precipitation in the 21st century may be affected by changes in both ENSO-driven precipitation variability and slowly-evolving mean rainfall. First, a dominant, time-invariant pattern of canonical ENSO variability (cENSO) is identified in observed SST data. Next, the fidelity with which 33 state-of-the-art CGCMs represent the spatial structure and temporal variability of this pattern (as well as its associated precipitation responses) is evaluated in simulations of 20th century climate change. Possible changes in both the temporal variability of this pattern and its associated precipitation teleconnections are investigated in 21st century climate projections. Models with better representation of the observed structure of the cENSO pattern produce winter rainfall teleconnection patterns that are in better accord with 20th century observations and more stationary during the 21st century. Finally, the model-predicted 21st century rainfall response to cENSO is decomposed into the sum of three terms: 1) the 21st century change in the mean state of precipitation; 2) the historical precipitation response to the cENSO pattern; and 3) a future enhancement in the rainfall response to cENSO, which amplifies rainfall extremes. By examining the three terms jointly, this conceptual framework allows the identification of regions likely to experience future rainfall anomalies that are without precedent in the current climate.

Celine J W Bonfils↗

Structured Overlapping Grid Simulations of Contra-rotating Open Rotor Noise

Computational simulations using structured overlapping grids with the Launch Ascent and Vehicle Aerodynamics (LAVA) solver framework are presented for predicting tonal noise generated by a contra-rotating open rotor (CROR) propulsion system. A coupled Computational Fluid Dynamics (CFD) and Computational AeroAcoustics (CAA) numerical approach is applied. Three-dimensional time-accurate hybrid Reynolds Averaged Navier-Stokes/Large Eddy Simulation (RANS/LES) CFD simulations are performed in the inertial frame, including dynamic moving grids, using a higher-order accurate finite difference discretization on structured overlapping grids. A higher-order accurate free-stream preserving metric discretization with discrete enforcement of the Geometric Conservation Law (GCL) on moving curvilinear grids is used to create an accurate, efficient, and stable numerical scheme. The aeroacoustic analysis is based on a permeable surface Ffowcs Williams-Hawkings (FW-H) approach, evaluated in the frequency domain. A time-step sensitivity study was performed using only the forward row of blades to determine an adequate time-step. The numerical approach is validated against existing wind tunnel measurements.

Grid↗