Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Integrated System-Level Optimization for Concurrent Engineering With Parametric Subsystem Modeling

The introduction of concurrent design practices to the aerospace industry has greatly increased the productivity of engineers and teams during design sessions as demonstrated by JPL's Team X. Simultaneously, advances in computing power have given rise to a host of potent numerical optimization methods capable of solving complex multidisciplinary optimization problems containing hundreds of variables, constraints, and governing equations. Unfortunately, such methods are tedious to set up and require significant amounts of time and processor power to execute, thus making them unsuitable for rapid concurrent engineering use. This paper proposes a framework for Integration of System-Level Optimization with Concurrent Engineering (ISLOCE). It uses parametric neural-network approximations of the subsystem models. These approximations are then linked to a system-level optimizer that is capable of reaching a solution quickly due to the reduced complexity of the approximations. The integration structure is described in detail and applied to the multiobjective design of a simplified Space Shuttle external fuel tank model. Further, a comparison is made between the new framework and traditional concurrent engineering (without system optimization) through an experimental trial with two groups of engineers. Each method is evaluated in terms of optimizer accuracy, time to solution, and ease of use. The results suggest that system-level optimization, running as a background process during integrated concurrent engineering sessions, is potentially advantageous as long as it is judiciously implemented.

Schuman, Todd↗

Modeling and Observational Framework for Diagnosing Local Land-Atmosphere Coupling on Diurnal Time Scales

Land-atmosphere interactions play a critical role in determining the diurnal evolution of both planetary boundary layer (PBL) and land surface temperature and moisture states. The degree of coupling between the land surface and PBL in numerical weather prediction and climate models remains largely unexplored and undiagnosed due to the complex interactions and feedbacks present across a range of scales. Further, uncoupled systems or experiments (e.g., the Project for Intercomparison of Land Parameterization Schemes, PILPS) may lead to inaccurate water and energy cycle process understanding by neglecting feedback processes such as PBL-top entrainment. In this study, a framework for diagnosing local land-atmosphere coupling is presented using a coupled mesoscale model with a suite of PBL and land surface model (LSM) options along with observations during field experiments in the U. S. Southern Great Plains. Specifically, the Weather Research and Forecasting (WRF) model has been coupled to the Land Information System (LIS), which provides a flexible and high-resolution representation and initialization of land surface physics and states. Within this framework, the coupling established by each pairing of the available PBL schemes in WRF with the LSMs in LIS is evaluated in terms of the diurnal temperature and humidity evolution in the mixed layer. The co-evolution of these variables and the convective PBL is sensitive to and, in fact, integrative of the dominant processes that govern the PBL budget, which are synthesized through the use of mixing diagrams. Results show how the sensitivity of land-atmosphere interactions to the specific choice of PBL scheme and LSM varies across surface moisture regimes and can be quantified and evaluated against observations. As such, this methodology provides a potential pathway to study factors controlling local land-atmosphere coupling (LoCo) using the LIS-WRF system, which will serve as a testbed for future experiments to evaluate coupling diagnostics within the community.

Santanello, Joseph A., Jr.↗

Performance Analysis of Cloud Computing Architectures Using Discrete Event Simulation

Cloud computing offers the economic benefit of on-demand resource allocation to meet changing enterprise computing needs. However, the flexibility of cloud computing is disadvantaged when compared to traditional hosting in providing predictable application and service performance. Cloud computing relies on resource scheduling in a virtualized network-centric server environment, which makes static performance analysis infeasible. We developed a discrete event simulation model to evaluate the overall effectiveness of organizations in executing their workflow in traditional and cloud computing architectures. The two part model framework characterizes both the demand using a probability distribution for each type of service request as well as enterprise computing resource constraints. Our simulations provide quantitative analysis to design and provision computing architectures that maximize overall mission effectiveness. We share our analysis of key resource constraints in cloud computing architectures and findings on the appropriateness of cloud computing in various applications.

Stocker, John C.↗

The SeaHorn Verification Framework

In this paper, we present SeaHorn, a software verification framework. The key distinguishing feature of SeaHorn is its modular design that separates the concerns of the syntax of the programming language, its operational semantics, and the verification semantics. SeaHorn encompasses several novelties: it (a) encodes verification conditions using an efficient yet precise inter-procedural technique, (b) provides flexibility in the verification semantics to allow different levels of precision, (c) leverages the state-of-the-art in software model checking and abstract interpretation for verification, and (d) uses Horn-clauses as an intermediate language to represent verification conditions which simplifies interfacing with multiple verification tools based on Horn-clauses. SeaHorn provides users with a powerful verification tool and researchers with an extensible and customizable framework for experimenting with new software verification techniques. The effectiveness and scalability of SeaHorn are demonstrated by an extensive experimental evaluation using benchmarks from SV-COMP 2015 and real avionics code.

Model Checking↗

A Diagnosis-Prognosis Feedback Loop for Improved Performance Under Uncertainties

The feed-forward relationship between diagnosis and prognosis is the foundation of both aircraft structural health management and the digital twin concept. Measurements of structural response are obtained either in-situ with mounted sensor networks or offline using more traditional techniques (e.g., nondestructive evaluation). Diagnosis algorithms process this information to detect and quantify damage and then feed this data forward to a prognostic framework. A prognosis of the structure's future operational readiness (e.g., remaining useful life or residual strength) is then made and is used to inform mission- critical decision-making. Years of research have been devoted to improving the elements of this process, but the process itself has not changed significantly. Here, a new approach is proposed in which prognosis information is not only fed forward for decision-making, but it is also fed back to the forthcoming diagnosis. In this way, diagnosis algorithms can take advantage of a priori information about the expected state of health, rather than operating in an uninformed condition. As a feasibility test, a diagnosis-prognosis feedback loop of this manner is demonstrated. The approach is applied to a numerical example in which fatigue crack growth is simulated in a simple aluminum alloy test specimen. A prognosis was derived from a set of diagnoses which provided feedback to a subsequent set of diagnoses. Improvements in accuracy and a reduction in uncertainty in the prognosis- informed diagnoses were observed when compared with an uninformed diagnostic approach.

Leser, Patrick E.↗

National Partnership for Aeronautical Ground Testing - Overview and NASA Perspective

The National Partnership for Aeronautical Testing is a cooperative agreement between NASA and the DoD which serves as a framework for cooperation on individual projects related to aeronautical test facilities, particularly wind tunnels and engine test facilities. The Aeronautics Evaluation and Test Capability is a NASA portfolio which provides resources and coordination for the large wind tunnels and engine test cells at NASA Ames, Glenn, and Langley. This presentation describes the history, organization, and activities of the NPAT, as well as the AETC.

Air Force↗

Ellicott City Disasters II: Enhancing a Statistical Flood Risk Model to Continue Improving Early Warning Systems and Public Safety in Ellicott City, Maryland

As flooding events in the United States grow in frequency and intensity, the use of technological advancements and applied science are increasingly necessary for effective flood monitoring and warning systems. The NASA DEVELOP Ellicott City Disasters II project investigated the use of machine learning for applications in flood risk detection to support the improvement of early warning systems. To strengthen the efforts of the Howard County Office of Emergency Management (OEM) in building a more robust flood monitoring system, the project improved the original statistical flood risk model, FLuME (Flood Learning Model Environment), programmed by the first DEVELOP term. The enhancements incorporated an additional six years of precipitation and soil moisture data from the North American Land Data Assimilation System (NLDAS), modeled using Aqua Advanced Microwave Scanning Radiometer for EOS and Tropical Rainfall Measuring Mission (TRMM) Microwave Imager. These Earth observations were supplemented by stream gauge data from the OEM and the US Geological Survey. The resultant flood risk model FLASH (Flood Learning Environment and Severity Assessment Hub) was trained to evaluate input variables and predict stage height in Ellicott City in real time. The addition of an advanced deep learning framework known as long short-term memory improved the model’s ability to capture relationships between variables. To assess the effectiveness of the new model, FLASH produced a model efficiency metric of 0.99, a significant improvement over the 0.85 value produced by the previous model. The project assisted the OEM in pursuing the integration of open data and NASA Earth observations into a threat matrix capable of informing near real-time decision making.

Disasters↗

Ellicott City Disasters II: Enhancing a Statistical Flood Risk Model to Continue Improving Early Warning Systems and Public Safety in Ellicott City, Maryland

As flooding events in the United States grow in frequency and intensity, the use of technological advancements and applied science are increasingly necessary for effective flood monitoring and warning systems. The NASA DEVELOP Ellicott City Disasters II project investigated the use of machine learning for applications in flood risk detection to support the improvement of early warning systems. To strengthen the efforts of the Howard County Office of Emergency Management (OEM) in building a more robust flood monitoring system, the project improved the original statistical flood risk model, FLuME (Flood Learning Model Environment), programmed by the first DEVELOP term The enhancements incorporated an additional six years of precipitation and soil moisture data from the North American Land Data Assimilation System (NLDAS), modeled using Aqua Advanced Microwave Scanning Radiometer for EOS and Tropical Rainfall Measuring Mission TRMM Microwave Imager. These Earth observations were supplemented by stream gauge data from the OEM and the US Geological Survey. The resultant flood risk model FLASH (Flood Learning Environment and Severity Assessment Hub) was trained to evaluate input variables and predict stage height in Ellicott City in real time. The addition of an advanced deep learning framework known as long short-term memory improved the model’s ability to capture relationships between variables. To assess the effectiveness of the new model, FLASH produced a model efficiency metric of 0.99, a significant improvement over the 0.85 value produced by the previous model. The project assisted the OEM in pursuing the integration of open data and NASA Earth observations into a threat matrix capable of informing near real-time decision making.

Disasters↗

Implementation of Combinatorial Optimization Techniques for Automated Fiber Placement Through Thickness Defect Stack-Up Minimization

The Computer Aided Process Planning (CAPP) module was developed to facilitate and accelerate the process planning workflow for Automated Fiber Placement (AFP). CAPP assists process planners in identifying optimal starting point locations and layup strategies for each ply of a laminate. Ply optimization operates on measurement and scoring of geometry-based defects such as gaps, overlaps, angle deviation, and steering. This paper expands on the established framework for analyzing defect stack-up through thickness of a laminate. Four different combinatorial optimization algorithms are implemented and evaluated: (1) genetic algorithm, (2) differential evolution, (3) particle swarm, and (4) greedy search. The algorithms identify the optimal combination of ply-level layup strategies, by scoring potential laminates on defect stacking, using two different objective functions. A final optimization approach is also presented which trades some performance for a large gain in efficiency. These approaches are compared to a randomized combination using a complex tool surface in a virtual case study. The result is a streamlined methodology for comparing different laminate-level manufacturing strategies and minimizing the through thickness defect stack up.

CAPP↗

Reinforcement Learning for Spacecraft Navigation & Environment Characterization in the Planar-Restricted Two-Body Problem

As science, exploration, and commercial space missions become increasingly complex, so does the need for efficient, autonomous, and integrated spacecraft navigation and operations techniques. Key operational functions, including data collection and transmission, environment characterization, systems constraints, human factors, and navigation, often are intertwined and conflicted. Deep Reinforcement Learning (DRL) offers a framework for addressing integrated spacecraft navigation and planning in an uncertain dynamical environment. The goal of this study is to evaluate the utility of DRL for integrated spacecraft navigation and planning. This is achieved by developing a simple environmental characterization training environment in the Planar-Restricted 2-Body Problem (PR2BP), establishing benchmarks and heuristic baselines, and designing a previously unstudied Markov Decision Process (MDP) formulation. This MDP formulation enables the spacecraft DRL agents to appropriately balance navigation and actuation capabilities. The resulting DRL-derived policy exceeds a random or untrained policy and meets or exceeds the level of performance of a heuristic without actuation. In the process, valuable intuition is gained about the problem with insight into how DRL methods could scale to increasingly more realistic scenarios, including net-work design and training architectures, efficient state space representations, and methods for encouraging exploration in a parametric action space, among others.

navigation↗

Predicting Real-Time Safety of the National Airspace System

To gain the situational awareness necessary for informed decision making regarding avoidance of airspace hazards, each operator must consolidate operations-relevant information from disparate sources and apply extensive domain knowledge to correctly interpret not just the current state of the NAS but forecast its (combined) evolution over the duration of the operation. This time- and workload-intensive process is periodically repeated throughout the operation so that changes can be managed in a timely manner.The imprecision, inaccuracies, inconsistency, and incompleteness of the incoming data further challenges the process. To facilitate informed decision making, this paper presents a model-based framework for the textitautomated real-time monitoring and prediction of possible effects of airspace hazards on the safety of the National Airspace System (NAS). First, hazards to flight are identified and transformed into sms, that is, quantities of interest that could be evaluated based on available data and are predictive of an unsafe event. The sms and associated thresholds that specify when an event transitions from emphsafe to emphunsafe are combined with models of airspace operations and aircraft dynamics. The framework can include any hazard to flight that can be modeled quantitatively. Models can be detailed and complex, or they can be considerably simplifed, as appropriate to the application. Real-time NAS safety monitoring and prediction begins with an estimate of the state of the NAS using the dynamic models. Given the state estimate and a probability distribution of future inputs to the NAS, we can then predict the evolution of the NAS - the future state - and the occurrence of hazards and unsafe events. The entire probability distribution of airspace sms is computed, not just point estimates, without significant assumptions regarding the distribution type andor parameters. We demonstrate our overall approach through a simulated scenario in which we predict the occurrence of some unsafe events and show how these predictions evolve in time as flight operations progress. Predictions accounting for common sources of uncertainty are included and it is shown how the predictions improve in time, become more confident, and change dynamically as new information is made available to the prediction algorithm.

safety↗

A methodology to emulate and evaluate a productive virtual workstation

The Advanced Display and Computer Augmented Control (ADCACS) Program at ACT is sponsored by NASA Ames to investigate the broad field of technologies which must be combined to design a 'virtual' workstation for the Space Station Freedom. This program is progressing in several areas and resulted in the definition of requirements for a workstation. A unique combination of technologies at the ACT Laboratory have been networked to effectively create an experimental environment. This experimental environment allows the integration of nonconventional input devices with a high power graphics engine within the framework of an expert system shell which coordinates the heterogeneous inputs with the 'virtual' presentation. The flexibility of the workstation is evolved as experiments are designed and conducted to evaluate the condition descriptions and rule sets of the expert system shell and its effectiveness in driving the graphics engine. Workstation productivity has been defined by the achievable performance in the emulator of the calibrated 'sensitivity' of input devices, the graphics presentation, the possible optical enhancements to achieve a wide field of view color image and the flexibility of conditional descriptions in the expert system shell in adapting to prototype problems.

Krubsack, David↗

Evaluation of automated decisionmaking methodologies and development of an integrated robotic system simulation, volume 2, part 1. Appendix A: Software documentation

Documentation of the preliminary software developed as a framework for a generalized integrated robotic system simulation is presented. The program structure is composed of three major functions controlled by a program executive. The three major functions are: system definition, analysis tools, and post processing. The system definition function handles user input of system parameters and definition of the manipulator configuration. The analysis tools function handles the computational requirements of the program. The post processing function allows for more detailed study of the results of analysis tool function executions. Also documented is the manipulator joint model software to be used as the basis of the manipulator simulation which will be part of the analysis tools capability.

Lowrie, J. W.↗

Building a Framework to Model Food Security Pressures and Shocks in a Changing World

• An Intersection of Challenges for Food Security • The Agricultural Model Intercomparison and Improvement Project (AgMIP) • Establishing and evaluating agricultural models • Responding to the transient signal of climate change • Understanding food systems and identifying potential for unexpected behaviors • Opportunities For a More Resilient Future

Food Security↗

The Robustness of Acoustic Analogies

Acoustic analogies for the prediction of flow noise are exact rearrangements of the flow equations N(right arrow q) = 0 into a nominal sound source S(right arrow q) and sound propagation operator L such that L(right arrow q) = S(right arrow q). In practice, the sound source is typically modeled and the propagation operator inverted to make predictions. Since the rearrangement is exact, any sufficiently accurate model of the source will yield the correct sound, so other factors must determine the merits of any particular formulation. Using data from a two-dimensional mixing layer direct numerical simulation (DNS), we evaluate the robustness of two analogy formulations to different errors intentionally introduced into the source. The motivation is that since S can not be perfectly modeled, analogies that are less sensitive to errors in S are preferable. Our assessment is made within the framework of Goldstein's generalized acoustic analogy, in which different choices of a base flow used in constructing L give different sources S and thus different analogies. A uniform base flow yields a Lighthill-like analogy, which we evaluate against a formulation in which the base flow is the actual mean flow of the DNS. The more complex mean flow formulation is found to be significantly more robust to errors in the energetic turbulent fluctuations, but its advantage is less pronounced when errors are made in the smaller scales.

Freund, J. B.↗

NASA Engineering and Safety Center Technical Bulletin No 12-01: Simulation Framework for Rapid Entry, Descent, and Landing Analysis

The NASA Engineering and Safety Center has archived a number of key historic Entry, Descent, and Landing (EDL) simulation models and developed several new models to enhance the capability of the Agency to evaluate a wide range of EDL systems for system analysis studies, preliminary design, mission development and execution, and time-critical assessments. The simulation models developed in this activity can be used to help define the required architectures and investment strategies for future robotic and human exploration missions.

Simulation Framework↗

Consideration of environmental noise effects in transportation planning by governmental entities

Environmental concerns are reviewed with respect to major transportation systems: the interstate highway system and commercial air transportation. The type of planning that was done for interstate highway systems is described, and the shift in social value emphasis that has become apparent since the interstate system was authorized is considered. Other topics discussed include the constitutional framework for the allocation of governmental power with respect to transportation systems planning, governmental assessment of the aircraft noise problem, and evaluating the social benefit of noise abatement.

Mayo, L. H.↗

Feasibility study of an Integrated Program for Aerospace-vehicle Design (IPAD) system. Volume 5: Design of the IPAD system. Part 2: System design. Part 3: General purpose utilities, phase 1, task 2

Viable designs are presented of various elements of the IPAD framework software, data base management system, and required new languages in relation to the capabilities of operating systems software. A thorough evaluation was made of the basic systems functions to be provide by each software element, its requirements defined in the conceptual design, the operating systems features affecting its design, and the engineering/design functions which it was intended to enhance.

Garrocq, C. A.↗