Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Decision making”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

PI-in-a-box: Intelligent onboard assistance for spaceborne experiments in vestibular physiology

In construction is a knowledge-based system that will aid astronauts in the performance of vestibular experiments in two ways: it will provide real-time monitoring and control of signals and it will optimize the quality of the data obtained, by helping the mission specialists and payload specialists make decisions that are normally the province of a principal investigator, hence the name PI-in-a-box. An important and desirable side-effect of this tool will be to make the astronauts more productive and better integrated members of the scientific team. The vestibular experiments are planned by Prof. Larry Young of MIT, whose team has already performed similar experiments in Spacelab missions SL-1 and D-1, and has experiments planned for SLS-1 and SLS-2. The knowledge-based system development work, performed in collaboration with MIT, Stanford University, and the NASA-Ames Research Center, addresses six major related functions: (1) signal quality monitoring; (2) fault diagnosis; (3) signal analysis; (4) interesting-case detection; (5) experiment replanning; and (6) integration of all of these functions within a real-time data acquisition environment. Initial prototyping work has been done in functions (1) through (4).

Colombano, Silvano↗

Autonomous Ocean World Exploration: Advancement of a Virtual Testbed

The search for life (extinct or extant) and potentially habitable bodies in our solar system and beyond is one of the 12 priority science questions outlined in the National Acadamies’ 2022 decadal survey [5]. Extraterrestrial destinations containing liquid water present an opportunity to search for life as we know it, and in recent years an increasing number of such locations have been discovered within our solar system. Several Jovian moons—Europa, Ganymede, and Callisto [10]—and the Saturnian moons Enceladus [8] and Titan [9] are known or suspected to harbor massive subsurface oceans. Of these "ocean worlds", Europa is the focus of at least one planned NASA orbiter mission, Europa Clipper [4], and an early lander mission concept, the Europa Lander [2, 3]. Whereas most robotic missions to the Moon and Mars (e.g. orbiters, rovers, landers) to date have had ground controllers on Earth tightly involved in mission operations, missions to more distant worlds will require a high degree of onboard autonomy due to long communication lags and blackouts, harsh environments (radiation, cold), and more limited battery and hardware life. The past decade has seen great advances in both AI technologies and computing scalability and performance that offer promising solutions for spacecraft autonomy and motivate the software system and research programs described in this paper. The Ocean Worlds Autonomy Testbed for Exploration, Research, and Simulation (OceanWATERS) [1], which has been in development at the NASA Ames Research Center since 2018, is a virtual environment for testing lander autonomy solutions. It is built on the Robot Operating System (ROS), runs on consumer-grade Linux workstations, and was released as open source in 2020. OceanWATERS provides a physical and visual simulation of a prototypical lander in a Europa-like environment (Figure 1). The lander was modeled after requirements and specifications made in JPL’s Europa Lander Study of 2016 [3]. Simulated lander systems include stereo cameras and spotlights mounted on an antenna mast that pans and tilts, a 6 degrees of freedom (DoF) robotic arm with a force-torque sensor and two interchangeable end effectors, and a battery pack power system. The environment consists of multiple terrain models including a highly detailed model sourced from the FROST dataset [11], simulation of surrounding planetary bodies based on an ephemeris model, and lighting from the sun with associated surface illumination, reflectance, and shadows. Operations supported by OceanWATERS include panoramic and directed imaging of the environment and lander workspace, Cartesian and joint-level arm commanding, grinding of the terrain surface (e.g. digging a trench), and scooping of ground material (Figure 2) which can be discarded or collected as science samples in a receptacle that can be emptied (science operations themselves are not simulated). These operations are realized as ROS Actions and are complimented by a wide selection of telemetry that is continually produced by each lander subsystem. The power system model is driven by the open-source Generic Software Architecture for Prognostics (GSAP) [11] that predicts the battery’s remaining useful life and other characteristics. As a testbed for high-level autonomy, OceanWATERS provides an execution framework based on PLEXIL [12], an open-source plan specification language and execution engine developed largely at Ames. NASA's initial development of OceanWATERS, as well the Ocean Worlds Lander Autonomy Testbed (OWLAT) [6], a complimentary physical testbed developed at JPL, was the first step in a plan for realizing candidate onboard autonomy solutions for such planetary landers. In 2020 NASA solicited applications for its Autonomous Robotics Research for Ocean Worlds (ARROW) program, and in 2021 the similar Concepts for Ocean worlds Life Detection Technology (COLDTech) program. Collectively six research teams, based in universities and companies across the United States, were awarded grants to develop and demonstrate autonomy solutions on OceanWATERS and OWLAT. These 1–2-year projects have now finished or are nearing completion, and a wide variety of autonomy challenges in ocean world surface missions were addressed. Prototyped and demonstrated solutions have included autonomous discovery, response and adaptation to system faults and unexpected environmental events, world model synthesis through perception, plan synthesis using learned models, methods to optimize sample target selection and prioritize science data transmission, extension of PLEXIL for stochastic decision-making, and an integration of a model of JPL’s mission-ready COLDArm [7]. Technologies used in these projects include many forms of machine learning, causal reasoning, automated planning, Markov decision processes, formal methods, and other advanced techniques. A more detailed summary of the ARROW and COLDTech projects is given herein. OceanWATERS has had significant enhancements since its open-source release in 2020. Many of its new features were driven or shaped by feedback from the ARROW and COLDTech teams and requirements of their projects. In support of enabling autonomous adaptation to spacecraft faults (a specific capability solicited by both programs), a fault injection and detection framework was developed that supports a wide and growing range of fault types such as locked joints, image loss, and battery failures. The power system model was completed and integrated into the simulator, starting as a single-cell battery model and later upgraded to a multi-cell model with associated faults such as cell disconnection. Arm/terrain interaction was improved by adding a force-torque sensor and associated faults, and an analytic dig force model based on the Balovnev bucket force equations. Environment fidelity was increased by modeling terrain deformation resulting from digging and scooping; visual improvements were made in textures, lighting, and shadows. To facilitate interoperation with OWLAT, a unified command and telemetry interface between the testbeds was developed at the ROS level, along with a PLEXIL interface. The number of lander operations was greatly expanded (e.g. with Cartesian-based arm and antenna movement), and a framework was designed for users to build their own lander actions. A GUI for PLEXIL plan selection was created (Figure 3), and an expansive set of plans were added, such as those that illustrate patterns for fault handling. This paper provides a self-contained high-level description of OceanWATERS, focusing on more detailed coverage of the aforementioned enhancements. It provides a high-level summary of the projects undertaken by participants in the ARROW and COLDTech programs and how these efforts have helped shape OceanWATERS. Finally, potential future work and directions for the testbed are listed, as likely informed by the recent planetary science decadal survey [5].

K Michael Dalal↗

The Effect of Corporate Influence in the Short Haul Business Travel Market

The importance of corporate involvement in the decision making process for business related air travel is being increasingly recognised in the literature. Business travellers consume air services (i.e. they take airline flights), however; they may not be the principal decision-maker in the purchase. Also it is the organization that employs the traveller that incurs die cost for air travel. Consequently this research addresses the relationship between the traveller and the employing organisation in the purchase of air travel. In this paper traveller opinions on their corporate travel policy are evaluated using a Likert summated rating scale. The benefits sought, by the traveller, from the air service are also investigated and these benefits are used to segment the short haul business air travel market in the EU. Changes in the market for short haul business travel since the full liberalisation of the aviation market in-the EU are evaluated by comparing the data to an earlier study of similar travellers in 1992.

Mason, Keith J.↗

Using Remote Sensing Data and Research Results for Urban Heat Island Mitigation

This paper provides information on the characteristics of the urban heat island, research designed to provide the data needed to develop effective urban heat island reduction strategies, and the development of local working groups to develop implementation plans. As background, an overview of research results on the urban heat island phenomenon and the resultant effect on energy usage and air quality will be explored. The use of more reflective roofing materials, paving materials, tree planting, and other initiatives will be explored as a basis for strategies to mitigate urban heat islands and improve the urban environment. Current efforts to use aircraft remote sensing data in Atlanta, Baton Rouge, Sacramento, and Salt Lake City and our work with non-profit organizations designated to lead public education and strategic development efforts will be presented. Efforts to organize working groups comprised of key stakeholders, the process followed in communicating research results, and methodology for soliciting feedback and incorporating ideas into local plans, policies and decision-making will be discussed. Challenges in developing and transferring data products and research results to stakeholders will be presented. It is our ultimate goal that such efforts be integrated into plans and/or decision models that encourage sustainable development.

Estes, Maury↗

The Effect of Corporate Influence in the Short Haul Business Travel Market

The importance of corporate involvement in the decision making process for business related air travel is being increasingly recognized in the literature. Business travellers consume air services (i.e. they take airline flights), however; they may not be the principal decision-maker in the purchase, Also it is the organization that employs the traveller that incurs the cost for air travel, Consequently this research addresses the relationship between the traveller and the employing organization in the purchase of air travel. In this paper traveller opinions on their corporate travel policy are evaluated using a Likert summated rating scale. The benefits sought, by the traveller, from the air service are also investigated and these benefits are used to segment the short haul business air travel market in the EU. Changes in the market for short haul business travel since the full liberalisation of the aviation market in the EU are evaluated by comparing the data to an earlier study of similar travellers in 1992.

Mason, Keith J.↗

Faults Discovery By Using Mined Data

Fault discovery in the complex systems consist of model based reasoning, fault tree analysis, rule based inference methods, and other approaches. Model based reasoning builds models for the systems either by mathematic formulations or by experiment model. Fault Tree Analysis shows the possible causes of a system malfunction by enumerating the suspect components and their respective failure modes that may have induced the problem. The rule based inference build the model based on the expert knowledge. Those models and methods have one thing in common; they have presumed some prior-conditions. Complex systems often use fault trees to analyze the faults. Fault diagnosis, when error occurs, is performed by engineers and analysts performing extensive examination of all data gathered during the mission. International Space Station (ISS) control center operates on the data feedback from the system and decisions are made based on threshold values by using fault trees. Since those decision-making tasks are safety critical and must be done promptly, the engineers who manually analyze the data are facing time challenge. To automate this process, this paper present an approach that uses decision trees to discover fault from data in real-time and capture the contents of fault trees as the initial state of the trees.

Lee, Charles↗

NASA’s Safety, Reliability, and Mission Assurance Digital Future

The evolution from “document-centric” to “data-centric” and “model-centric” information leveraging structured data and model-based approaches is at the heart of digital engineering transformational efforts underway across industry and government. It is these approaches that pave the way for data lakes, Authoritative Sources of Truth (ASOTs), and systems- of-systems interoperability and the corresponding transformational benefits thereof. Such benefits include increased data availability, data access equity, data traceability, real-time analytics, batch analytics, and (most importantly) acceleration of the time-to-value and time-to-insights associated with engineering products and analyses. The longer-term benefits of reusability, customization and traceability are even more promising. For Safety and Mission Assurance (SMA), and Mission Success (SMS) activities; realization of such benefits is essential to provide engineers and analysts alike vital information when needed to support critical decision making throughout the entire life cycle. The SMA community often operate in parallel with engineering activities, for which information exchange with relevant context is paramount. Far too often, such information lags key decision points and/or is absent of the robust, integrated, knowledge needed, given inherent barriers associated with traditional document-centric means to data sharing, analysis, and reporting. This paper provides an overview of how NASA’s Office of Safety and Mission Assurance (OSMA) is evolving its policies, standards, guidance, and training to transform to eliminate such barriers, thus realizing the benefits emerging in this new digital era. A roadmap for achieving this digital future is presented along with key building blocks involving use and implementation of concepts such as: Objectives-Hierarchies, Objective-Driven Requirements, Accepted Standards, Safety and Assurance Cases, data digitization (i.e., ontologies, structured data, and model-centric data), FAIR (Findable, Accessible, Interoperable, & Reusable) and/or FAIRUST (Findable, Accessible, Interoperable, Reusable, Understandable, Secure, and Trusted) principles [1]. This paper also describes how OSMA, leveraging the Agency’s overall commitment to Digital Transformation (DT), is using the power of Policy, “Digital” Domain representation, Product Evolution, and Community Outreach and Engagement as part of a strategic vision and roadmap to evolve and transform its SMA organizations to become better able to serve its stakeholders and customers. Future publications will elaborate on these building blocks and deeper concepts.

Authoritative Source of Truth (ASOT),↗

Modeling, analysis, and optimization of complex nuclear processes and facilities via computational methods: The HALEU process case study

Improving and adapting industrial systems to timely meet changing programmatic and market demands is an important goal to achieve, including when operating and maintaining complex nuclear processes and facilities. However, changes to these complex systems are costly, particularly when they are already in place and bounded to stringent requirements and constraints such as when handling radioactive material and contaminated equipment. These conditions often exist when treating spent nuclear fuel remotely within shielded nuclear radiation chambers, commonly referred as hot cells, to condition nuclear material and/or fabricate products for utilization in other nuclear enterprises such as in the manufacture of advanced nuclear fuel. The illustrative case considered here is the production of high assay low enriched uranium (HALEU) products supporting the deployment of advanced nuclear reactors. For the HALEU program, resources invested were and are being systematically analyzed so that these investments are maximized in a facility that is nearly 60 years old. A methodology that has effectively enabled optimized and improvements in the Spent Fuel Treatment (SFT) program, and consequently the HALEU program, involves discrete event simulation as addressed in this article. Here, the quantification of multiple productivity metrics, including material processing rates, cycle times, bottlenecks, number of material transfers as well as equipment, workstation, and material handling utilization, has resulted in a myriad of diverse discoveries and data-informed decisions regarding process layout and constituent, labor levels and schedules, selection of new process units, storage needs, and other critical process configurations. This article describes such a computational capability being applied for decision-making, illustrates its application to an actual process and program, provides illustrative results, and argues how computational methods for the modeling, analysis, and optimization of complex processes and facilities does lead to informed decisions derived from data and not only from intuition.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Modeling the evolution of the US opioid crisis for national policy development

The opioid crisis remains one of the greatest public health challenges in the United States. The crisis is complex, with long delays and feedbacks between policy actions and their effects, which creates a risk of unintended consequences and complicates policy decision-making. We present SOURCE (Simulation of Opioid Use, Response, Consequences, and Effects), an operationally detailed national-level model of the opioid crisis, intended to enhance understanding of the crisis and guide policy decisions. Drawing on multiple data sources, SOURCE replicates how risks of opioid misuse initiation and overdose have evolved over time in response to behavioral and other changes and suggests how those risks may evolve in the future, providing a basis for projecting and analyzing potential policy impacts and solutions.

59 BASIC BIOLOGICAL SCIENCES↗

Intelligent data presentation for real-time spacecraft monitoring

This paper describes an intelligent user interface that is currently under development. The interface serves as a front end for real-time spacecraft monitoring software. The software operates under circumstances in which neither an intelligent human without automated assistance nor an automated system without intelligence are sufficiently effective. The user interface is supported by dynamic trade-off evaluation (DTE), a new technique that has been developed to automate general types of performance trade-offs in real-time problem solving systems. In this application, DTE is used to perform dynamic input data management for the purpose of determining which input data should be monitored in time constrained situations and how to present the monitoring information to a human analyst who has the ultimate responsibility for the spacecraft. This application has demonstrated that DTE can be used to dynamically vary the data that is monitored, making it possible to detect and correctly analyze all anomalous data by examining only a subset of the total input data. In carefully structured experimental evaluations that use real spacecraft data and real decision making, DTE provides the ability to handle a three-fold increase in input data (in real-time) without loss of performance and to intelligently present the information to a mission analyst.

Schwuttke, U. M.↗

Computational Evaluation of Inlet Distortion on an Ejector Powered Hybrid Wing Body at Takeoff and Landing Conditions

Due to the aft, upper surface engine location on the Hybrid Wing Body (HWB) planform, there is potential to shed vorticity and separated wakes into the engine when the vehicle is operated at off-design conditions and corners of the envelope required for engine and airplane certification. CFD simulations were performed of the full-scale reference propulsion system, operating at a range of inlet flow rates, flight speeds, altitudes, angles of attack, and angles of sideslip to identify the conditions which produce the largest distortion and lowest pressure recovery. Pretest CFD was performed by NASA and Boeing, using multiple CFD codes, with various turbulence models. These data were used to make decisions regarding model integration, characterize inlet flow distortion patterns, and help define the wind tunnel test matrix. CFD was also performed post-test; when compared with test data, it was possible to make comparisons between measured model-scale and predicted full-scale distortion levels. This paper summarizes these CFD analyses.

Wind Tunnel Testing↗

Medical Data Architecture Prototype Development - Summary of Recent Work and Proposed Ideas for Upcoming Work

The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions, and in ExMC Gap Med 10: We do not have the capability to provide computed medical decision support during exploration missions. These gaps recognize the need for a comprehensive medical data management system and the accompanying computational support to provide autonomous medical care during long duration exploration missions. As the MDA matures—including the capability to comprehensively process and discover medically-relevant information to support medical operations during exploration missions—project focus will shift to maturing and extending the MDA platform to enable clinical decision support and real-time guidance. To date, the MDA foundational architecture has recommended exploration medical system Level of Care IV requirements through a series of test bed prototype developments and analog demonstrations. The next stage in the development will focus on more autonomous clinical decision making necessary to address challenges in executing a self-contained medical system that enables health care both with and without assistance from ground support. A thorough understanding of current state of medical decision support systems, advanced machine learning algorithms and vast and varied data sources is required. The development of a clinical decision support for exploration missions (Level of Care V) roadmap is needed: one that assesses of current state of the art of clinical decision support systems (CDSS), interoperability issues, identification of challenges in health and performance monitoring, obtaining and processing information from biosensors, knowledge and data management, data integration and fusion, and advanced algorithm development. This roadmap must also include rapid prototype development in the areas of data processing, advanced analysis and prediction of medical events, and treatment based on medically relevant information processing and evidence-based best practices. In this presentation, an overview of the relevant issues and the beginning framework of a Level of Care V CDSS development roadmap will be provided.

Krihak, M.↗

Algorithmic considerations of integrated design for CSI on a hypercube architecture

An approach is presented to the integrated design problem for actively controlled large, flexible mechanical systems for which Control Structure Interaction (CSI) problems are of concern. The two coupled design problems were identified as the optimal Structural Design problem the optimal Controller Design problem. These two problems can be addressed within a decision making loop that would consider each separately, and then sequentially analyze the effects of one on the other. Embedded in such a loop would be the simulation and coordination tasks as part of the decision tools required in a total (software) package. All of the above are compute-intensive tasks. In any such task, possible decompositions and gains due to the inherent parallelism have to be exploited. The problems under consideration, as applied to large flexible mechanical structures, are particularly suited to be mapped onto multicomputer systems in a hypercube topology.

Oezguener, UE.↗

Statistical Engineering in Air Traffic Management Research

NASA is working to develop an integrated set of advanced technologies to enable efficient arrival operations in high-density terminal airspace for the Next Generation Air Transportation System. This integrated arrival solution is being validated and verified in laboratories and transitioned to a field prototype for an operational demonstration at a major U.S. airport. Within NASA, this is a collaborative effort between Ames and Langley Research Centers involving a multi-year iterative experimentation process. Designing and analyzing a series of sequential batch computer simulations and human-in-the-loop experiments across multiple facilities and simulation environments involves a number of statistical challenges. Experiments conducted in separate laboratories typically have different limitations and constraints, and can take different approaches with respect to the fundamental principles of statistical design of experiments. This often makes it difficult to compare results from multiple experiments and incorporate findings into the next experiment in the series. A statistical engineering approach is being employed within this project to support risk-informed decision making and maximize the knowledge gained within the available resources. This presentation describes a statistical engineering case study from NASA, highlights statistical challenges, and discusses areas where existing statistical methodology is adapted and extended.

Wilson, Sara R.↗

Improving Trust in Deep Neural Networks with Nearest Neighbors

Deep neural networks are used increasingly for perception and decision-making in UAVs. For example, they can be used to recognize objects from images and decide what actions the vehicle should take. While deep neural networks can perform very well at complex tasks, their decisions may be unintuitive to a human operator. When a human disagrees with a neural network prediction, due to the black box nature of deep neural networks, it can be unclear whether the system knows something the human does not or whether the system is malfunctioning. This uncertainty is problematic when it comes to ensuring safety. As a result, it is important to develop technologies for explaining neural network decisions for trust and safety. This paper explores a modification to the deep neural network classification layer to produce both a predicted label and an explanation to support its prediction. Specifically, at test time, we replace the final output layer of the neural network classifier by a k-nearest neighbor classifier. The nearest neighbor classifier produces 1) a predicted label through voting and 2) the nearest neighbors involved in the prediction, which represent the most similar examples from the training dataset. Because prediction and explanation are derived from the same underlying process, this approach guarantees that the explanations are always relevant to the predictions. We demonstrate the approach on a convolutional neural network for a UAV image classification task. We perform experiments using a forest trail image dataset and show empirically that the hybrid classifier can produce intuitive explanations without loss of predictive performance compared to the original neural network. We also show how the approach can be used to help identify potential issues in the network and training process.

Lee, Ritchie↗

Development of Analysis Methods that Integrate Numeric and Textual Equipment Reliability Data

Within the Light Water Reactor Sustainability (LWRS) program, the Risk-Informed Systems Analysis (RISA) Pathway is performing collaborative research on the development and deployment of technologies designed to assist operating nuclear power plants (NPPs) to reduce operating costs improve plant reliability and availability. One of the RISA research areas is focusing on the development of methods and tools designed to optimize plant operations (e.g., maintenance/replacement schedules, optimal maintenance postures for plant structures, systems, and components [SSCs]) in a manner that is more cost effective than current approaches and makes better use of available SSC health data. The Risk-Informed Asset Management (RIAM) project targets this research area by creating a direct bridge between component equipment reliability (ER) data and system engineer decision making regarding maintenance activity scheduling and component aging management. In this respect, one challenge that NPP system engineers are facing is that the amount of ER data being continuously generated is not only extremely large in size, but it comes in different forms: textual (e.g., condition or maintenance reports) and numeric (e.g., generated by monitoring systems). All these data elements provide them with valuable insights and information regarding: 1) the discovery of anomalous behaviors or degradation trends, 2) the identification of the possible causes behind such behaviors/trends, and 3) the prediction of their direct consequences. However, several challenges have proved to be roadblocks to this process. While some of these challenges are technical in nature (i.e., data are often distributed over several physical servers/databases), others are conceptual in nature: data elements come in different formats (e.g., numeric or textual), and measured values have different scales (e.g., vibration spectra and oil temperature). The activities performed by the RIAM project during FY23 directly tackles the need to simultaneously integrate the analysis of ER data in all its forms, numeric and textual. Note that such task has never been performed before due to the complexity of the systems under consideration but, most importantly, because of the technical challenges behind the harmonization of ER data formats and the lack of adequate computational methods to analyze them. Our approach borrows ideas and concepts from the medical field where integration of several data sources is vital to assist medical practitioners to perform correct diagnosis and indicate optimal treatments. In our view a NPP asset is equivalent to a patient in a medical context. The main difference is the complexity of a human body is a magnitude more complex when compared to typical assets commonly present in NPPs (e.g., centrifugal pumps, or motor operated valves). This simplifies our first requirement when analyzing heterogenous ER data formats: to put data into “context”. Context is here intended as the additional piece of information that is needed by ER data analysis tools to understand what these data elements are referring to, i.e., which king of knowledge they are generating. In our context, this knowledge can be translated into models that capture the form and functional architecture of assets/systems, their dependencies, and how they interact. These models actually emulate the knowledge that that NPP system engineers possess about assets and systems; this is their key of success when analyzing ER data, their challenge is ability to handle large amount of data. Here, we employ model-based system engineering (MBSE) models of systems and assets to represent and capture their architecture and functional, i.e. cause-effect, relations. Then, ER data elements are processed by identifying first of all which elements of the developed MBSE elements they are referring to. For numeric ER data this task is fairly easy since it is possible to precisely pinpoint what MBSE elements the corresponding sensor are observing (e.g., bearing temperature of a centrifugal pump). Task is much harder for textual data since the information contained in issue or maintenance reports needs to “be understood” by a computational tool. Here we called this process as “knowledge extraction”. Once again, we borrow the experience in the medical field where methods to extract knowledge from textual data have been developed in the past decade. The missing element for us is the availability of a complete dictionary of NPP related concepts (in addition to the MBSE models presented earlier) that can put “text into context”. In FY23, such dictionary has been developed along with all the computational elements required for knowledge extraction. Lastly, once numeric and textual ER data elements have been processed and “understood”, then the last step is the discovery of possible cause-effect relations among them. This is performed by observing if a logical connection through the MBSE models exists, and if the

97 MATHEMATICS AND COMPUTING↗

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

On top of machine learning (ML) models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Towards this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. In conclusion, two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage (case study 1) and the remaining useful life prediction of turbofan engines (case study 2).

97 MATHEMATICS AND COMPUTING↗