Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supervised”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Unreal Engine Testbed for Computer Vision of Tall Lunar Tower Assembly

The Tall Lunar Tower project at the NASA Langley Research Center is focused on the design, modeling, fabrication, and testing of a supervised autonomously assembled engineering development unit for tall lunar towers. The lunar south pole environment poses many challenges for robotic assembly of the tall tower, particularly to computer vision camera systems due to a high-contrast lighting environment. This paper will present an Unreal Engine 5 video game engine Lunar South Pole Lighting Testbed to simulate realistic lunar lighting conditions for synthetic image generation. The fidelity of the simulation environment is investigated by comparing the accuracy of computer vision models trained using synthetic image data and trained from real image data collected in a lunar analog environment.

Unreal Engine↗

iMETRO (Integrated Mobile Evaluation Testbed for Robotics Operations) Facility

Exploration crew time in space is precious – every hour could yield immense scientific discovery. However, overhead tasks such as logistics, maintenance, and assembly greatly limit crew time available for science and exploration. Robotic remote operations capabilities offer a solution, but operating mobile dexterous robots in human centered environments presents many unknowns and challenges, therefore testing is needed. iMETRO is a NASA JSC robotics test facility for terrestrial robotic technology adaptation for space exploration use cases, including logistics, maintenance, and science utilization. iMETRO focuses on Intra-Vehicular task environments, such as surface habitats, pressurized rover cabins, and space station modules (both Gateway & LEO). Its goal is to advance the Technology Readiness Levels (TRL) of remote space robot operations systems with Earth supervision.

Robotics↗

Ch. 12. A Theoretical Approach to Management of Limited Attentional Resources to Support the m:N Operation in Advanced Air Mobility Ecosystem

Advanced air mobility (AAM) technologies incorporate increasingly autonomous systems that allow fully remote, independent, and intelligent operation of air vehicles to support the transportation of goods and passengers within and across urban and rural areas. With a myriad of automated technologies enabling the AAM ecosystem, the human operator’s role will likely be a passive supervisory monitor of the air vehicles, involving increasingly fewer humans (m) that manage many more autonomous systems (N), or m:N operations. Unfortunately, the human performance literature suggests that human operators will exhibit poor supervision of numerous autonomous agents due to the limits of attentional resources in the operators. In the general human information-processing model, a human operator exercises a limited pool of attentional resources to engage various information-processing stages including detecting, perceiving, comprehending, and predicting objects around them. Yamani and Horrey (2018) expanded the human information-processing model to characterize a tradeoff between information-processing demand and resource relief that automation brings in the context of automated driving. In their model, a driver interacting with an automated driving system is assumed to reallocate resources “freed” by automation to support other information-processing stages required for successful task performance. A future AAM ecosystem enabled by an orchestration of advanced automated systems, however, requires a single operator to interact with more than one air vehicle with varying levels and degrees of automated systems, making the traditional framework of human-automation interaction insufficient. To address this gap, we provide a review of the literature on situation assessment and trust, two constructs identified as critical for a fuller understanding of intimate and intricate interactions between a human operator and multiple air vehicles equipped with increasingly autonomous systems. Then, we propose an expansion of Yamani and Horrey’s (2018) model to motivate systematic research on the human operator’s role, identify factors that influence resource allocation and guide human-centered design of an interface supporting the m:N operation in the AAM environment.

Advanced Air Mobility↗

Unreal Engine Testbed for Computer Vision of Tall Lunar Tower Assembly

The Tall Lunar Tower project at the NASA Langley Research Center is focused on the design, modeling, fabrication, and testing of a supervised autonomously assembly engineering development unit for tall lunar towers. The lunar south pole environment poses many challenges for robotic assembly of the tall tower, particularly to computer vision camera systems due to a high-contrast lighting environment. This paper will present an Unreal Engine 5 video game engine Lunar South Pole Lighting Testbed to simulate realistic lunar lighting conditions for synthetic image generation. The fidelity of the simulation environment is investigated by comparing the accuracy of computer vision models trained using synthetic image data and trained from real image data collected in a lunar analog environment.

Unreal Engine↗

AI Foundation Models for Science: An Open Collaborative Initiative

Foundation Models (FMs), AI models designed to replace task-specific models, are increasingly being recognized for their versatility across numerous downstream applications. These models, trained using self-supervised techniques on any type of sequence data, circumvent the need for large annotated datasets, a major bottleneck in traditional AI model development. FMs can be applied to downstream tasks using few-shot learning and fine-tuning, significantly reducing the need for large labeled training datasets and computational resources. However, the development of FMs requires substantial resources, including access to data and compute power, expertise in the latest models, and specialized scientific knowledge for systematic evaluation. It is challenging for a single group to possess all these capabilities. To address this, NASA IMPACT has initiated an open collaborative effort, leveraging partnerships with the private sector and other groups within and outside NASA, to jointly build FMs. The overarching goal is to develop a consistent and collaborative approach to building FMs for high-value science datasets. This initiative has fostered collaboration within NASA and with external partners, including IBM Research, Clark University, DOE’s ORNL, ESA, and USGS. The effort focuses on identifying key datasets with a wide range of downstream applications, pretraining and building FMs using modified transformer architectures, evaluating compute infrastructure needs, and sharing models, pretraining and fine-tuning code, and data with the community. Furthermore, it aims to train the Earth science community to fine-tune these models for various downstream applications. Our initial effort resulted in the creation of a 100 million parameter HLS Geospatial Model within six months, which was released on HuggingFace. We are now expanding our scope to include data from weather and climate models and investigating multimodal models. We invite those interested in participating in this effort to join us by sharing their use cases, expertise, or data.

Rahul Ramachandran↗

Deep Neural Network Based Convergence Classification for Computational Fluid Dynamics

A supervised deep learning approach is coupled with heuristic convergence criteria to construct a classification model for detecting the completion (convergence) of computational fluid dynamics (CFD) simulations. Heuristic convergence criteria alone are not always sufficient and more complex decisions are often left to a human analyst. The proposed approach leverages heuristic convergence criteria as well as two deep neural network (DNN) models, one binary and one multi-class, to improve the efficiency and consistency of convergence classification across a wide range of flight regimes. The DNN models presented are each trained on a subset of ascent aerodynamic CFD simulations for NASA’s Space Launch System and were produced using NASA’s unstructured Navier-Stokes solver FUN3D. Individual solutions are analyzed intermittently and are classified as sufficiently converged, further iterations required, or switch from steady Reynolds Averaged Navier-Stokes (RANS) to unsteady RANS CFD based on the iterative histories of four aerodynamic coefficients. The implemented classification model is shown to produce solutions that closely correlate to solutions produced by a human analyst. This work lays groundwork for expanding the capabilities of DNNs for automating and improving more of the CFD process.

SLS↗

Astrobee Operations on the Iss: Gui’S Impact on the Operators’ Cognitive Load

The Astrobee free-flying robots have completed their fifth successful year of operations housed in the Japanese Experimental Module (JEM) on the International Space Station (ISS). In this paper, we introduce two Graphical User Interfaces (GUI) used to operate and monitor in real-time the Astrobee free-flying robots onboard the ISS and report on the impact these GUIs have in the operator’s cognitive load. A review of the state-of-the-art GUI design for remotely teleoperated scenarios with minimal time delay is presented and the study’s conclusion used to determine the elements and recommendations to create an interface that minimizes its impact on the overall performance of an operator during an activity at the ISS. The Ground Data System (GDS) is one of the two GUIs in the study: it contains several tabs, each of which displays a different set of controls for specific tasks e.g. Overview, Run Plan, Teleoperate, Guest Science; some also display video and a three-dimensional (3D) representation of the ISS and robot based on the Astrobee’s telemetry. Most tabs enable a single operator-robot connection, however some of its tabs are capable to monitor and control up to three Astrobees simultaneously. The GDS Helper is a text-based user interface created to facilitate commanding and monitoring of an Astrobee robot directly from an SSH session. In full interactive mode it displays a maximum of 5 sections: general commanding, feedback/ack, telemetry, guest science commanding, and data, all in one view. In batch mode, it enables complex command scripting while retaining some interactive capabilities. A comparative analysis between these GUIs is carried out at an analogous ISS environment at the NASA Ames Research Center’s Granite Lab and its results presented. While GDS is able to provide an operator with control and situational awareness via its video and 3D displays, its several tabs may introduce an overwhelming amount of information confusing and delaying the operator especially during time-sensitive maneuvers where the operator may need to switch back and forth between them. GDS helper in the other hand does not provide video or 3D displays thus not allowing an operator to attain situational awareness, however it provides the operator with a design displaying commonly used data in a single window, enabling the operator to understand the state of the robot at a glance and control it through a commands entered via keyboard instead of a combination of mouse clicks and keyboard input. The results of the experiments measure the cognitive load across several operators maneuvering Astrobee to accomplish tasks ranging from fully manual to supervised activities. A GUI combining a single window displaying data along video and a 3D display is expected to reduce the operator’s cognitive load.

Astrobee↗

GeoAI advances in specific landform mapping

Landform mapping (also referred to as geomorphology or geomorphometry) can be divided into two domains: general and specific (Evans 2012). Whereas general landform mapping categorizes all elements of the study area into landform classes, such as ridges, valleys, peaks, and depressions, the mapping of specific landforms requires the delineation (even if fuzzy) of individual landforms. The former is mainly driven by physical properties such as elevation, slope, and curvature. The latter, however, must consider the cognitive (human) reasoning that discriminates individual landforms in addition to these physical properties (Arundel and Sinha 2018). Both mapping forms are important. General geomorphometry is needed to understand geological and ecological processes and as boundary layer input to climate and environmental models. Specific geomorphometry supports such activities as disaster management and recovery, emergency response, transportation, and navigation. In the United States, individual landforms of interest are named in the U.S. Geological Survey (USGS) Geographic Names Information System, a point dataset captured specifically to digitize geographic names from the USGS Historical Topographic Map Collection (HTMC). Named landform extent is represented only by the name placement in the HTMC. Recent work has investigated CNN-based deep learning methods to capture these extents in machine-readable form. These studies first relied on physical properties (Arundel et al. 2020) and then included the HTMC as a band in RGB images in limited testing (Arundel et al. 2023). Results from the HTMC dataset surpassed those using just physical properties and using the HTMC alone performed best due to the hillshading and elevation (contour) data incorporated into the topographic maps. However, results fell short of an operational capacity to map all named landforms in the United States. Thus, our current work expands upon past research by focusing on the HTMC and physical information as inputs and the named landform label extents. Specifically, we propose to leverage pre-trained foundation models for segmentation and optical character recognition (OCR) models to jointly map landforms in the United States. Our approach aims to bridge the disparities among the independent information sources to facilitate informed decision-making. The modeling pipeline performs (1) segmentation using the physical information and (2) information extraction using OCR, in parallel. Then a computer vision approach merges the two branches into a labeled segmentation. References: Arundel, Samantha T., Wenwen Li, and Sizhe Wang. 2020. “GeoNat v1.0: A Dataset for Natural Feature Mapping with Artificial Intelligence and Supervised Learning.” Transactions in GIS 24 (3): 556–72. https://doi.org/10.1111/tgis.12633. Arundel, Samantha T, and Gaurav Sinha. 2018. “Validating GEOBIA Based Terrain Segmentation and Classification for Automated Delineation of Cognitively Salient Landforms BT - Proceedings of Workshops and Posters at the 13th International Conference on Spatial Information Theory (COSIT 2017).” In Proceedings of Workshops and Posters at the 13th International Conference on Spatial Information Theory (COSIT 2017), Lecture Notes in Geoinformation and Cartography, edited by Paolo Fogliaroni, Andrea Ballatore, and Eliseo Clementini, 9–14. Cham: Springer International Publishing. Arundel, Samantha T., Gaurav Sinha, Wenwen Li, David P. Martin, Kevin G. McKeehan, and Philip T. Thiem. 2023. “Historical Maps Inform Landform Cognition in Machine Learning.” Abstracts of the ICA 6 (August): 1–2. https://doi.org/10.5194/ica-abs-6-10-2023. Evans, Ian S. 2012. “Geomorphometry and Landform Mapping: What Is a Landform?” Geomorphology 137 (1): 94–106. https://doi.org/10.1016/j.geomorph.2010.09.029.

machine learning↗

Rhode Island Ecological Conservation: Methods for Monitoring Rhode Island Habitats: Contributing to a Framework for Targeted Conservation and Management

Global avian population decline since the 1970s is largely attributable to habitat loss and degradation from anthropogenic disturbances. NASA DEVELOP’s Rhode Island Ecological Conservation team partnered with the Audubon Society of Rhode Island to compute land use land cover (LULC) maps of Rhode Island to aid in the conservation of the state’s 140 bird species. This project aimed to support the partner’s land acquisition strategies with updated and specific LULC classifications showing potential bird-habitat locations across the state. We incorporated remotely sensed data from Landsat 8 and 9 Operational Land Imager (OLI) into LULC maps using unsupervised classification techniques in ArcGIS Pro and supervised classification in Google Earth Engine. We generated six land classifications for 2023, which showed land cover dominated by upland habitats (forests, scrub/shrub, and grasslands), followed by development. We used TerrSet’s Land Change Modeler to forecast LULC change through 2043, using 2011 and 2021 National Land Cover Database (NLCD) land cover maps derived from Landsat 8 and 9 imagery. Project results suggest that non-urban upland and wetland habitats will decrease over time, while development will continue to encroach on non-urban avian habitats. Our maps and associated data will allow for more efficient land acquisition and management efforts to support avian habitat conservation across Rhode Island. Our study shows that data acquisition and processing from open data sources is feasible and further analysis can be done through GIS classification tools. More analysis is needed beyond this study to obtain more detailed land cover maps, though Audubon can aid its targeted conservation efforts with our current, historic, and forecasted LULC maps.

Remote sensing↗

Cali Urban Development: Using NASA Earth Observations to Assess Wetlands and Land Reclamation in Cali, Colombia

Recent research has documented the global decline of wetlands, largely attributed to increased urbanization and agriculture. This NASA DEVELOP study partnered with two local environmental entities in Cali, Colombia: The Fundación Dinamizadores Ambientales and the Departamento Administrativo de Gestión del Medio Ambiente. The team utilized Earth observations to evaluate trends in wetland extent, potential, and land cover in Cali between 2002 and 2023. A supervised classifier was generated within Google Earth Engine to create land use analyses of the region using Landsat 5 TM, Landsat 8 OLI, and Landsat 9 OLI-2 imagery. To identify locations of wetland potential within the study area, wetland probability was assessed by inputting PlanetScope, Sentinel-2 MSI, and partner-provided datasets into the Wetland Intrinsic Potential Tool in ArcGIS Pro and R. Data from Sentinel-1 C-SAR, Sentinel 2-MSI, and Suomi-NPP VIIRS were used to evaluate wetland extent using the Wetland Extent 3.0 Tool in3 Python. Overall, results indicated high wetland potential, particularly in the southeast region where agricultural areas were previously wetlands. Outputs also suggest a vast network of riparian wetlands in Cali. This study did not investigate socioeconomic data as it relates to wetlands, which is an avenue for future research. This project supplemented research into links between land use change, wetland extent, and wetland potential, and provided partner organizations with an objective foundation from which they can identify at-risk wetlands and develop community initiatives for management, conservation, and education.

Cali↗

Finite Element Thermal Model for Ultrasonic Welding of Thermoplastic Composites

Ultrasonic welding, UW, is a fast and energy-efficient technique for joining thermoplastic composites. It involves the use of high-frequency mechanical vibrations and a static welding force to melt and join adherends. Ultrasonic welding is an enabling technology to reduce the cost and complexity of in-space construction because lightweight thermoplastic composite components can be packaged compactly for launch and then efficiently assembled using supervised autonomous robotic technologies on site. However, the temperatures in space present challenges to UW, and it is critical that the efficacy of process parameters selected for manufacturing in space is understood prior to launch. To this end, a three-dimensional finite element model is presented in this technical presentation. The model incorporates equations for effects of viscoelastic heating and heat transfer on the welding process. The proposed model is applied to predict the temperature distribution in single lap shear, SLS, samples composed of AS4/PEEK (TC1200) composite that were welded using a terrestrial machine as part of a comprehensive weldability study. Thermocouple and infrared spot sensor data from the SLS samples provide empirical temperature measurements for calibration and validation of the UW thermal model. Calibration and validation of the model is an important step given the significant uncertainties in material properties such as the loss modulus and necessary assumptions in the physics implementations which allow the model to converge in an acceptable amount of time. The validated thermal model can be used to simulate the process for the space environment. Future validation sample testing is planned in a vacuum chamber. The result will be a model capable of guiding process parameter selection to ensure acceptable weld bonds when manufacturing in space.

Josh Fody↗

A Machine Learning Approach to Improve Air Traffic Management Initiatives

Collaborating closely with commercial air carriers and related organizations, the Federal Aviation Administration(FAA) regulates air traffic and ensures the safety and efficiency of air operations. Air traffic controllers make strategic decisions, such as delaying, rerouting, or canceling flights, partly based on guidance provided by the FAA’s Air TrafficControl System Command Center (ATCSCC). The guidance includes, among other things, control measures known asTraffic Management Initiatives (TMIs) designed to enhance safety and improve operational efficiency. TMIs play a crucial role in managing the demand and capacity within the U.S. National Airspace System (NAS). Two major TMIs that are routinely used (primarily to mitigate the adverse effects of bad weather) are Ground Delay Programs (GDPs) andGround Stops (GSs). In a GDP, flights destined for airports facing thunderstorm activity experience delays at their origin airports. This proactive approach minimizes the risk of routing aircraft through hazardous weather conditions and also replaces (fuel burning) airborne delays with ground delays. In a GS, a temporary restriction is imposed on the departure or arrival of aircraft at a specific airport or within a designated airspace. Although other TMIs (e.g., miles-in-trail) are also implemented as part of (air) traffic flow management in the NAS, the focus of this work is on GDPs and GSs. Since TMIs, by design, lead to flight delays or cancellations, it is crucial to put in place the right set of parameters(e.g., scope and duration of the GDP). For example, when the end time of a GDP extends beyond what is necessary, it imposes unnecessary delays on departing flights. This situation could occur as a result of inaccurate prediction of the(required) duration of the GDP based on the weather forecast. On the other hand, if a GDP ends prematurely before the underlying capacity constraints are resolved at the destination airport, it may result in airborne holding. The delicate balance lies in matching the termination of the GDP precisely with the resolution of capacity constraints, avoiding both the imposition of unnecessary ground delays and the need for airborne holding due to premature program termination.Failing to specify the right parameters for TMIs also leads to flight delays, creating a significant obstacle in managing the increasing traffic volumes causing increased work load for the controllers. To address this issue, we propose the integration of Machine Learning (ML) models in the traffic flow management(TFM) pipeline. In current operations, decisions are made by human experts based on extensive training, historical patterns, available traffic and weather data. Since we have an abundance of data from past events that tell us the likely impact of various TMIs, by ingesting historical data, properly trained ML models can offer valuable insights and aid human decision-making. With the FAA increasingly exploring advanced analytics, ML emerges as a focal point for enhancing TFM within the National Airspace System (NAS). As a first step, this study aims to provide traffic controllers with decision-making support for the issuance and adjustment of TMIs. Data analytics and machine learning have been previously employed to address some of the challenges associated with TMIs. Numerous studies have concentrated on various facets of TMI issuance, exploring factors influencing TMI parameters, including arrival rate, airport capacity, and delay prediction. For example, using weather forecasts, several statistical methods were used to produce probabilistic capacity profiles which in conjunction with deterministic models provided insights into the GDP planning process [1–4]. The downside of using deterministic models is that they rely on fixed inputs and predetermined rules, which lack the ability to account for the inherent uncertainty and variability present in real-world scenarios. In a separate series of studies, researchers aimed to predict the occurrences of GDPs and GSs. The majority of these studies utilized various supervised learning methods, including Decision Trees, Naive Bayes, Support VectorMachines, and Random Forests to analyze the influence of weather conditions and arrival demand on TMI incidents[5–8]. However, these studies primarily focused on predicting the incidence of TMIs without explicitly addressing the scope of TMIs, including their duration and their geographical coverage. Furthermore, the emphasis of these studies was largely on GDPs, given their higher frequency and longer duration when compared to GSs. A limited number of studies focused on predicting the parameters of TMIs, specifically addressing their duration and extent. In one such study focusing on optimizing the TMI parameters at San Francisco International Airport (SFO),the authors utilized a probabilistic forecast of fog [9]. They simulated various capacity scenarios based on the (fog)burn-off forecasts, selecting GDP parameters that minimized airborne and overall ground delays. However, this approach exclusively emphasizes stratus (fog) burn-off as the primary determinant of GDP and GS, neglecting other influential factors like severe weather events, runway closures, lower capacity than traffic demand, and other important variables. Given the complexity of predicting the TMI and determining its scope, we seek a more holistic approach. We aim to consider all significant factors that could impact TMIs and their parameters. What sets this research apart is the fusion of all data sources relevant to the issuance and adjustment of TMIs and it represents the first comprehensive attempt to optimize TMIs in this manner. Since this comprehensive solution involves various aspects, we break down the problem into smaller components and input all parameters into a unified model called the “TMI Adjuster”. Figure 1 shows the overall framework and the list of datasets used in each model. The objective of the TMI Adjuster module is to deliver reliable, consistent and expedited recommendations for the progression, adjustment, and termination of TMIs. The ML solution entails developing a pipeline capable of predicting the necessity of a TMI (e.g., GS or GDP) along with its various parameters. For example, in the case of a GS, this includes the scope of the GS either in terms of distance from the destination airport or based on pre-defined airspace sectors. Here, scope refers to those regions and departing airports that are subject to the GS. In this paper, we concentrate on the issuance of GSs in the three major airports in the New York area — LaGuardia(LGA), John F. Kennedy International (JFK), and Newark Liberty International (EWR). We fuse traffic, weather and other relevant aviation data from years 2017 to 2019 to train and validate the ML models. In particular, we use the following datasets: •Terminal Aerodrome Forecast (TAF): meteorological forecasts specific to each airport, issued four times a day, covering predefined time periods. •TMI data: includes all GSs and GDPs along with their respective parameters. •Aviation System Performance Metrics (ASPM): includes traffic related data such as aircraft delays, arrival, and departure rates. •Notices to Airmen (NOTAMs): utilized to extract runway closure data and manage interdependencies between terminals in close proximity. •Flight cancellation data •Airspace Flow Programs (AFP): includes information on flight airborne holdings caused by TMIs. The data preprocessing entails transforming ASPM, TMI, AFP, NOTAMs, and weather data into an hourly format and consolidating all datasets by merging them based on date and time as the primary key. The TMI Adjuster framework comprises two parallel models: one dedicated to GS and a second model focused on GDP. As previously mentioned, our specific focus is on the GS model as a multi-classification problem. In this framework, each data point of the GS model input summarizes ten hours of data. Specifically, the data loader for the GS model generates the input and output of the model as follows: at a given time step, the input includes the actual traffic, weather, and TMI data from the two-hour window before the time step, alongside the weather forecast and scheduled traffic for the next 8 hours starting from the time step. Based on this information, the output of the GS model for each time interval consists of three dimensions. The first dimension represents a binary decision on whether there should be a GS in place for the next hour or not. The second dimension is related to the scope of the GS in the United States, and the third dimension is related to the scope of the GS in Canada (i.e., to determine if the GS impacts airports in Canada).One of the challenges with TMI modeling is the sparsity of TMI events, particularly regarding its scope. To address this challenge in the scope of the GS model output, we implement grouping. The GS scope for the US region is defined based on a list of centers that should be included when the GS is in place. With 20 centers in the US, we utilized historical data to group them into 4 categories. In particular, we summarized our historical data in a graph format where nodes represent centers, and link weights are defined based on the co-occurrence of centers in the scope parameter ofTMIs. By identified strongly connected components in this graph, we were able to partition the centers into four groups. We consider two model structures for the GS Model. Firstly, a hierarchical classification model [10], where the human decision-making for a GS is of hierarchical nature. The decision-maker first decides whether there is a need fora GS, and if the answer is yes, determines the scope. A hierarchical classification model organizes the problem into a class hierarchy, typically a tree or a Directed Acyclic Graph (DAG) structure, and considers the dependency of the decision in the previous step to the next component [10]. Here, we employ the local classifier per level approach, which involves training one multi-class classifier for each level of the class hierarchy. The second structure is the independent structure. In this setting, as the name suggests, we do not consider the dependency of the decisions in the different dimensions of the output of the model. Instead, for each dimension, we train a multi-class classifier independently. Table 1 summarizes GS model statistics for training, validation and testing. The table documents the effect of limiting data to the time steps when there was actually a TMI in place or when a TMI had just terminated. This resulted in a more balanced distribution of the GS class(GS positive class)versus “No GS”(GS negative class), which might help the training process. While JFK and LGA follow very similar distributions, with 40% and 42% GS positive class respectively, EWR has proportionally fewer GS incidents at 28%. Our subsequent phase involves evaluating the performance of both hierarchical structure and independent structure using different state-of-the-art multi-class classifier models such as Random Forest, Decision Trees, K-nearest Neighbors, and Logistic Regression and forecast the duration and scope of the GSs.

Farzan Masrour Shalmani↗

Recommendations on Evidence and Process for Certification of Learning-enabled Components in Aerospace Systems

This report primarily identifies a collection of relevant and necessary evidence for assurance of machine learnt components (MLCs)—also known as learning-enabled components—integrated into aircraft systems, and gives preliminary suggestions on the elements of a certification process that invoke the identified evidence. The main focus is on feedforward neural networks that are static and trained offline through supervised learning. A brief background on the generic elements of the lifecycle of an MLC is given to contextualize the assurance considerations and, consequently, the evidence that is relevant and necessary to support certification. At the level of an MLC, those considerations relate to: (i) the consistency and correctness of MLC contributions to system functions in the context of a validated functional intent; and (ii) the absence of MLC contributions to aircraft-level failure conditions. At an ML model level, confidence in model and data properties contribute to assurance of the containing MLC, in particular: (a) generalizability and robustness of models, in the presence of inputs not previously seen during training, disturbances to inputs, and unexpected inputs; and (b) valid data, i.e., data that are at least representative, relevant, complete, and accurate. Evidence for the above span the elements of the ML lifecycle, and includes, at a minimum, lifecycle artifacts that pertain to: (1) properties of requirements capturing functional intent, safety constraints, and aspects of the intended use and operating environment; (2) model performance, model complexity and design, and algorithm choice; (3) achievement of required performance at the levels of a trained model during model development, a trained model after model development is complete, and a trained model that is transformed into an executable equivalent; (4) model implementation aspects necessary for transforming a trained model into the executable equivalent; (5) integration of the executable trained model into the containing MLC, and eventually the larger system; and, (6) lastly, the verification and validation (V&V) of each of the above. Such V&V lifecycle artifacts themselves include: aspects of coverage, e.g., of various levels of requirements by the input space of the model and the data; traceability (where applicable); application of formal methods for property specification, analysis, and checking. Examples of evidence generation methods and tools further ground the discussion on what constitutes evidence, and the contribution to assurance during certification. The identified assurance considerations and supporting evidence is not a comprehensive set. Additionally, neither what should be considered as sufficient evidence relative to the assigned criticality of an MLC, nor how criticality ought to be determined and adjusted, have been considered in this report. However, suggestions are made for potential activities of the ML lifecycle that are aimed at providing confidence that an MLC can be relied upon when integrated into its containing (aircraft) system. Those activities are proposed as candidate elements of a certification process for MLCs. The main purpose of this report to inform regulatory guidance and consensus standards that may be used to meet the safety intent of the applicable regulations.

Aviation safety↗

The Development and Deployment of Machine Learning Models for Aircraft Engine Concept Assessment

In today's competitive landscape, the effective development and utilization of machine-learning (ML) applications have become imperative across various sectors. This study presents an outline of the procedure involved in creating and implementing ML models for conceptualizing and evaluating aircraft engines. These models leverage supervised deep-learning algorithms to analyze patterns within an open-source repository containing data on both production and research conventional turbofan engines. The main areas of focus encompass crucial engine parameters like thrust-specific fuel consumption (TSFC), engine weight, engine diameter, and turbomachinery stage counts. While the creation of ML models is fundamental for their utilization, ensuring their seamless deployment holds equal significance. To address this aspect, a conversational AI chatbot is constructed, utilizing natural language processing (NLP) techniques, to facilitate the deployment of these ML models. The comprehensive workflow encompasses several key stages: gathering and enhancing engine data, training and cross validating the ML models, testing and evaluating their performance, and finally, deploying, monitoring, and updating the ML models. By following this systematic approach, the aim is to streamline the development and deployment process of ML models tailored for aircraft engine assessment.

Development↗

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learningautonomous systems; flight simulat↗

Wildfire Segmentation From Remotely Sensed Data Using Quantum-Compatible Conditional Vector Quantized-Variational Autoencoders

Wildfires represent a critical environmental hazard with multifaceted implications for ecosystems, communities, and public health [1]. The escalating frequency and intensity of wildfires globally have intensified the urgency for robust segmentation methodologies to facilitate effective mitigation, response, and recovery strategies [2]. Accurate wildfire segmentation is pivotal for delineating fire boundaries, assessing progression patterns, and prioritizing resource allocation during emergency scenarios. Furthermore, precise segmentation enables stakeholders, including policymakers, environmental scientists, and emergency responders, to formulate evidence-based strategies, thereby minimizing socio-economic disruptions and ecological degradation. Consequently, advancing wildfire segmentation techniques through innovative technological interventions remains a paramount research imperative. Although foundational in wildfire segmentation, traditional deterministic models exhibit inherent limitations that compromise their efficacy in dynamic and uncertain environments. These models often operate on rigid algorithms prioritizing deterministic classifications, thereby overlooking the inherent complexities and uncertainties associated with wildfire behavior and satellite data variability. Such deterministic frameworks tend to produce oversimplified representations that fail to capture the intricate nuances of evolving fire dynamics, spatial heterogeneity, and environmental interactions [1]. Consequently, the deterministic approach’s propensity for uncertainty collapsing [1, 3] hampers the accuracy, reliability, and applicability of segmentation outcomes in real-world scenarios. Contrastingly, stochastic models offer a more nuanced and adaptable framework for wildfire segmentation. By integrating probabilistic elements into the modeling paradigm, stochastic approaches, particularly probabilistic approaches such as variational auto encoders (VAEs) [4], facilitate comprehensive uncertainty assessment, enabling researchers to quantify and incorporate uncertainties into segmentation outcomes effectively. This probabilistic nature empowers stochastic models to encapsulate variability, account for data inconsistencies, and adapt to evolving environmental conditions, enhancing segmentation accuracy, reliability, and robustness. Embracing stochastic methodologies thus catalyzes advancements in wildfire science by fostering a more holistic, adaptive, and resilient segmentation framework. Despite VAEs demonstrating significant promise in various applications, they come with inherent limitations that have garnered attention within the machine learning community. One of the primary drawbacks lies in their reliance on static priors, which essentially assume a fixed distribution for latent variables, thereby limiting the model’s flexibility to capture complex data structures effectively [5]. This static nature leads to suboptimal representations, especially when dealing with complex and high-dimensional data. Additionally, VAEs often struggle with generating sharp and realistic samples, a phenomenon commonly referred to as mode collapse [5, 7, 6]. Furthermore, the optimization process in VAEs, which involves balancing the reconstruction loss and the regularization term, can sometimes be challenging to fine-tune [7]. In recent efforts to address these shortcomings, alternative approaches like Vector Quantized Variational Auto encoders(VQ-VAEs) [7], address the challenges by incorporating discrete latent variables and leveraging techniques that enhance the quality and diversity of generated samples while maintaining efficient training dynamics. VQ-VAEs propose a dynamic prior distribution generation mechanism that diverges from the static priors commonly associated with traditional VAEs. This dynamic approach allows for more adaptive and context-aware latent variable representations, thereby potentially capturing complex data structures more effectively. Unlike autoregressive prior models such as PixelCNN, which, despite their ability to model dependencies across data dimensions, suffer from significant computational inefficiencies and lack flexibility in handling diverse datasets. In our work, we propose to use a generative quantum-compatible approach to help alleviate the shortcomings of autoregressive prior model in VQ-VAEs. Restricted Boltzmann Machines (RBMs) are a viable alternative prior model that can learn prior distributions in a faster and more flexible manner. In this research endeavor, we meticulously curate a state-of-the-art dataset leveraging satellite MODIS data in conjunction with VIIRS fire masks, derived from Fire Radiative Power (FRP), thereby encapsulating diverse wildfire scenarios and environmental contexts. We developed a conditional VQ-VAE architecture with the RBM prior model that is trained in a supervised manner for segmenting wildfire masks. This innovative approach synergistically harnesses deep learning capabilities, enabling the generation of segmentation maps characterized by heightened precision, granularity, and contextual relevance. Furthermore, replacing the autoregressive prior learning method proposed by the original VQ-VAE with a prior density approximation via quantum-compatible RBM facilitates expedited inference processes, augments flexibility in prior sampling, optimizes computational efficiency and establishes a groundbreaking benchmark in wildfire segmentation methodologies.

quantum machine learning↗

Robotic Manipulation Testbeds with HPSC

Interoperability and scalability of robotic manipulators will be key to develop and sustain a lunar surface and cislunar ecosystem. From in-space servicing, assembly, and manufacturing (ISAM) to logistics, maintenance, and science operations, robotic manipulation is a critical NASA capability need and the demand for high-performance spaceflight computing will only rise as robotic tasks become more autonomous. With increased complexity, testbeds for research, feasibility studies, and technology demonstrations will be essential. The Dexterous Robotics Team at NASA Johnson Space Center has established multiple robotic manipulation testbeds taking a supervised autonomous remote operations approach and plans to infuse High-Performance Spaceflight Computing (HPSC) to emulate the flight environment and close the gap between space technology development and flight operations.

Alex Sowell↗

Anomaly Detection for the Roman Space Telescope Wide Field Instrument’s Science Data Processing Pipeline

The Roman Space Telescope (RST) Wide Field Instrument (WFI) will be utilizing a preliminary Science Data Processing (SDP) pipeline during its Integration and Test, and to some extent during Operations, to track basic statistics and identify known features such as cosmic rays, snowballs as well as possible anomalies in raw detector data. In our detectors, these anomalies appear as jumps in the ramp of a readout and are classified as cosmic rays if they appear as a streak or snowballs if they’re more circular. The WFI employs an array of 18 H4RG-10 detectors that collect image samples. Each set of raw frames within a non-destructive exposure is packaged by the SDP pipeline into image cubes for each detector. Each cube is a time series of 4096 × 4096 accumulating pixel frames. The preliminary analysis pipeline is used to locate anomalies in these time-series accumulation frames and identify the type of anomaly, either natural phenomena or detector characteristic. To compare different methods, we’ve implemented both heuristic-based and data-driven methods to identify anomalies. For the heuristic-based approach, we identify snowballs and cosmic rays by the size and shape of outlier pixel clusters between consecutive frames. For data driven methods, we evaluated a Convolutional Neural Network (CNN) model, and more traditional methods like Principal Component Analysis (PCA). CNN is a supervised learning/classification method. Thus, we used a labeled dataset of anomalies to perform segmentation of the image and identify anomalies. We used previously identified cosmic rays and snowballs to measure the accuracy and efficiency of the mentioned approaches. In evaluating these methods, we aim to pick the best fit for the SDP pipeline’s anomaly detection in terms of both performance and runtime.

Paul Horton↗