Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61

A Framework for Preliminary Design of Aircraft Structures Based on Process Information

This report discusses the general framework and development of a computational tool for preliminary design of aircraft structures based on process information. The described methodology is suitable for multidisciplinary design optimization (MDO) activities associated with integrated product and process development (IPPD). The framework consists of three parts: (1) product and process definitions; (2) engineering synthesis, and (3) optimization. The product and process definitions are part of input information provided by the design team. The backbone of the system is its ability to analyze a given structural design for performance as well as manufacturability and cost assessment. The system uses a database on material systems and manufacturing processes. Based on the identified set of design variables and an objective function, the system is capable of performing optimization subject to manufacturability, cost, and performance constraints. The accuracy of the manufacturability measures and cost models discussed here depend largely on the available data on specific methods of manufacture and assembly and associated labor requirements. As such, our focus in this research has been on the methodology itself and not so much on its accurate implementation in an industrial setting. A three-tier approach is presented for an IPPD-MDO based design of aircraft structures. The variable-complexity cost estimation methodology and an approach for integrating manufacturing cost assessment into design process are also discussed. This report is presented in two parts. In the first part, the design methodology is presented, and the computational design tool is described. In the second part, a prototype model of the preliminary design Tool for Aircraft Structures based on Process Information (TASPI) is described. Part two also contains an example problem that applies the methodology described here for evaluation of six different design concepts for a wing spar.

Rais-Rohani, Masoud↗

Implementing an Applied Science Program

The work implied in the NASA Applied Science Program requires a delicate balancing act for the those doing it. At the implementation level there are multiple tensions intrinsic to the program. For example each application of an existing product to a decision support process requires deep knowledge about the data and deep knowledge about the decision making process. It is highly probable no one person has this range of knowledge. Otherwise the decision making process would already be using the data. Therefore, a team is required. But building a team usually requires time, especially across agencies. Yet the program mandates efforts of relatively short duration. Further, those who know the data are scientists, which makes them essential to the program. But scientists are evaluated on their publication record. Anything which diverts a scientist from the research for his next publication is an anathema to him and potential death to their career. Trying to get another agency to use NASA data does not strike most scientists as material inherently suitable for publication. Also, NASA wishes to rapidly implement often substantial changes to another agency's process. For many reasons, such as budget and program constraints, speed is important. But the owner of a decision making process is tightly constrained, usually by law, regulation, organization and custom. Changes when made are slow, cautious, even hesitant, and always done according a process specific to the situation. To manage this work MSFC must balance these and other tensions. Some things we have relatively little control over, such as budget. These we try to handle by structural techniques. For example by insisting all of our people work on multiple projects simultaneously we inherently have diversification of funding for all of our people. In many cases we explicitly use some elements of tension to be productive. For example the need for the scientists to constantly publish is motivation to keep tasks short and to the immediate point. As another example, the DSS agency and it personnel are treated as integral to the effort at every step. Thus, their ways of doing things, their problems and their assets become part of the solution. MSFC also manages the Applied Science work' within a strategic framework.' First, the scientists are necessarily at the core of all of this work as well as all of the work within the larger' organization. We therefore strive to keep a roughly 50-50 balance between the work done on tasks funded directly by the research side of NASA's Science Mission Directorate and funding from the applied side. Done at both the organization level and the level of the individual, this keeps the scientist both happy and productive over the long term. We also try diligently to remove as much burden from the scientist as practical by employing people such as the authors and others to do tasks not requiring; scientific knowledge. We also have designed our effort to take full use of external partners. We actively seek and support, including fund, people from multiple organizations to join us as committed collaborators. In this we use today's money and today's problem to help us diversify and strengthen for tomorrow. MSFC also considers the Applied Science work holistically. Each element is viewed as a step in a larger process. At a management level we can chose to emphasize or encourage certain areas which service long term goals. Thus, if we think work in a particular area should be developed, we can start with the smaller, less costly elements and grow.

Rickman, Doug↗

Expanding the Spatiotemporal Range of Soil Moisture Analysis Utilizing NASA Earth Observations and In Situ Measurements

Drought can cause immense agricultural and ecological damage resulting in high mitigation and compensation costs. Climate variability in future decades is expected to cause severe drought conditions and threaten necessary water resources. Stakeholders seek to implement effective drought assessments in preparation for potential economic and environmental damage invoked by drought. Although in-situ measurements are accurate, the current infrastructure is spatially limited and costly to maintain. A framework was created to compare modeled, satellite and in-situ data in drought monitoring. Here we show that the comparison of in-situ and remotely sensed soil moisture (SM) measurements can increase the spatiotemporal range of SM assessments. Data collected between 2003 and 2021 by NASA’s SPoRT Land Information System (SPoRT-LIS) and Soil Moisture Active Passive (SMAP) mission were standardized and compared with in-situ data provided through the Illinois Climate Network (WARM). Statistical analysis results including the Pearson correlation coefficient (r), root mean squared error, mean absolute error and others were calculated to compare the WARM measurements to the SMAP and SPoRT-LIS products. Results indicate that both satellite products demonstrate seasonally variable bias that is not present in the in-situ measurements. Bias was highest in the winter months and lowest in the late summer and early fall months in both satellite datasets. Overall, WARM-SPoRT comparisons resulted in lower seasonal variability. However, on average, the SMAP comparison demonstrated higher correlation values and lower error values. The WARM-SMAP average correlation (r) was 0.61 compared to the WARM-SPoRT average correlation (r) value of 0.54. Average mean absolute error values calculated for the SMAP and SPoRT comparisons were 0.07 and 0.08 percent soil moisture by volume, respectively. These analyses suggest integrating in-situ measurements and those provided by NASA Earth observations can be utilized in a multi-faceted SM evaluation, a valuable contribution to drought monitoring and water resource decision making.

Emma P Myrick↗

Uncertainty and Intelligence in Computational Stochastic Mechanics

Classical structural reliability assessment techniques are based on precise and crisp (sharp) definitions of failure and non-failure (survival) of a structure in meeting a set of strength, function and serviceability criteria. These definitions are provided in the form of performance functions and limit state equations. Thus, the criteria provide a dichotomous definition of what real physical situations represent, in the form of abrupt change from structural survival to failure. However, based on observing the failure and survival of real structures according to the serviceability and strength criteria, the transition from a survival state to a failure state and from serviceability criteria to strength criteria are continuous and gradual rather than crisp and abrupt. That is, an entire spectrum of damage or failure levels (grades) is observed during the transition to total collapse. In the process, serviceability criteria are gradually violated with monotonically increasing level of violation, and progressively lead into the strength criteria violation. Classical structural reliability methods correctly and adequately include the ambiguity sources of uncertainty (physical randomness, statistical and modeling uncertainty) by varying amounts. However, they are unable to adequately incorporate the presence of a damage spectrum, and do not consider in their mathematical framework any sources of uncertainty of the vagueness type. Vagueness can be attributed to sources of fuzziness, unclearness, indistinctiveness, sharplessness and grayness; whereas ambiguity can be attributed to nonspecificity, one-to-many relations, variety, generality, diversity and divergence. Using the nomenclature of structural reliability, vagueness and ambiguity can be accounted for in the form of realistic delineation of structural damage based on subjective judgment of engineers. For situations that require decisions under uncertainty with cost/benefit objectives, the risk of failure should depend on the underlying level of damage and the uncertainties associated with its definition. A mathematical model for structural reliability assessment that includes both ambiguity and vagueness types of uncertainty was suggested to result in the likelihood of failure over a damage spectrum. The resulting structural reliability estimates properly represent the continuous transition from serviceability to strength limit states over the ultimate time exposure of the structure. In this section, a structural reliability assessment method based on a fuzzy definition of failure is suggested to meet these practical needs. A failure definition can be developed to indicate the relationship between failure level and structural response. In this fuzzy model, a subjective index is introduced to represent all levels of damage (or failure). This index can be interpreted as either a measure of failure level or a measure of a degree of belief in the occurrence of some performance condition (e.g., failure). The index allows expressing the transition state between complete survival and complete failure for some structural response based on subjective evaluation and judgment.

Ayyub, Bilal M.↗

A Multimodel Assessment of the Influence of Regional Anthropogenic Emission Reductions on Aerosol Direct Radiative Forcing and the Role of Intercontinental Transport

In this study, we assess changes of aerosol optical depth (AOD) and direct radiative forcing (DRF) in response to the reduction of anthropogenic emissions in four major pollution regions in the Northern Hemisphere by using results from nine global models in the framework of the Hemispheric Transport of Air Pollution (HTAP). DRF at top of atmosphere (TOA) and surface is estimated based on AOD results from the HTAP models and AOD-normalized DRF (NDRF) from a chemical transport model. The multimodel results show that, on average, a 20% reduction of anthropogenic emissions in North America, Europe, East Asia, and South Asia lowers the global mean AOD (all-sky TOA DRF) by 9.2% (9.0%), 3.5% (3.0%), and 9.4% (10.0%) for sulfate, particulate organic matter (POM), and black carbon (BC), respectively. Global annual average TOA all-sky forcing efficiency relative to particle or gaseous precursor emissions from the four regions (expressed as multimodel mean +/- one standard deviation) is -3.5 +/-0.8, -4.0 +/- 1.7, and 29.5+/-18.1mW / sq m per Tg for sulfate (relative to SO2), POM, and BC, respectively. The impacts of the regional emission reductions on AOD and DRF extend well beyond the source regions because of intercontinental transport (ICT). On an annual basis, ICT accounts for 11 +/- 5% to 31 +/- 9% of AOD and DRF in a receptor region at continental or subcontinental scale, with domestic emissions accounting for the remainder, depending on regions and species. For sulfate AOD, the largest ICT contribution of 31 +/- 9% occurs in South Asia, which is dominated by the emissions from Europe. For BC AOD, the largest ICT contribution of 28 +/- 18% occurs in North America, which is dominated by the emissions from East Asia. The large spreads among models highlight the need to improve aerosol processes in models, and evaluate and constrain models with observations.

Europe↗

Locating Anomalies in Complex Data Sets Using Visualization and Simulation

The research goals are to create a simulation framework that can accept any combination of models written at the gate or behavioral level. The framework provides the ability to fault simulate and create scenarios of experiments using concurrent simulation. In order to meet these goals we have had to fulfill the following requirements. The ability to accept models written in VHDL, Verilog or the C languages. The ability to propagate faults through any model type. The ability to create experiment scenarios efficiently without generating every possible combination of variables. The ability to accept adversity of fault models beyond the single stuck-at model. Major development has been done to develop a parser that can accept models written in various languages. This work has generated considerable attention from other universities and industry for its flexibility and usefulness. The parser uses LEXX and YACC to parse Verilog and C. We have also utilized our industrial partnership with Alternative System's Inc. to import vhdl into our simulator. For multilevel simulation, we needed to modify the simulator architecture to accept models that contained multiple outputs. This enabled us to accept behavioral components. The next major accomplishment was the addition of "functional fault models". Functional fault models change the behavior of a gate or model. For example, a bridging fault can make an OR gate behave like an AND gate. This has applications beyond fault simulation. This modeling flexibility will make the simulator more useful for doing verification and model comparison. For instance, two or more versions of an ALU can be comparatively simulated in a single execution. The results will show where and how the models differed so that the performance and correctness of the models may be evaluated. A considerable amount of time has been dedicated to validating the simulator performance on larger models provided by industry and other universities.

Panetta, Karen↗

A Preliminary Study on the Feasibility of Large Language Models for Detecting Micro-Behaviors Among Team Members in Space Missions

Large-language models (LLMs) have been recently used for spoken language understanding (SLU) to infer meaning and semantics from speech in tasks such as speaker intent and sentiment classification. Due to being trained on large amounts of data, and their ability to understand context and relationships between words, LLMs are competent, enabling them to generalize across tasks without requiring many task-specific training samples. This research examines the feasibility of few-shot learning in LLMs for detecting subtle, brief, and possibly unconscious interactions between team members, called ``micro-behaviors," and provides insights into the appropriate design of LLMs for this task. Our data came from 5 teams participating in a 45-day mission at the US National Aeronautics and Space Administration’s (NASA) Human Exploration Research Analog (HERA). More specifically we used data collected from team interaction battery (TIB) tasks teams performed five times in-mission which comprise an average 1.5 hours of conversation data per day. Micro-behaviors were coded according to an adapted version of Smith & Griffins (2022) theoretical framework in terms of Violation (i.e., presence of valenced behavior, uplifting/positive or discouraging/negative), Intensity (i.e., force of behavior in terms of how uplifting or discouraging is the behavior), and Intent (i.e., motive of the behavior in terms of whether it was deliberate or unintentional). We explore the ability of LLMs to detect the presence and intensity of micro-behaviors. We examine employing and fine-tuning readily available LLMs (i.e., RoBERTa, DistilBERT), as well as prompting state-of-the-art sequence classification models (i.e., Llama-2, Llama-3). In a total of 13,058 conversational turns (17.8% uplifting, 3.3% discouraging, 75.76% neutral, 3.14% nulls), we compute the macro F1-score of the 3-way micro-behavior classification task (i.e., classifying among uplifting, discouraging, and neutral; 33% chance). Results indicate that the RoBERTa model achieves a F1-score of 36.2% (uplift: 43.3% precision (P), 15.1% recall (R); discourage: 20% P, 0.5% R). These results significantly improve when we augment the data via paraphrasing in the RoBERTa model, reaching a 41.2% macro F1-score (uplift: 37.7% P, 86.3% R; discourage: 3.5% P, 1.8% R). Finally, the Llama-2 model with 3-shot prompting yields 38% macro F1-score (uplift: 28.7% P, 20% R; discourage: 7.2% P, 18% R), which is slightly better compared to the RoBERTa model without data augmentation, highlighting the effectiveness of sequence classification models in detecting minority classes with a small sample size. Findings indicate that LLMs hold potential to detect subtle behaviors in conversations, which could be valuable in assessing team behavior in space exploration missions. Future studies will evaluate the performance of different LLM prompting strategies or fine-tuning methods.

Ankush Raut↗

Joint ESA-NASA Multi-Mission Algorithm and Analysis Platform (MAAP)

The scientific community is faced with a need for greatly improved data sharing, analysis, visualization and advanced collaboration based firmly on open science principles. Recent and upcoming launches of new satellite missions with more complex and voluminous data, as well as the ever more urgent need to better understand the global carbon budget and related ecological processes, provided the immediate rational for the ESA-NASA Multi-mission Algorithm and Analysis Platform (MAAP). This highly collaborative joint project of ESA and NASA established a framework between ESA and NASA to share data, science algorithms and compute resources in order to foster and accelerate scientific research conducted by ESA and NASA EO data users. Presented to the public in October 2021, the current version of MAAP provides a common cloud-based platform with computing capabilities co-located with the data, a collaborative coding and analysis environment, and a set of interoperable tools and algorithms developed to support the estimation and visualization of global above-ground biomass. Data from the Global Ecosystem Dynamics Investigation (GEDI) mission on the International Space Station and the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) have been instrumental in the first products of MAAP including the first comprehensive map of Boreal above-ground Biomass and a current Global Biomass Harmonization Activity, but the platform is also being specifically designed to support the forthcoming ESA Biomass mission and incorporate data from the upcoming NASA-ISRO SAR (NISAR) mission. While these missions and the corresponding research which includes airborne, field, and calibration/validation data collection and analyses, provide a wealth of data and information relating to global biomass estimation, they also present data storing, processing and sharing challenges. The NISAR mission alone will produce about 80TB/day. These large data volumes present a challenge that would otherwise place accessibility limits on the scientific community and impact scientific progress. Other challenges being addressed by MAAP include: 1) Enabling researchers to easily discover, process, visualize and analyze large volumes of data from both agencies; 2) Providing a wide variety of data in the same coordinate reference frame to enable comparison, analysis, data evaluation, and data generation; 3) Providing a version-controlled science algorithm development environment that supports tools, co-located data and processing resources; and 4) Addressing intellectual property and sharing challenges related to collaborative algorithm development and sharing of data and algorithms. MAAP products can be explored on the MAAP Dashboard at https://earthdata.nasa.gov/maap-biomass or the joint platform entrance at scimaap.net. MAAP also can be accessed through individual NASA (https://maap-project.org) and ESA (https://esa-maap.org/) landing pages.

cloud computing↗

Computational Models of the Cardiovascular System and Its Response to Microgravity

Computational models of the cardiovascular system are powerful adjuncts to ground-based and in-flight experiments. We will provide NSBRI with a model capable of simulating the short-term effects of gravity on cardiovascular function. The model from this project will: (1) provide a rational framework which quantitatively defines interactions among complex cardiovascular parameters and which supports the critical interpretation of experimental results and testing of hypotheses. (2) permit predictions of the impact of specific countermeasures in the context of various hypothetical cardiovascular abnormalities induced by microgravity. Major progress has been made during the first 18 months of the program: (1) We have developed an operational first-order computer model capable of simulating the cardiovascular response to orthostatic stress. The model consists of a lumped parameter hemodynamic model and a complete reflex control system. The latter includes cardiopulmonary and carotid sinus reflex limbs and interactions between the two. (2) We have modeled the physiologic stress of tilt table experiments and lower body negative pressure procedures (LBNP). We have verified our model's predictions by comparing them with experimental findings from the literature. (3) We have established collaborative efforts with leading investigators interested in experimental studies of orthostatic intolerance, cardiovascular control, and physiologic responses to space flight. (4) We have established a standardized method of transferring data to our laboratory from the ongoing NSBRI bedrest studies. We use this data to estimate input parameters to our model and compare our model predictions to actual data to further verify our model. (5) We are in the process of systematically simulating current hypotheses concerning the mechanism underlying orthostatic intolerance by matching our simulations to stand test data from astronauts pre- and post-flight. (6) We are in the process of developing a JAVA version of the simulator which will be distributed amongst the cardiovascular team members. Future work on this project involves modifications of the model to represent a rodent (rat) model, further evaluation of the bedrest astronaut and animal data, and systematic investigation of specific countermeasures.

Kamm, Roger D.↗

Evaluating an Automated Approach for Monitoring Forest Disturbances in the Pacific Northwest from Logging, Fire and Insect Outbreaks with Landsat Time Series Data

Forests are the largest aboveground sink for atmospheric carbon (C), and understanding how they change through time is critical to reduce our C-cycle uncertainties. We investigated a strong decline in Normalized Difference Vegetation Index (NDVI) from 1982 to 1991 in Pacific Northwest forests, observed with the National Ocean and Atmospheric Administration's (NOAA) series of Advanced Very High Resolution Radiometers (AVHRRs). To understand the causal factors of this decline, we evaluated an automated classification method developed for Landsat time series stacks (LTSS) to map forest change. This method included: (1) multiple disturbance index thresholds; and (2) a spectral trajectory-based image analysis with multiple confidence thresholds. We produced 48 maps and verified their accuracy with air photos, monitoring trends in burn severity data and insect aerial detection survey data. Area-based accuracy estimates for change in forest cover resulted in producer's and user's accuracies of 0.21 +/- 0.06 to 0.38 +/- 0.05 for insect disturbance, 0.23 +/- 0.07 to 1 +/- 0 for burned area and 0.74 +/- 0.03 to 0.76 +/- 0.03 for logging. We believe that accuracy was low for insect disturbance because air photo reference data were temporally sparse, hence missing some outbreaks, and the annual anniversary time step is not dense enough to track defoliation and progressive stand mortality. Producer's and user's accuracy for burned area was low due to the temporally abrupt nature of fire and harvest with a similar response of spectral indices between the disturbance index and normalized burn ratio. We conclude that the spectral trajectory approach also captures multi-year stress that could be caused by climate, acid deposition, pathogens, partial harvest, thinning, etc. Our study focused on understanding the transferability of previously successful methods to new ecosystems and found that this automated method does not perform with the same accuracy in Pacific Northwest forests. Using a robust accuracy assessment, we demonstrate the difficulty of transferring change attribution methods to other ecosystems, which has implications for the development of automated detection/attribution approaches. Widespread disturbance was found within AVHRR-negative anomalies, but identifying causal factors in LTSS with adequate mapping accuracy for fire and insects proved to be elusive. Our results provide a background framework for future studies to improve methods for the accuracy assessment of automated LTSS classifications.

Landsat↗

Preparing for Mars: The Evolvable Mars Campaign 'Proving Ground' Approach

As the National Aeronautics and Space Administration (NASA) prepares to extend human presence beyond Low Earth Orbit, we are in the early stages of planning missions within the framework of an Evolvable Mars Campaign. Initial missions would be conducted in near-Earth cis-lunar space and would eventually culminate in extended duration crewed missions on the surface of Mars. To enable such exploration missions, critical technologies and capabilities must be identified, developed, and tested. NASA has followed a principled approach to identify critical capabilities and a "Proving Ground" approach is emerging to address testing needs. The Proving Ground is a period subsequent to current International Space Station activities wherein exploration-enabling capabilities and technologies are developed and the foundation is laid for sustained human presence in space. The Proving Ground domain essentially includes missions beyond Low Earth Orbit that will provide increasing mission capability while reducing technical risks. Proving Ground missions also provide valuable experience with deep space operations and support the transition from "Earth-dependence" to "Earth-independence" required for sustainable space exploration. A Technology Development Assessment Team identified a suite of critical technologies needed to support the cadence of exploration missions. Discussions among mission planners, vehicle developers, subject-matter-experts, and technologists were used to identify a minimum but sufficient set of required technologies and capabilities. Within System Maturation Teams, known challenges were identified and expressed as specific performance gaps in critical capabilities, which were then refined and activities required to close these critical gaps were identified. Analysis was performed to identify test and demonstration opportunities for critical technical capabilities across the Proving Ground spectrum of missions. This suite of critical capabilities is expected to provide the foundation required to enable a variety of possible destinations and missions consistent with the Evolvable Mars Campaign.. The International Space Station will be used to the greatest extent possible for exploration capability and technology development. Beyond this, NASA is evaluating a number of options for Proving Ground missions. An "Asteroid Redirect Mission" will demonstrate needed capabilities (e.g., Solar Electric Propulsion) and transportation systems for the crew (i.e., Space Launch System and Orion) and for cargo (i.e., Asteroid Redirect Vehicle). The Mars 2020 mission and follow-on robotic precursor missions will gather Mars surface environment information and will mature technologies. NASA is considering emplacing a small pressurized module in cis-lunar space to support crewed operations of increasing duration and to serve as a platform for critical exploration capabilities testing (e.g., radiation mitigation; extended duration deep space habitation). In addition, "opportunistic mission operations" could demonstrate capabilities not on the Mars critical path that may, nonetheless, enhance exploration operations (e.g., teleoperations, crew assisted Mars sample return). The Proving Ground may also include "pathfinder" missions to test and demonstrate specific capabilities at Mars (e.g., entry, descent, and landing). This paper describes the (1) process used to conduct an architecture-driven technology development assessment, (2) exploration mission critical and supporting capabilities, and (3) approach for addressing test and demonstration opportunities encompassing the spectrum of flight elements and destinations consistent with the Evolvable Mars Campaign.

Bobskill, Marianne R.↗

Current Scientific Progress and Future Scientific Prospects Enabled by Spaceborne Precipitation Radar Measurements

First, we examine current scientific progress and understanding that have been possible through use of spaceborne precipitation radar measurements being provided by the TRMM and CloudSat satellites. Second, we look across a future 20-year time frame to assess how and why anticipated improvements in space radar systems will further advance scientific progress into topic areas once considered beyond the realm of space-based remote sensing. JAXA's 13.8 GHz Ku-band cross-track scanning Precipitation Radar (PR) developed for flight on NASA's non-sun-synchronous, diurnally-precessing TRMM satellite, was the first Earth radar flown in space that was designed specifically for precipitation measurement. Its proven accuracy in measuring global rainfall in the tropics and sub-tropics and its unanticipated longevity in continuing these measurements beyond a full decade have established the standards against which all follow-up and future space radars will be evaluated. In regards to the current PR measurement time series, we will discuss a selection of major scientific discoveries and impacts which have set the stage for future radar measuring systems. In fact, the 2nd contemporary space radar applicable for terrestrial precipitation measurement, i.e., JPL-CSA's 94 GHz nadir-staring Cloud Profiling Radar (CPR) flown on NASA's sun-synchronous CloudSat satellite, although designed primarily for measurement of non-precipitating cloud hydrometeors and aerosols, has also unquestionably advanced precipitation measurement because CPR's higher frequency and greatly increased sensitivity (approximately 30 dBZ) has enabled global observations of light rain rate spectrum processes (i.e., rain rates below 0.05 mm per hourand of precipitation processes in the high troposphere (particularly ice phase processes). These processes are beyond reach of the TRMM radar because the PR sensitivity limit is approximately 17 dBZ which means its lower rain rate cutoff is around 0.3 mm per hour and its vertical profiling acuity is greatly limited above the melting layer. Thus, the newer CPR measurements have become important for a variety of scientific reasons that will be highlighted and assessed. In considering scientific progress likely to stem from future precipitation radar systems, we will specifically examine possible scientific impacts from three anticipated missions for which NASA and various of its space agency partners are expected to lead the way. These three missions are: (1) the nearterm Global Precipitation Measuring (GPM) Mission; (2) the decadal timeline Aerosol and Cloud Experiment (ACE) Mission; and the post-decadal timeline NEXRAD in Space (NIS) Mission. The observational capabilities of the planned radar systems for each of these three satellite missions are distinct from each other and each provides progressive improvements in precipitation measuring and scientific research capabilities relative to where we are now -- insofar as TRMM PR and the CloudSat CPR capabilities. The potential innovations in scientific research will be discussed in a framework of likely synergisms between next-generation radar capabilities and accessible dynamical and microphysical properties that have heretofore evaded detection.

Smith, Eric A.↗

Evaluation of a Shuttle Derived Vehicle (SDV) for Cargo Transportation

In this new era of space exploration, a host of launch vehicles are being examined for possible use in transporting cargo and crew to low Earth orbit and beyond. Launch vehicles derived from the Space Shuttle Program (SSP), known as Shuttle Derived Vehicles (SDVs), are prime candidates for heavy-lift duty because of their potential to minimize non-recurring costs and because the Shuttle can leverage off proven high-performance flight systems with established ground and flight support. To determine the merits of SDVs, a detailed evaluation was performed. This evaluation included a trade study and risk assessment of options based on performance, safety reliability, cost, operations, and evolution. The purpose of this paper is to explain the approach, processes, and tools used to evaluate launch vehicles for heavy lift cargo transportation. The process included defining the trade space, characterizing the concepts, analyzing the systems, and scoring the options. The process also included a review by subject experts from NASA and industry to compare past and recent study data and assess the risks. A set of technical performance measures (TPMs) was generated based on the study requirements and constraints. Tools such as INTROS and POST were used to calculate performance, FIRST was used for prediction of reliability, and other software packages, both commercial and NASA-owned, were applied to study the trade space. By following a clear process and using the right tools a thorough assessment was performed. An SDV can be classified as either a side-mount vehicle (SMV) or an in-line vehicle OLV). An SMV is a Space Shuttle where the Orbiter is replaced by a cargo carrier. An ILV is comprised of a modified Shuttle External Tank (ET) with engines mounted to the bottom and cargo mounted atop. For both families of vehicles, Solid Rocket Boosters (SRBs) are attached to the ET. The first derivate of Shuttle is defined as the vehicle with minimum changes necessary to transform the Space Shuttle into an SDV. Deltas from the first derivate were also formulated to study more SDV options. Examples of deltas include replacing the SRBs with larger and/or more SRBs, adding an upper stage, increasing the size of the ET, changing the engines, and modifying the elements. Challenges for SDV range from tailoring infrastructure to meeting the exploration schedule. Although SDV is based on the Space Shuttle, it still includes development risk for designing and building a Cargo Carrier. There are also performance challenges in that Shuttle is not optimized for cargo-only missions, but it is a robust system built on reusability. Balancing the strengths and weaknesses of the Shuttle to meet Lunar and Mars mission objectives provides the framework for an informative trade study. SDV was carefully analyzed and the results of the study provide invaluable data for use in the new exploration initiative.

Roman, Jose M.↗

Identification and Control of Aircrafts using Multiple Models and Adaptive Critics

We compared two possible implementations of local linear models for control: one approach is based on a self-organizing map (SOM) to cluster the dynamics followed by a set of linear models operating at each cluster. Therefore the gating function is hard (a single local model will represent the regional dynamics). This simplifies the controller design since there is a one to one mapping between controllers and local models. The second approach uses a soft gate using a probabilistic framework based on a Gaussian Mixture Model (also called a dynamic mixture of experts). In this approach several models may be active at a given time, we can expect a smaller number of models, but the controller design is more involved, with potentially better noise rejection characteristics. Our experiments showed that the SOM provides overall best performance in high SNRs, but the performance degrades faster than with the GMM for the same noise conditions. The SOM approach required about an order of magnitude more models than the GMM, so in terms of implementation cost, the GMM is preferable. The design of the SOM is straight forward, while the design of the GMM controllers, although still reasonable, is more involved and needs more care in the selection of the parameters. Either one of these locally linear approaches outperform global nonlinear controllers based on neural networks, such as the time delay neural network (TDNN). Therefore, in essence the local model approach warrants practical implementations. In order to call the attention of the control community for this design methodology we extended successfully the multiple model approach to PID controllers (still today the most widely used control scheme in the industry), and wrote a paper on this subject. The echo state network (ESN) is a recurrent neural network with the special characteristics that only the output parameters are trained. The recurrent connections are preset according to the problem domain and are fixed. In a nutshell, the states of the reservoir of recurrent processing elements implement a projection space, where the desired response is optimally projected. This architecture trades training efficiency by a large increase in the dimension of the recurrent layer. However, the power of the recurrent neural networks can be brought to bear on practical difficult problems. Our goal was to implement an adaptive critic architecture implementing Bellman s approach to optimal control. However, we could only characterize the ESN performance as a critic in value function evaluation, which is just one of the pieces of the overall adaptive critic controller. The results were very convincing, and the simplicity of the implementation was unparalleled.

Principe, Jose C.↗

Use of Collocated KWAJEX Satellite, Aircraft, and Ground Measurements for Understanding Ambiguities in TRMM Radiometer Rain Profile Algorithm

Coordinated ground, aircraft, and satellite observations are analyzed from the 1999 TRMM Kwajalein Atoll field experiment (KWAJEX) to better understand the relationships between cloud microphysical processes and microwave radiation intensities in the context of physical evaluation of the Level 2 TRMM radiometer rain profile algorithm and uncertainties with its assumed microphysics-radiation relationships. This talk focuses on the results of a multi-dataset analysis based on measurements from KWAJEX surface, air, and satellite platforms to test the hypothesis that uncertainties in the passive microwave radiometer algorithm (TMI 2a12 in the nomenclature of TRMM) are systematically coupled and correlated with the magnitudes of deviation of the assumed 3-dimensional microphysical properties from observed microphysical properties. Re-stated, this study focuses on identifying the weaknesses in the operational TRMM 2a12 radiometer algorithm based on observed microphysics and radiation data in terms of over-simplifications used in its theoretical microphysical underpinnings. The analysis makes use of a common transform coordinate system derived from the measuring capabilities of the aircraft radiometer used to survey the experimental study area, i.e., the 4-channel AMPR radiometer flown on the NASA DC-8 aircraft. Normalized emission and scattering indices derived from radiometer brightness temperatures at the four measuring frequencies enable a 2-dimensional coordinate system that facilities compositing of Kwajalein S-band ground radar reflectivities, ARMAR Ku-band aircraft radar reflectivities, TMI spacecraft radiometer brightness temperatures, PR Ku-band spacecraft radar reflectivities, bulk microphysical parameters derived from the aircraft-mounted cloud microphysics laser probes (including liquid/ice water contents, effective liquid/ice hydrometeor radii, and effective liquid/ice hydrometeor variances), and rainrates derived from any of the individual ground, aircraft, or satellite algorithms applied to the radar or radiometer measurements, or their combination. The results support the study's underlying hypothesis, particularly in context of ice phase processes, in that the cloud regions where the 2a12 algorithm's microphysical database most misrepresents the microphysical conditions as determined by the laser probes, are where retrieved surface rainrates are most erroneous relative to other reference rainrates as determined by ground and aircraft radar. In reaching these conclusions, TMI and PR brightness temperatures and reflectivities have been synthesized from the aircraft AMPR and ARMAR measurements with the analysis conducted in a composite framework to eliminate measurement noise associated with the case study approach and single element volumes obfuscated by heterogeneous beam filling effects. In diagnosing the performance of the 2a12 algorithm, weaknesses have been found in the cloud-radiation database used to provide microphysical guidance to the algorithm for upper cloud ice microphysics. It is also necessary to adjust a fractional convective rainfall factor within the algorithm somewhat arbitrarily to achieve satisfactory algorithm accuracy.

Smith, Eric A.↗

Developing Concepts of Operations Using Multi-Step Tool Techniques With Large Language Models

The National Aeronautics and Space Administration (NASA) Air Mobility Pathfinders (AMP) project is developing and evaluating concepts of operations (ConOps) for safe, secure, and scalable Urban Air Mobility (UAM) operations. The AMP project’s Operational Concepts, Architecture, and Requirements Integration (OCARI) Team is using a Model Based System Engineering (MBSE) approach for integration, interoperability, and traceability of Advanced Air Mobility (AAM) ecosystems centered around urban air taxi services. The team’s goal is to define structures and behaviors needed for system feasibility, readiness, and interoperability, establish a UAM knowledge base, and trace and validate assumptions and requirements relevant to AAM. NASA Langley Research Center (LaRC) is spearheading an innovative digital engineering approach to integrate, communicate, and facilitate the research of multi-modal transportation systems. The Knowledge-based Digital Platform (KbDP) is a concept being developed that ties the workflows of Project Managers (PM), Principal Investigators (PI), and System Engineers together across organizational boundaries. It does so through the management of an information database defined by mathematical, data science, and system engineering principles. Machine Learning (ML) algorithms play a key role in this concept by extracting meaningful knowledge from relational and graph databases, document repositories, and system artifacts, which the human user leverages to greatly improve the efficiency and effectiveness of their research. Recent advancements in the field of Large Language Models (LLMs), specifically models trained for tool use, such as Command-R , now allow for the reliable implementation of single-step and multi-step tool-centric systems. These techniques provide the LLM with a set of tools, in our case Python functions, that can be called on to answer a much wider range of questions compared to LLMs implemented using a traditional single-source or Retrieval Augmented Generation (RAG) approach. Through this method, the LLM can pull information from multiple data sources, such as relational or graph databases, document repositories, application programming interfaces (APIs), and SysML artifacts depending on the user’s question. The LLM can also output the information in a variety of different formats, using output generation tools, such as CSV, UML, or SysML artifacts. Additionally, tools can be assigned roles and can work together to provide answers to queries in an “agent” like approach, similar to that implemented by Microsoft’s AutoGen framework where different agents can converse with each other to accomplish tasks. Previously, our team developed a chatbot system with “agent like” functionality in the form of different “modes” the user could select from a user interface (UI), this architecture can be seen on the left in figure 1. Three different modes were implemented, the first mode allowed the LLM to utilize the structures and algorithms within a graph database to trace UAM requirements. The second mode gave the LLM access to a vector search capable of providing relevant information from thousands of document pages related to UAM ConOps and requirements. The third mode served as a general assistant where users could enter open-ended questions and custom prompts to utilize the LLM for different use-cases. This system improved the process surrounding generating and analyzing information related to UAM requirements, however, the implementation provided a clunky user experience. Users were required to know what mode to select within the UI in advance before entering their question to the selected tool. Moreover, the different tools were isolated from each other, they lacked bidirectional links that would allow for tools to collaborate to generate better responses. Our team is working on a new architecture, seen on the right in the below figure, with the goal to address many of the UX shortcomings of our original system while improving the accuracy and depth of responses from the LLM. This new system will automatically select the appropriate tool to use based off the user’s question. Each tool will be capable of calling on any of the other tools available to the LLM, resulting in a collaborative pipeline where tools can pass data between other tools until enough data is received to generate an answer to the user’s question. Using a locally deployed, open-source, LLM, the NASA OCARI team, in collaboration with Collins Aerospace, will implement a prototype application that will bridge knowledge across multiple sources to assist System Engineers (SEs) with requirements discovery and tracing, research question and use case identification, and assumption validation. Such a system will also allow SEs to more easily, and intuitively, explore the AAM ecosystem, ultimately improving the efficiency and effectiveness of the SE's research and decision-making processes surrounding ConOps development and validation. In this session, our team will provide a video demonstration of our new prototype architecture in action. We will also present an overview of our prototype system architecture and talk about its advantages over traditional LLM deployments along with how those advantages can provide additional value to the field of System Engineering.

systems engineering↗

Observer-Based Magnetic Bearing Controller Developed for Aerospace Flywheels

A prototype of a versatile, observer-based magnetic bearing controller for aerospace flywheels was successfully developed and demonstrated on a magnetic bearing test rig (see the photograph) and an actual flywheel module. The objective of this development included a fast, yet low risk, control development process, and a robust, high-performance controller for a large variety of flywheels. This required a good system model, an efficient development procedure, and a model-based controller that addressed the key problems associated with flywheel and bearing imbalance, sensor error, and vibration. The model used in this control development and tuning procedure included the flexible rotor dynamics and motor-induced vibrations. Such a model was essential for low-risk scheduling of speed-dependent control parameters and for reliable evaluation of novel control strategies. The successfully tested control prototype utilized an extended Kalman filter to estimate the true rotor principal-axis motion from the raw sensor position feedback. For control refinement, the extended Kalman filter also estimated and eliminated the combined effects of mass-imbalance and sensor runouts from the input data. A key advantage of the design based on the extended Kalman filter is its ability to accurately estimate both the rotor's principal-axis position and gyroscopic rates with the least amount of phase lag. This is important for control parameter scheduling to dampen the gyroscopic motions. Because of large uncertainties in the magnetic bearing and imbalance characteristics, this state-estimation scheme alone is insufficient for containing the rotor motion within the desired 1-mil excursion radius. A nonlinear gain adjustment based on an estimation of the principal-axis orbit size was needed to provide a coarse (nonoptimal), but robust, control of the orbit growth. Control current minimization was achieved with a (steepest gradient) search of synchronous errors in the principal-axis position input data. Actual flywheel tests of this observer-based controller (developed entirely in-house) at the NASA Glenn Research Center showed that the model correctly predicted the rotor orbit growth as a function of rotational speed, and it demonstrated the capability of gain adjustments to arrest this growth. Data from these tests on an actual flywheel module spun to 26,000 rpm proved that the controller was able to contain the shaft motion to within much less than 0.5 mils of radial excursion with axis currents less than 300 mA in root-mean-square estimate. The test speed range was limited because of thermal expansion concerns for this particular flywheel unit, not because of any deficiency in the controller. Simulations for this unit indicated that the controller should be robust up to its top operating speed of 60,000 rpm. Aside from these important achievements, and most significantly, it took less than 1 week to adapt this controller from the simple test rig to the actual flywheel and to demonstrate full five-axis levitation and control. This demonstration showed that both the controller and the model-based development and tuning framework are easily adaptable to a wide range of rotors and bearing configurations and, hence, are capable of reducing design risks and costs for many future flywheel technology developments.

Le, Dzu K.↗

Open-source Numerical Modeling of Solidification Cracking Susceptibility: Application to Refractory Alloy Systems

Introduction. Alloys such as aluminum, nickel-base, and austenitic stainless steels are susceptible to solidification cracking during welding and 3D printing. Compositional optimization is one method used to effectively mitigate solidification cracking of those alloy systems. With the surge in hypersonic and in-space propulsion activities, refractory metals (Nb, Mo, Ta, W, and Re) and their alloy derivatives are increasing in importance due to their extreme high melting point and retention of high-temperature strength; however, their chemistry was most typically optimized to promote ductility during mechanical operations such as drawing and forming. Welding of such alloys has been a challenge due to a number of issues including solidification cracking, atmospheric contamination (O, C, and N), as well as a shift in ductile-to-brittle transition to higher temperature following grain growth induced by welding. Compositional optimization of refractory alloys for solidification cracking resistance in particular is desirable as their usage increases with the advent of advanced manufacturing methods such as 3D printing. This work evaluates the effect of compositional variation in refractory metal systems on the solidification cracking susceptibility with the goals of optimizing existing alloys and joining process techniques, and formulating new alloys with increased solidification cracking resistance. Experimental Procedures. A python code was developed in a Jupyter notebook environment (Michael and Sowards, 2023) to facilitate the calculation of crack susceptibility index proposed by Kou (2015). Composition is entered as a single point, or as a 1-D or 2-D array. The notebook calls pycalphad (Otis and Liu, 2017 and Bocklund et al, 2020) to calculate the evolution of fraction solid as a function of temperature (under either Scheil or equilibrium assumptions) and then evaluates steepness of the fraction solid curve near the terminal stage of solidification to predict solidification cracking resistance. Open source thermodynamic databases available at online repositories are used (van de Walle). The process is setup in an automated fashion to generate plots that show variation in solidification cracking susceptibility according to composition on 1-D line plots or 2-D contour plots. The Jupyter notebook and crack susceptibility algorithm was also integrated with a widely used commercial CALPHAD code for validation and alloy exploration. Results and Discussion. The crack susceptibility model was first validated against a series of refractory alloy compositions evaluated in past work which utilized a specialized Varestraint test built inside a vacuum chamber environment (Lessman and Gold, 1971). The alloys tested in the Varestraint apparatus included T-111 (Ta-8W-2Hf), ASTAR-811C (Ta-8W-1Re-0.7Hf-0.025C), FS-85 (Nb-27Ta-10W-1Zr), T-222 (Ta-9.6W-2.4Hf-0.01C), Ta-10W, B-66 (Nb-5Mo-5V-1Zr), and SCb-291 (Nb-10W-10Ta). The initial test of the model showed a strong correlation with empirical Varestraint data, i.e., a Spearman rank correlation between model predictions and hot cracking measurements was observed to be greater than 0.8. Following the validation, a set of refractory metal binary mixtures was investigated to evaluate sensitivity of Nb, Mo, W, and Ta to C, N, and O content. A series of plots were produced that suggest ppmw ranges of C, N, and O where solidification cracking increases significantly and reaches a maximum. Also comparative ranking of each primary refractory metal to each interstitial was produced. For example C produces greater cracking response in Mo whereas O produces greater cracking response in Ta and Nb. Such compositional values have utility in setting limits on pickup of these interstitial elements during welding and printing rather than using a one-size-fits-all approach. Furthermore, the results have use in determining additive powder recycling requirements, which is especially pertinent for refractory metal powders due to their high cost compared to conventional alloys. Another application created thousands of hypothetical alloys within the nominal specified composition range of two widely used refractory alloys C103 (Nb-10Hf-1Ti) and TZM (Mo-0.5Ti-0.1Zr). The cracking index was calculated for the alloys and results were fed into machine learning regression techniques including Multiple Linear Regression, Ridge Regression, and Lasso Regression to determine relative potency each alloying element had on computed solidification cracking index. A series of linear equations were produced that relate composition of C103 and TZM to solidification cracking index. The crack susceptibility of C103 for example is described by an equation of the form: cracking index ~ O + 0.667*C + 0.635*N + 0.00037*Ta – 0.0008*Hf (in wt.%) From that equation, it is clear that O has strong propensity to induce solidification cracking. Interestingly, Hf is shown to reduce calculated cracking response. Finally, realizing the potential of this method to discover new refractory alloy formulations across the period table that have low solidification cracking sensitivity, the code was applied to new untested alloy systems including W-Zr-C, W-Ta-C, and others. Conclusions. In summary, an open source numerical method has been developed using Python code to calculate Kou’s crack susceptibility index. The method was applied to refractory metals which are inherently difficult to study from a weldability testing standpoint since inert shielding gas is not sufficient and welding is typically done in vacuum, especially in light of findings presented here where oxygen has profound influence on solidification cracking. This work revealed the effect of compositional variations on a series of refractory metals and showed the framework defined here will be useful in 1) the development of new alloys that have improved weldability and 3D printability, 2) placing compositional limits on existing alloys, and 3) ensuring adequate controls of manufacturing processes such as 3D printing where powder reuse is critical. Keywords. pycalphad; Python; refractory metals; solidification cracking. References. B. Bocklund et. al. (2020) http://doi.org/10.5281/zenodo.3630657. S. Kou. (2015) https://doi.org/10.1016/j.actamat.2015.01.034. G.G. Lessmann and R.E. Gold. Welding Journal, issue 1, pp. 1-s – 8-s (1971). F.N. Michael and J.W. Sowards. NASA/TM-20230002218 (2023). R. Otis and Z.-K. Liu. (2017) http://doi.org/10.5334/jors.140. A. Van de Wallle et. al. (2018) https://doi.org/10.1016/j.calphad.2018.04.003.

pycalphad↗