Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

Assimilation of Soil Moisture Observations Over Land Improves Analysis and Prediction of Tropical Cyclone Idai

Soil moisture conditions can impact the circulation and structure of a tropical cyclone (TC) when part or all of the circulation is over land. Dry land surface conditions may lead to faster dissipation of a TC over land, whereas very wet conditions may lead to a prolonged maintenance of its intensity. While this relationship is relatively well understood in theory, applications of these findings in the context of numerical weather prediction (NWP) have been limited. Here we present a case study that explores the potential of improving TC predictions through an improved soil moisture initialization in an NWP framework. Specifically, we examine the impact of assimilating observations from the NASA Soil Moisture Active Passive (SMAP) mission into the NASA Goddard Earth Observing System (GEOS) global weather model on the prediction of South-West Indian Ocean TC Idai (2019). SMAP provides accurate L-band (1.4 GHz) brightness temperatures (Tb) observations that are sensitive to soil moisture globally and at high revisit times of 2-3 days. It has previously been shown that the assimilation of SMAP Tbs significantly improves modeled land surface states. Here we evaluate: (i) forecasts initialized from an analysis that is comparable to the GEOS operational analysis (without SMAP Tb assimilation) and (ii) forecasts initialized from an analysis that additionally assimilates SMAP Tb observations. We find that in the analysis with SMAP assimilation, the TC has a better-defined, more aligned vertical structure over land relative to the control run; moreover, the analyzed TC size, as measured by the wind speed radius, better matches the observed TC size. We further find significant reductions in the forecast intensity error and the forecast along-track error, measured against observations. The largest error reductions occur at lead times of 36 to 72 hours, suggesting that the land with its longer memory gains in importance as a source of predictability at this timescale. An investigation of the underlying mechanisms leading to the skill improvements from SMAP data assimilation revealed that the assimilation of SMAP leads to wetter soil moisture conditions and an increased latent heat flux in the SMAP analysis, which results in a TC with higher column-integrated total moisture content and total energy compared to the control analysis.

Jana Kolassa↗

Pathways of Understanding: the Interactions of Humanity and Global Environmental Change

How humans, interacting within social systems, affect and are affected by global change is explored. Recognizing the impact human activities have on the environment and responding to the need to document the interactions among human activities, the Consortium for International Earth Science Information Network (CIESIN) commissioned a group of 12 scientists to develop a framework illustrating the key human systems that contribute to global change. This framework, called the Social Process Diagram, will help natural and social scientists, educators, resource managers and policy makers envision and analyze how human systems interact among themselves and with the natural system. The Social Process Diagram consists of the following blocks that constitute the Diagram's structural framework: (1) fund of knowledge and experience; (2) preferences and expectations; (3) factors of production and technology; (4) population and social structure; (5) economic systems; (6) political systems and institutions; and (7) global scale environmental processes. To demonstrate potential ways the Diagram can be used, this document includes 3 hypothetical scenarios of global change issues: global warming and sea level rise; the environmental impact of human population migration; and energy and the environment. These scenarios demonstrate the Diagram's usefulness for visualizing specific processes that might be studied to evaluate a particular global change issues. The scenario also shows that interesting and unanticipated questions may emerge as links are explored between categories on the Diagram.

Jacobson, Harold K.↗

The Empirical Analysis of Impact of Alliances on Airline Operations

Airline alliances are dominating the current air transport industry with the largest carriers of the world belonging to one of the four alliance groupings - "Wings", Star Alliance, one world, SkyTeam - which represent 56% of world Revenue Passenger Kilometers. Although much research has been carried out to evaluate the impact of alliance membership on performance of airlines, it would be of interest to ascertain the degree of impact perceived by participating airlines in alliances. It is the purpose of this paper to gather the opinion of all the airlines, belonging to the four global alliance groupings on the impact alliances have had on their traffic and on their performance in general To achieve this, a comprehensive survey of the alliance management departments of airlines participating in the four global strategic alliances was carried out. With this framework the survey has examined which type of cooperation among carriers (FFP, Code Share, Strategic Alliance without antitrust immunity, Strategic Alliance with antitrust immunity) has produced the most positive impact on traffic and which type of route (short haul, long haul, hub-hub, hub-non hub, non hub-non hub) has been mostly affected. In addition, the respondent airlines quantified the effect alliances have had on specific areas of their operation, such as load factors, traffic, costs, revenue and fares. Their responses have been analysed under each global alliances grouping, under airline and under geographic region to establish which group, type of carrier and geographic region has benefited most. The results show that each of the four global alliances groupings has experienced different results according to the type of collaboration agreed amongst their member airlines.

Iatrou, Kostas↗

Bounding the Role of Black Carbon in the Climate System: a Scientific Assessment

Black carbon aerosol plays a unique and important role in Earth's climate system. Black carbon is a type of carbonaceous material with a unique combination of physical properties. This assessment provides an evaluation of black-carbon climate forcing that is comprehensive in its inclusion of all known and relevant processes and that is quantitative in providing best estimates and uncertainties of the main forcing terms: direct solar absorption; influence on liquid, mixed phase, and ice clouds; and deposition on snow and ice. These effects are calculated with climate models, but when possible, they are evaluated with both microphysical measurements and field observations. Predominant sources are combustion related, namely, fossil fuels for transportation, solid fuels for industrial and residential uses, and open burning of biomass. Total global emissions of black carbon using bottom-up inventory methods are 7500 Gg/yr in the year 2000 with an uncertainty range of 2000 to 29000. However, global atmospheric absorption attributable to black carbon is too low in many models and should be increased by a factor of almost 3. After this scaling, the best estimate for the industrial-era (1750 to 2005) direct radiative forcing of atmospheric black carbon is +0.71 W/sq m with 90% uncertainty bounds of (+0.08, +1.27)W/sq m. Total direct forcing by all black carbon sources, without subtracting the preindustrial background, is estimated as +0.88 (+0.17, +1.48) W/sq m. Direct radiative forcing alone does not capture important rapid adjustment mechanisms. A framework is described and used for quantifying climate forcings, including rapid adjustments. The best estimate of industrial-era climate forcing of black carbon through all forcing mechanisms, including clouds and cryosphere forcing, is +1.1 W/sq m with 90% uncertainty bounds of +0.17 to +2.1 W/sq m. Thus, there is a very high probability that black carbon emissions, independent of co-emitted species, have a positive forcing and warm the climate. We estimate that black carbon, with a total climate forcing of +1.1 W/sq m, is the second most important human emission in terms of its climate forcing in the present-day atmosphere; only carbon dioxide is estimated to have a greater forcing. Sources that emit black carbon also emit other short-lived species that may either cool or warm climate. Climate forcings from co-emitted species are estimated and used in the framework described herein. When the principal effects of short-lived co-emissions, including cooling agents such as sulfur dioxide, are included in net forcing, energy-related sources (fossil fuel and biofuel) have an industrial-era climate forcing of +0.22 (0.50 to +1.08) W/sq m during the first year after emission. For a few of these sources, such as diesel engines and possibly residential biofuels, warming is strong enough that eliminating all short-lived emissions from these sources would reduce net climate forcing (i.e., produce cooling). When open burning emissions, which emit high levels of organic matter, are included in the total, the best estimate of net industrial-era climate forcing by all short-lived species from black-carbon-rich sources becomes slightly negative (0.06 W/sq m with 90% uncertainty bounds of 1.45 to +1.29 W/sq m). The uncertainties in net climate forcing from black-carbon-rich sources are substantial, largely due to lack of knowledge about cloud interactions with both black carbon and co-emitted organic carbon. In prioritizing potential black-carbon mitigation actions, non-science factors, such as technical feasibility, costs, policy design, and implementation feasibility play important roles. The major sources of black carbon are presently in different stages with regard to the feasibility for near-term mitigation. This assessment, by evaluating the large number and complexity of the associated physical and radiative processes in black-carbon climate forcing, sets a baseline from which to improve future climate forcing estimates.

black carbon↗

Overview of the current MITRE effort

A logical view of the applications data service (ADS) as a distributed system was adopted. A hierarchical classification scheme of ADS features comprising 70 modes was developed. The framework for a preliminary assessment of the applicability Federal, national, and international standards to ADS was provided by the classification. Approximately 300 standards were gathered, screened, and documented. The survey was enlarged to incorporate standards from NASA headquarters and centers. Two efforts were initiated to (1) survey methodologies of the three ADS pilots, and (2) identify the requirements for standards of ADS members based on a survey of the pilots and on representative future members. Preliminary results of the requirements survey were used in the development of criteria for the evaluation of candidate standards. An evaluation process was designed incorporating these criteria.

Kuch, T.↗

Reciprocity-based experimental determination of dynamic forces and moments: A feasibility study

BBN Systems and Technologies has been tasked by the Georgia Tech Research Center to carry Task Assignment No. 7 for the NASA Langley Research Center to explore the feasibility of 'In-Situ Experimental Evaluation of the Source Strength of Complex Vibration Sources Utilizing Reciprocity.' The task was carried out under NASA Contract No. NAS1-19061. In flight it is not feasible to connect the vibration sources to their mounting points on the fuselage through force gauges to measure dynamic forces and moments directly. However, it is possible to measure the interior sound field or vibration response caused by these structureborne sound sources at many locations and invoke principle of reciprocity to predict the dynamic forces and moments. The work carried out in the framework of Task 7 was directed to explore the feasibility of reciprocity-based measurements of vibration forces and moments.

Ver, Istvan L.↗

Intelligent Agent Architectures: Reactive Planning Testbed

An Integrated Agent Architecture (IAA) is a framework or paradigm for constructing intelligent agents. Intelligent agents are collections of sensors, computers, and effectors that interact with their environments in real time in goal-directed ways. Because of the complexity involved in designing intelligent agents, it has been found useful to approach the construction of agents with some organizing principle, theory, or paradigm that gives shape to the agent's components and structures their relationships. Given the wide variety of approaches being taken in the field, the question naturally arises: Is there a way to compare and evaluate these approaches? The purpose of the present work is to develop common benchmark tasks and evaluation metrics to which intelligent agents, including complex robotic agents, constructed using various architectural approaches can be subjected.

Rosenschein, Stanley J.↗

Photochemical Phenomenology Model for the New Millennium

The "Photochemical Phenomenology Model for the New Millennium" project tackles the issue of reengineering and extension of validated physics-based modeling capabilities ("legacy" computer codes) to application-oriented software for use in science and science-support activities. While the design and architecture layouts are in terms of general particle distributions involved in scattering, impact, and reactive interactions, initial Photochemical Phenomenology Modeling Tool (PPMT) implementations are aimed at construction and evaluation of photochemical transport models with rapid execution for use in remote sensing data analysis activities in distributed systems. Current focus is on the Composite Infrared Spectrometer (CIRS) data acquired during the CASSINI flyby of Jupiter. Overall, the project has stayed on the development track outlined in the Year 1 annual report and most Year 2 goals have been met. The issues that have required the most attention are: implementation of the core photochemistry algorithms; implementation of a functional Java Graphical User Interface; completion of a functional CORBA Component Model framework; and assessment of performance issues. Specific accomplishments and the difficulties encountered are summarized in this report. Work to be carried out in the next year center on: completion of testing of the initial operational implementation; its application to analysis of the CASSINI/CIRS Jovian flyby data; extension of the PPMT to incorporate additional phenomenology algorithms; and delivery of a mature operational implementation.

Bishop, James↗

Spaceflight Operations Services Grid (SOSG)

In an effort to adapt existing space flight operations services to new emerging Grid technologies we are developing a Grid-based prototype space flight operations Grid. This prototype is based on the operational services being provided to the International Space Station's Payload operations located at the Marshall Space Flight Center, Alabama. The prototype services will be Grid or Web enabled and provided to four user communities through portal technology. Users will have the opportunity to assess the value and feasibility of Grid technologies to their specific areas or disciplines. In this presentation descriptions of the prototype development, User-based services, Grid-based services and status of the project will be presented. Expected benefits, findings and observations (if any) to date will also be discussed. The focus of the presentation will be on the project in general, status to date and future plans. The End-use services to be included in the prototype are voice, video, telemetry, commanding, collaboration tools and visualization among others. Security is addressed throughout the project and is being designed into the Grid technologies and standards development. The project is divided into three phases. Phase One establishes the baseline User-based services required for space flight operations listed above. Phase Two involves applying Gridlweb technologies to the User-based services and development of portals for access by users. Phase Three will allow NASA and end users to evaluate the services and determine the future of the technology as applied to space flight operational services. Although, Phase One, which includes the development of the quasi-operational User-based services of the prototype, development will be completed by March 2004, the application of Grid technologies to these services will have just begun. We will provide status of the Grid technologies to the individual User-based services. This effort will result in an extensible environment that incorporates existing and new spaceflight services into a standards-based framework providing current and future NASA programs with cost savings and new and evolvable methods to conduct science. This project will demonstrate how the use of new programming paradigms such as web and grid services can provide three significant benefits to the cost-effective delivery of spaceflight services. They will enable applications to operate more efficiently by being able to utilize pooled resources. They will also permit the reuse of common services to rapidly construct new and more powerful applications. Finally they will permit easy and secure access to services via a combination of grid and portal technology by a distributed user community consisting of NASA operations centers, scientists, the educational community and even the general population as outreach. The approach will be to deploy existing mission support applications such as the Telescience Resource Kit (TReK) and new applications under development, such as the Grid Video Distribution System (GViDS), together with existing grid applications and services such as high-performance computing and visualization services provided by NASA s Information Power Grid (IPG) in the MSFC s Payload Operations Integration Center (POIC) HOSC Annex. Once the initial applications have been moved to the grid, a process will begin to apply the new programming paradigms to integrate them where possible. For example, with GViDS, instead of viewing the Distribution service as an application that must run on a single node, the new approach is to build it such that it can be dispatched across a pool of resources in response to dynamic loads. To make this a reality, reusable services will be critical, such as a brokering service to locate appropriate resource within the pool. This brokering service can then be used by other applications such as the TReK. To expand further, if the GViDS application is constructed using a services-based mel, then other applications such as the Video Auditorium can then use GViDS as a service to easily incorporate these video streams into a collaborative conference. Finally, as these applications are re-factored into this new services-based paradigm, the construction of portals to integrate them will be a simple process. As a result, portals can be tailored to meet the requirements of specific user communities.

Bradford, Robert N.↗

An Evaluation of Clouds and Radiation in a Large-Scale Atmospheric Model Using a Cloud Vertical Structure Classification

We revisit the concept of the cloud vertical structure (CVS) classes we have previously employed to classify the planet's cloudiness (Oreopoulos et al., 2017). The CVS classification reflects simple combinations of simultaneous cloud occurrence in the three standard layers traditionally used to separate low, middle, and high clouds and was applied to a dataset derived from active lidar and cloud radar observations. This classification is now introduced in an atmospheric global climate model, specifically a version of NASA's GEOS-5, in order to evaluate the realism of its cloudiness and of the radiative effects associated with the various CVS classes. Such classes can be defined in GEOS-5 thanks to a sub column cloud generator paired with the model's radiative transfer algorithm, and their associated radiative effects can be evaluated against observations. We find that the model produces 50% more clear skies than observations in relative terms and produces isolated high clouds that are slightly less frequent than in observations, but optically thicker, yielding excessive planetary and surface cooling. Low clouds are also brighter than in observations, but underestimates of the frequency of occurrence (by ~20% in relative terms) help restore radiative agreement with observations. Overall the model better reproduces the longwave radiative effects of the various CVS classes because cloud vertical location is substantially constrained in the CVS framework.

Lee, Dongmin↗

Modernization of software quality assurance

The customers satisfaction depends not only on functional performance, it also depends on the quality characteristics of the software products. An examination of this quality aspect of software products will provide a clear, well defined framework for quality assurance functions, which improve the life-cycle activities of software development. Software developers must be aware of the following aspects which have been expressed by many quality experts: quality cannot be added on; the level of quality built into a program is a function of the quality attributes employed during the development process; and finally, quality must be managed. These concepts have guided our development of the following definition for a Software Quality Assurance function: Software Quality Assurance is a formal, planned approach of actions designed to evaluate the degree of an identifiable set of quality attributes present in all software systems and their products. This paper is an explanation of how this definition was developed and how it is used.

Bhaumik, Gokul↗

Job Scheduling in a Heterogeneous Grid Environment

Computational grids have the potential for solving large-scale scientific problems using heterogeneous and geographically distributed resources. However, a number of major technical hurdles must be overcome before this potential can be realized. One problem that is critical to effective utilization of computational grids is the efficient scheduling of jobs. This work addresses this problem by describing and evaluating a grid scheduling architecture and three job migration algorithms. The architecture is scalable and does not assume control of local site resources. The job migration policies use the availability and performance of computer systems, the network bandwidth available between systems, and the volume of input and output data associated with each job. An extensive performance comparison is presented using real workloads from leading computational centers. The results, based on several key metrics, demonstrate that the performance of our distributed migration algorithms is significantly greater than that of a local scheduling framework and comparable to a non-scalable global scheduling approach.

Shan, Hong-Zhang↗

Evaluation of User Experience of Self-Scheduling Software for Astronauts: Defining a Satisfaction Baseline

As NASA turns its sights to deep-space exploration, a greater focus on sup-porting crew autonomy has led to the development of Playbook, a self-scheduling software tool. Evaluating the user satisfaction of Playbook is essential in ensuring its usability for critical spaceflight operations. Satisfaction of an interface is often quantified with attitude surveys, such as the User Experience Questionnaire (UEQ). This paper demonstrates an application of the UEQ in comparing the user experience of Playbook interface de-signs for displaying graphical data. We lay the foundation for future user experience comparisons by defining a satisfaction baseline, which is crucial as more features are integrated into Playbook’s interface. This work ex-tends a validated user experience framework into a spaceflight domain, allowing optimization of human-computer interaction as future operational tools are developed.

user experience↗

Systems engineering studies of lunar base construction

Many ingenious concepts have been proposed for lunar base construction, but few systematic studies exist which relate time-consistent lunar base construction technologies and the choice of lunar base approach with the long-term SEI objectives - i.e., lunar indigenous base construction and Mars Exploration equipment development. To fill this gap, CSC has taken a two-pronged approach. First, the Center undertook basic geotechnical investigations of lunar soil, fabrication of a scale prototype of a lunar construction crane, a multi-robot construction team laboratory experiment, and a preliminary design of lunar base structures. Second, during Jun. and Jul. 1991 two lunar base construction systems engineering studies were accomplished - a 'near term lunar base' study, and a 'far-term lunar base' study. The goals of these studies were to define the major lunar base construction research problems in consistent technology/construction frameworks, and to define design requirements for construction equipment such as a lunar crane and a regolith mover. The 'near-term lunar base' study examined three different construction concepts for a lunar base comprised of pre-fabricated, pre-tested, Space Station Freedom-type modules, which would be covered with regolith shielding. Concept A used a lunar crane for unloading and transportation; concept B, a winch and cart; and concept C, a walker to move the modules from the landing site to the base site and assemble them. To evaluate the merits of each approach, calculations were made of mass efficiency measure, source mass, reliability, far-term base mass, Mars base mass, and base assembly time. The model thus established was also used to define the requirements for crane speed and regolith mover m(sup 3)/sec rates. A major problem addressed is how to 'mine' the regolith and stack it over the habitats as shielding. To identify when the cost of using indigenous lunar materials to construct the base exceeds the cost of development and delivery of the equipment for processing lunar materials, a study of construction of a candidate sintered regolith 'far term lunar base' was undertaken. A technique was devised for casting slabs of sintered (basaltic) regolith and assembling these into a hemispherical (or geodesic) dome. The major problem occurs with the inner liner. At 14.7 psi and 20 percent oxygen internal atmosphere, the entire structure is in tension, even with the regolith load. Also, another study has indicated that at 14.7 psi major resupply of air will be needed because of leakage, and astronauts may have to engage in extensive pre-breathing and post-breathing for extravehicular activity (EVA) tasks, thus detracting from useful mission work time. An alternative is to operate part of the base at, say, 5 psi and 70 percent oxygen, or to equip the astronauts with hard suits at 8.3 psi or greater. All of these choices directly influence base design and construction techniques.

Morgenthaler, George W.↗

On the design of fault-tolerant robotic manipulator systems

Robotic systems are finding increasing use in space applications. Many of these devices are going to be operational on board the Space Station Freedom. Fault tolerance has been deemed necessary because of the criticality of the tasks and the inaccessibility of the systems to maintenance and repair. Design for fault tolerance in manipulator systems is an area within robotics that is without precedence in the literature. In this paper, we will attempt to lay down the foundations for such a technology. Design for fault tolerance demands new and special approaches to design, often at considerable variance from established design practices. These design aspects, together with reliability evaluation and modeling tools, are presented. Mechanical architectures that employ protective redundancies at many levels and have a modular architecture are then studied in detail. Once a mechanical architecture for fault tolerance has been derived, the chronological stages of operational fault tolerance are investigated. Failure detection, isolation, and estimation methods are surveyed, and such methods for robot sensors and actuators are derived. Failure recovery methods are also presented for each of the protective layers of redundancy. Failure recovery tactics often span all of the layers of a control hierarchy. Thus, a unified framework for decision-making and control, which orchestrates both the nominal redundancy management tasks and the failure management tasks, has been derived. The well-developed field of fault-tolerant computers is studied next, and some design principles relevant to the design of fault-tolerant robot controllers are abstracted. Conclusions are drawn, and a road map for the design of fault-tolerant manipulator systems is laid out with recommendations for a 10 DOF arm with dual actuators at each joint.

Tesar, Delbert↗

Co-evolution for Problem Simplification

This paper explores a co-evolutionary approach applicable to difficult problems with limited failure/success performance feedback. Like familiar "predator-prey" frameworks this algorithm evolves two populations of individuals - the solutions (predators) and the problems (prey). The approach extends previous work by rewarding only the problems that match their difficulty to the level of solut,ion competence. In complex problem domains with limited feedback, this "tractability constraint" helps provide an adaptive fitness gradient that, effectively differentiates the candidate solutions. The algorithm generates selective pressure toward the evolution of increasingly competent solutions by rewarding solution generality and uniqueness and problem tractability and difficulty. Relative (inverse-fitness) and absolute (static objective function) approaches to evaluating problem difficulty are explored and discussed. On a simple control task, this co-evolutionary algorithm was found to have significant advantages over a genetic algorithm with either a static fitness function or a fitness function that changes on a hand-tuned schedule.

Haith, Gary L.↗

Radiation Quality Effects on Transcriptome Profiles in 3-d Cultures After Particle Irradiation

In this work, we evaluate the differential effects of low- and high-LET radiation on 3-D organotypic cultures in order to investigate radiation quality impacts on gene expression and cellular responses. Reducing uncertainties in current risk models requires new knowledge on the fundamental differences in biological responses (the so-called radiation quality effects) triggered by heavy ion particle radiation versus low-LET radiation associated with Earth-based exposures. We are utilizing novel 3-D organotypic human tissue models that provide a format for study of human cells within a realistic tissue framework, thereby bridging the gap between 2-D monolayer culture and animal models for risk extrapolation to humans. To identify biological pathway signatures unique to heavy ion particle exposure, functional gene set enrichment analysis (GSEA) was used with whole transcriptome profiling. GSEA has been used extensively as a method to garner biological information in a variety of model systems but has not been commonly used to analyze radiation effects. It is a powerful approach for assessing the functional significance of radiation quality-dependent changes from datasets where the changes are subtle but broad, and where single gene based analysis using rankings of fold-change may not reveal important biological information. We identified 45 statistically significant gene sets at 0.05 q-value cutoff, including 14 gene sets common to gamma and titanium irradiation, 19 gene sets specific to gamma irradiation, and 12 titanium-specific gene sets. Common gene sets largely align with DNA damage, cell cycle, early immune response, and inflammatory cytokine pathway activation. The top gene set enriched for the gamma- and titanium-irradiated samples involved KRAS pathway activation and genes activated in TNF-treated cells, respectively. Another difference noted for the high-LET samples was an apparent enrichment in gene sets involved in cycle cycle/mitotic control. It is plausible that the enrichment in these particular pathways results from the complex DNA damage resulting from high-LET exposure where repair processes are not completed during the same time scale as the less complex damage resulting from low-LET radiation.

Patel, Z. S.↗

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗