Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Selective classification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

499 records · Page 28

TPSAS-NF1676L-20108-DND

This presentation describes several enhancements planned for the version 4 aerosol subtyping and lidar ratio selection algorithms of the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) instrument. The CALIOP subtyping algorithm determines the most likely aerosol type from CALIOP measurements (attenuated backscatter, estimated particulate depolarization ratios de, layer altitude), and surface type. The aerosol type, so determined, is associated with a lidar ratio (LR) from a discrete set of values. In the version 3 algorithms, there are 6 pairs of 532 and 1064 nm lidar ratios. Some of these lidar ratios will be updated in the version 4 algorithms. In particular, the dust and polluted dust will be adjusted to reflect the latest measurements and model studies of these types. The algorithms are being updated to eliminate the occasional confusion between smoke and clean marine aerosols seen in version 3 by modifications to the elevated layer flag definitions that are used to determine the presence of smoke aerosols over the ocean. In the subtyping algorithms pure dust is determined by high estimated particulate depolarization ratios [de > 0.20]. Mixtures of dust and other aerosol types are determined by intermediate values of the estimated depolarization ratio [0.075< de <0.2]. The version 3 algorithms are limited to mixtures of dust and smoke, the so-called polluted dust aerosol type. To differentiate between mixtures of dust and smoke, and dust and marine aerosols, a new aerosol type will be added in the version 4 data products. In the revised classification algorithms, polluted dust will still defined as dust + smoke/pollution but in the marine boundary layer instances of moderate depolarization will be typed as dusty marine aerosols with a lower lidar ratio [LR = 35 sr] than polluted dust [currently LR = 55 sr]. The dusty marine type introduced in version 4 is modeled as a mixture of dust + marine aerosol. In the v3 algorithms the frequency of dust and polluted dust aerosols at daytime is higher than at nighttime. We present possible reasons for this and present the v4 distributions resulting from both improved background slope corrections of the daytime depolarization ratios and changes to the daytime thresholds for the polluted dust and dusty marine types of version 4. To gauge the impact of the enhancements, we contrast the following between versions 3 and 4: aerosol type, parameter distributions of each type, layer heights of maximum frequency, and distributions of smoke in biomass burning regions. To illustrate specific impacts the presentation shows case studies of version 3 and version 4 vertical feature masks of the aerosol subtypes, where appropriate, for the above enhancements.

A Omar↗

Environmental Signatures for Habitability: What to Measure and How to Rank the Habitability Potential of Mars

The environmental signatures for habitability are not necessarily biosignatures, even though on Earth, they are definitive proof of habitability. It is the constant overprint of the chemical signatures of life that makes it difficult to recognize the chemical and physical properties of a potentially habitable environment as distinct from an inhabited one. Mars Science Laboratory (MSL) will soon embark on a mission to Mars to assess its past or present habitability, so it is useful to examine how we measure habitability on Earth and prepare for how that approach may differ for Mars. This exercise includes: (a) articulation of fundamental assumptions about habitability, (b) an inventory of factors that affect habitability, (c) development of metrics, measurement approach and implementation, and (d) a new classification scheme for planetary habitability that goes beyond the binary "yes" or "no." There may be dozens of factors that affect habitability and they can be weighted as a function of specific environment. However a robotic, in situ investigation even on Earth has constraints that prevent the measurement of every environmental factor, so metrics must be reduced to the most relevant subset, given available time, cost, technical feasibility and scientific importance. Many of the factors could be measured with a combination of orbital data and the MSL payload. We propose that, at a minimum, a designation of high habitability potential requires the following conditions be met: (a) thermally stable with respect to extremes and frequency of fluctuation, (b) has more than one energy source, (c) sufficient chemical diversity to make compounds with covalent and hydrogen bonding, (d) can moderate ionizing radiation enough to allow a stable or evolving pool of organic molecules, (e) must have water or other high quality polar solvent, (f) must be able to renew chemical resources (e.g., plate tectonics, volcanism or something else we haven't envisioned). A measurement approach we have taken to measure habitability on Earth is : 1. Study remote sensing data, maps, etc. 2. Decide how big an area to measure. 3. Determine the spatial sampling rate. 4. Determine the temporal sampling rate. 5. Determine the order of measurements 6. Decide where to begin measurements 7. Select locations at field site and proceed While science drives each of the steps, there are additional constraints, e.g., technical, time, cost, safety (risk). This approach is also executable on Mars. Measurement of past habitability is more challenging both for Earth and Mars where access to the past means subsurface access and confrontation with unknowns about preservation of the martian past. Some environments preserve evidence of past habitability better than others, and this is where selection of the landing site to maximize the preservation potential of habitability indicators will be key. Mars presents an opportunity to discover transitional states between habitable or not, and we offer a ranking scale for planetary habitability with Mars as the second test subject: CLASS ONE Uninhabitable and likely has never been so CLASS TWO Has a high potential but no confirmed observation of life (as defined above) CLASS THREE Inhabited (we find life) 3-A Globally inhabited 3-B Primitive life; early in its evolution, but not yet globally established 3-C Exists only in refugia -- planet heading toward class four CLASS FOUR Post-habitable (there once was life, but now it's gone) MSL provides an opportunity to carefully investigate the habitability of at least one site on Mars and it will reveal much about the possible states of planetary habitability

Conrad, Pamela G.↗

Association of American Geographers, Remote Sensing Specialty Group Special Issue of Geocarto International

This special issue continues the precedence of the Association of American Geographers (AAG), Remote Sensing Specialty Group (RSSG) for publishing selected articles in Geocarto International as a by-product from the AAG annual meeting. As editors, we issued earlier this year, a solicitation for papers to be published in a special issue of Geocarto International that were presented in RSSG-sponsored sessions at the 2001 AAG annual meeting held in New York City on February 27-March 3. Although not an absolute requisite for publication, the vast majority of the papers in this special issue were presented at this year's AAG meeting in New York. Other articles in this issue that were not part of a paper or poster session at the 2001 AAG meeting are authored by RSSG members. Under the auspices of the RSSG, this special Geocarto International issue provides even more compelling evidence of the inextricable linkage between remote sensing and geography. The papers in this special issue fall into four general themes: 1) Urban Analysis and Techniques for Urban Analysis; 2) Land Use/Land Cover Analysis; 3) Fire Modeling Assessment; and 4) Techniques. The first four papers herein are concerned with the use of remote sensing for analysis of urban areas, and with use or development of techniques to better characterize urban areas using remote sensing data. As the lead paper in this grouping, Rashed et al., examine the usage of spectral mixture analysis (SMA) for analyzing satellite imagery of urban areas as opposed to more 'standard' methods of classification. Here SMA has been applied to IRS-1C satellite multispectral imagery to extract measures that better describe the 'anatomy' of the greater Cairo, Egypt region. Following this paper, Weng and Lo describe how Landsat TM data have been used to monitor land cover types and to estimate biomass parameters within an urban environment. The research reported in this paper applies an integrated GIS (Geographic Information System) approach for detecting urban growth and assessing its impact on biomass in the Zhujiang Delta, China. The remaining two papers in this first grouping deal with improved techniques for characterizing and analyzing urban areas using remote sensing data. Myint examines the use of texture analysis to better classify urban features. Here wavelet analysis has been employed to assist in deriving a more robust classification of the urban environment from high spatial resolution, multispectral aircraft data. Mesev provides insight on how through the modification of the standard maximum likelihood image analysis technique, population census data can be used enhance the overall robustness of urban image classification through the modification of the standard maximum likelihood image analysis technique.

Allen, Thomas R.↗

GPS Navigation for the Magnetospheric Multi-Scale Mission

In 2014. NASA is scheduled to launch the Magnetospheric Multiscale Mission (MMS), a four-satellite formation designed to monitor fluctuations in the Earth's magnetosphere. This mission has two planned phases with different orbits (1? x 12Re and 1.2 x 25Re) to allow for varying science regions of interest. To minimize ground resources and to mitigate the probability of collisions between formation members, an on-board orbit determination system consisting of a Global Positioning System (GPS) receiver and crosslink transceiver was desired. Candidate sensors would be required to acquire GPS signals both below and above the constellation while spinning at three revolutions-per-minute (RPM) and exchanging state and science information among the constellation. The Intersatellite Ranging and Alarm System (IRAS), developed by Goddard Space Flight Center (GSFC) was selected to meet this challenge. IRAS leverages the eight years of development GSFC has invested in the Navigator GPS receiver and its spacecraft communication expertise, culminating in a sensor capable of absolute and relative navigation as well as intersatellite communication. The Navigator is a state-of-the-art receiver designed to acquire and track weak GPS signals down to -147dBm. This innovation allows the receiver to track both the main lobe and the much weaker side lobe signals. The Navigator's four antenna inputs and 24 tracking channels, together with customized hardware and software, allow it to seamlessly maintain visibility while rotating. Additionally, an extended Kalman filter provides autonomous, near real-time, absolute state and time estimates. The Navigator made its maiden voyage on the Space Shuttle during the Hubble Servicing Mission, and is scheduled to fly on MMS as well as the Global Precipitation Measurement Mission (GPM). Additionally, Navigator's acquisition engine will be featured in the receiver being developed for the Orion vehicle. The crosslink transceiver is a 1/4 Watt transmitter utilizing a TDMA schedule to distribute a science quality message to all constellation members every ten seconds. Additionally the system generates one-way range measurements between formation members which is used as input to the Kalman filter. In preparation for the MMS Preliminary Design Review (PDR), the Navigator was required to pass a series of Technology Readiness Level (TRL) tests to earn the necessary TRL-6 classification. The TRL-6 level is achieved by demonstrating a prototype unit in a relevant end-to-end environment. The IRAS unit was able to meet all requirements during the testing phase, and has thus been TRL-6 qualified

Bamford, William↗

Peak Wind Tool for General Forecasting

The expected peak wind speed of the day is an important forecast element in the 45th Weather Squadron's (45 WS) daily 24-Hour and Weekly Planning Forecasts. The forecasts are used for ground and space launch operations at the Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS). The 45 WS also issues wind advisories for KSC/CCAFS when they expect wind gusts to meet or exceed 25 kt, 35 kt and 50 kt thresholds at any level from the surface to 300 ft. The 45 WS forecasters have indicated peak wind speeds are challenging to forecast, particularly in the cool season months of October - April. In Phase I of this task, the Applied Meteorology Unit (AMU) developed a tool to help the 45 WS forecast non-convective winds at KSC/CCAFS for the 24-hour period of 0800 to 0800 local time. The tool was delivered as a Microsoft Excel graphical user interface (GUI). The GUI displayed the forecast of peak wind speed, 5-minute average wind speed at the time of the peak wind, timing of the peak wind and probability the peak speed would meet or exceed 25 kt, 35 kt and 50 kt. For the current task (Phase II ), the 45 WS requested additional observations be used for the creation of the forecast equations by expanding the period of record (POR). Additional parameters were evaluated as predictors, including wind speeds between 500 ft and 3000 ft, static stability classification, Bulk Richardson Number, mixing depth, vertical wind shear, temperature inversion strength and depth and wind direction. Using a verification data set, the AMU compared the performance of the Phase I and II prediction methods. Just as in Phase I, the tool was delivered as a Microsoft Excel GUI. The 45 WS requested the tool also be available in the Meteorological Interactive Data Display System (MIDDS). The AMU first expanded the POR by two years by adding tower observations, surface observations and CCAFS (XMR) soundings for the cool season months of March 2007 to April 2009. The POR was expanded again by six years, from October 1996 to April 2002, by interpolating 1000-ft sounding data to 100-ft increments. The Phase II developmental data set included observations for the cool season months of October 1996 to February 2007. The AMU calculated 68 candidate predictors from the XMR soundings, to include 19 stability parameters, 48 wind speed parameters and one wind shear parameter. Each day in the data set was stratified by synoptic weather pattern, low-level wind direction, precipitation and Richardson Number, for a total of 60 stratification methods. Linear regression equations, using the 68 predictors and 60 stratification methods, were created for the tool's three forecast parameters: the highest peak wind speed of the day (PWSD), 5-minute average speed at the same time (A WSD), and timing of the PWSD. For PWSD and A WSD, 30 Phase II methods were selected for evaluation in the verification data set. For timing of the PWSD, 12 Phase\I methods were selected for evaluation. The verification data set contained observations for the cool season months of March 2007 to April 2009. The data set was used to compare the Phase I and II forecast methods to climatology, model forecast winds and wind advisories issued by the 45 WS. The model forecast winds were derived from the 0000 and 1200 UTC runs of the 12-km North American Mesoscale (MesoNAM) model. The forecast methods that performed the best in the verification data set were selected for the Phase II version of the tool. For PWSD and A WSD, linear regression equations based on MesoNAM forecasts performed significantly better than the Phase I and II methods. For timing of the PWSD, none of the methods performed significantly bener than climatology. The AMU then developed the Microsoft Excel and MIDDS GUls. The GUIs display the forecasts for PWSD, AWSD and the probability the PWSD will meet or exceed 25 kt, 35 kt and 50 kt. Since none of the prediction methods for timing of the PWSD performed significantly better thanlimatology, the tool no longer displays this predictand. The Excel and MIDDS GUIs display forecasts for Day-I to Day-3 and Day-I to Day-5, respectively. The Excel GUI uses MesoNAM forecasts as input, while the MIDDS GUI uses input from the MesoNAM and Global Forecast System model. Based on feedback from the 45 WS, the AMU added the daily average wind speed from 30 ft to 60 ft to the tool, which is one of the parameters in the 24-Hour and Weekly Planning Forecasts issued by the 45 WS. In addition, the AMU expanded the MIDDS GUI to include forecasts out to Day-7.

Barrett, Joe H., III↗

Recommendations on the Use of Commercial-Off-The-Shelf (COTS) Electrical, Electronic, and Electromechanical (EEE) Parts for NASA Missions - Phase II

This assessment had two Phases. Phase I captured NASA Centers’ current practices for commercial-off-the-shelf (COTS) Electrical, Electronic, and Electromechanical (EEE) parts 1 used in spaceflight systems and ground support equipment (available at https://ntrs.nasa.gov/citations/20205011579) [ref. 1]. The Phase II report provides guidance for selecting and using COTS parts in NASA missions. The approaches proposed in this report differ from current agency practices. This top-level executive summary touches on these new approaches for using COTS parts but does not provide the detailed information that is critical in understanding the rationale behind these new approaches. Readers will need to read the entire report to gain full understanding and effectively use the recommendations herein. NASA’s historical approach to selecting and applying parts has been to define certain parts, primarily specific classes of military specification (MIL-SPEC) parts, as “standard”, leaving all others, including COTS parts, as nonstandard. Standard parts typically are used without further testing (“use-as-is”). Nonstandard parts are subjected to initial screening and subsequent lot acceptance testing of representative samples from each procured lot per MIL-SPEC or similar requirements. Decades later, top-tier commercial part manufacturers have evolved significant manufacturing, statistical control, and technological improvements that can now provide parts as reliable or more reliable than MIL-SPEC parts, when used within their datasheet limits. Concurrently, the space science and exploration community’s needs demand technological advances unavailable with MIL-SPEC parts. This ongoing change necessitates using COTS parts for space missions. Properly selected COTS parts in appropriate applications can offer performance and supply availability advantages compared to MIL-SPEC parts. Their utility and demonstrated reliability result from large volumes and automated production and testing processes. However, careful review and a thorough understanding of their specifications (i.e., datasheet limitations) is needed, and verifying that manufacturer specifications and reliability meet space hardware application needs are necessary. This report recommends MIL-SPEC screening and non-radiation-related lot acceptance testing be reduced or eliminated in cases where evidence of sufficient quality and reliability exists for COTS parts. The extent of NASA's insight into COTS manufacturers and the amount and nature of the needed evidence will differ by mission and will likely be driven by a mission's resources and associated risk posture. To facilitate this goal, two new terminologies have been defined and described: “Industry Leading Parts Manufacturer (ILPM)” and “Established COTS parts.” An ILPM is a COTS manufacturer that produces high quality and reliable parts. Some parts produced by ILPMs, defined as Established COTS parts, do not need any additional MIL-SPEC or NASA screening and lot acceptance testing to be used in space applications. This report provides guidance for selecting, procuring, and applying COTS parts and for performing part-, board-, and system-level COTS parts verification. The recommendation to select Established COTS parts from ILPMs will assure those COTS parts will have comparable quality to corresponding MIL-SPEC parts. Selecting, applying, and verifying Established COTS parts from ILPMs requires a holistic team approach, engaging parts engineers, circuit designers, quality, reliability, and systems engineers, procurement specialists, radiation specialists, avionics leads, and program/project managers. A mission-specific approach tailored to a project’s Mission, Environment, Applications and Lifetime (MEAL) [ref. 2] requirements should be developed and approved by program/project managers. Any associated risks should be clearly identified, quantified, mitigated, and/or accepted. Different approaches are recommended according to program/project Risk Classes A, B, C, and D [ref. 3] and human-rated missions [ref. 4]: 1. Recommend Classes A and B and human-rated missions consider a “MIL-SPEC parts- based design” approach. ”MIL-SPEC parts-based design” approach is one in which most parts are MIL-SPEC parts and Established COTS parts from ILPMs are used only when an equivalent MIL-SPEC part does not meet functional or size, weight, and power (SWaP) or performance requirements, or is not available. 2. Recommend Classes D and Sub-D missions consider a “System of COTS” approach. “System of COTS” approach is one which most parts are Established COTS parts from ILPMs. 3. Recommend Class C missions determine which approach is the best for their projects; that is, use either a “MIL-SPEC parts-based design” approach, “System of COTS” approach, or a combined approach utilizing elements of both. This report intends to provide guidance in using COTS parts for NASA missions with risk classifications of A through D and human-rated missions; but it does not address the costs of using COTS parts. Costs of using COTS parts in different NASA mission classes can vary significantly even if the same parts are used in different risk postures, due to differing verification levels needed. The guidance does not distinguish between critical or non-critical systems, and a given project will need to apply the appropriate guidance based on their risk posture. The intended audience of this report are NASA personnel and commercial practitioners who support NASA’s spaceflight missions, including spaceflight program or project managers, parts engineers, parts manufacturers, radiation engineers, avionics engineers, system engineers, circuit design engineers, reliability engineers, safety and mission assurance (SMA) personnel, and parts procurement specialists. The NEPP Program will perform a pathfinder study to explore implementing the guidance in this NESC report. An ILPM verification process is not the same as conventional vendor qualification processes performed according to military standards and specifications. This NESC report intends to provide guidance in utilizing available parts data from ILPM manufacturers for parts assurance assessments needed for NASA missions. The report also captured the current practices from DoD and Federal Aviation Administration (FAA) in Section 10. Note each DoD and FAA report was provided by the corresponding agencies regarding their practices, which are independent from the NESC recommendations in the report.

Commercial-Off-The-Shelf↗

In-Situ and Remote-Sensing Data Fusion Using Machine Learning Techniques to Infer Urban and Fire Related Pollution Plumes

Airmass type characterization is key in understanding the relative contribution of various emission sources to atmospheric composition and air quality and can be useful in bottom-up model validation and emission inventories. However, classification of pollution plumes from space is often not trivial. Sub-orbital campaigns, such as SEAC4RS (Studies of Emissions, Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys) give us a unique opportunity to study atmospheric composition in detail, by using a vast suite of in-situ instruments for the detection of trace gases and aerosols. These measurements allow identification of spatial and temporal atmospheric composition changes due to various pollution plumes resulting from urban, biogenic and smoke emissions. Nevertheless, to transfer the knowledge gathered from such campaigns into a global spatial and temporal context, there is a need to develop workflow that can be applicable to measurements from space. In this work we rely on sub-orbital in-situ and total column remote sensing measurements of various pollution plumes taken aboard the NASA DC-8 during 2013 SEAC4RS campaign, linking them through a neural-network (NN) algorithm to allow inference of pollution plume types by input of columnar aerosol and trace-gas measurements. In particular, we use the 4STAR (Spectrometer for Sky-Scanning, Sun-Tracking Atmospheric Research) airborne measurements of wavelength dependent aerosol optical depth (AOD), particle size proxies, O3, NO2 and water vapor to classify different pollution plumes. Our method relies on assigning a-priori ground-truth labeling to the various plumes, which include urban pollution, different fire types (i.e. forest and agriculture) and fire stage (i.e. fresh and aged) using cluster analysis of aerosol and trace-gases in-situ and auxiliary (e.g. trajectory) data and the training of a NN scheme to fit the best prediction parameters using 4STAR measurements as input. We explore our misclassification rates as related to our ground-truth labels, and with multi-layered pollution plume cases. The next step in our analysis is to optimize parameter selection for a scheme that can be applied to space-borne aerosol and trace-gas observation platforms such as OMI, and future geostationary satellites such as TEMPO and GEO-CAPE.

Neural-network↗

Refractory Materials for Flame Deflector Protection System Corrosion Control: Refractory Ceramics Literature Survey

Ceramics can be defmed as a material consisting of hard brittle properties produced from inorganic and nonmetallic minerals made by firing at high temperatures. These materials are compounds between metallic and nonmetallic elements and are either totally ionic, or predominately ionic but having some covalent character. This definition allows for a large range of materials, not all applicable to refractory applications. As this report is focused on potential ceramic materials for high temperature, aggressive exposure applications, the ceramics reviewed as part of this report will focus on refractory ceramics specifically designed and used for these applications. Ceramic materials consist of a wide variety of products. Callister (2000) 1 characterized ceramic materials into six classifications: glasses, clay products, refractories, cements, abrasives, and advanced ceramics. Figure 1 shows this classification system. This review will focus mainly on refractory ceramics and cements as in general, the other classifications are neither applicable nor economical for use in large structures such as the flame trench. Although much work has been done in advanced ceramics over the past decade or so, these materials are likely cost prohibitive and would have to be fabricated off-site, transported to the NASA facilities, and installed, which make these even less feasible. Although the authors reviewed the literature on advanced ceramic refractories 2 center dot 3 center dot 4 center dot 5 center dot 6 center dot 7 center dot 8 center dot 9 center dot 10 center dot 11 center dot 12 after the review it was concluded that these materials should not be ' the focus of this report. A review is in progress on materials and systems for prefabricated refractory ceramic panels, but this review is focusing more on typical refractory materials for prefabricated systems, which could make the system more economically feasible. Refractory ceramics are used for a wide variety of applications. Figure 2 shows many ofthese applications, their life expectancy or requirement, and the exposure temperature for the refractory ceramic. Note that the exposure temperatures for refractory ceramics are very similar to the exposure conditions for specialty ceramics (rocket nozzles, space vehicle re-entry fields, etc.) and yet the life expectancy or requirement is relatively low. Currently NASA is repairing the refractory lining in the flame trench after every launch - although this is not a direct indication of low life expectancy, it does indicate that the current system may not be sufficiently durable to maximize economy. Better performing refractory ceramics are needed to improve the performance, economy, and safety during and after launches at the flame trenches at Kennedy Space Center (KSC). To achieve this goal a current study is underway to assess different refractory systems for possible use in the flame trenches at KSC. This report will target the potential applicability of refractory ceramics for use in the flame trenches. An overview of the different refractory ceramics will be provided (see Figure I). This will be followed with a brief description of the structure of refractory products, the properties and characteristics of different systems, the methodology for selecting refractories, and then a general design methodology. Based on these sections, future challenges and opportunities will be identified with the objective of improving the durability, performance, economy, and safety of the launch complex. Refractory ceramics are used for a wide variety of applications. Figure 2 shows many ofthese applications, their life expectancy or requirement, and the exposure temperature for the refractory ceramic. Note that the exposure temperatures for refractory ceramics are very similar to the exposure conditions for specialty ceramics (rocket nozzles, space vehicle re-entry fields, etc.) and yet the life expectancy or requirement is relatively low. Currently NASA is repairing the refractory lining in the flame trench after every launch - although this is not a direct indication of low life expectancy, it does indicate that the current system may not be sufficiently durable to maximize economy. Better performing refractory ceramics are needed to improve the performance, economy, and safety during and after launches at the flame trenches at Kennedy Space Center (KSC). To achieve this goal a current study is underway to assess different refractory systems for possible use in the flame trenches at KSC. This report will target the potential applicability of refractory ceramics for use in the flame trenches. An overview of the different refractory ceramics will be provided (see Figure I). This will be followed with a brief description of the structure of refractory products, the properties and characteristics of different systems, the methodology for selecting refractories, and then a general design methodology. Based on these sections, future challenges and opportunities will be identified with the objective of improving the durability, performance, economy, and safety of the launch complex.

Calle, Luz Marina↗

Potential of VIIRS Time Series Data for Aiding the USDA Forest Service Early Warning System for Forest Health Threats: A Gypsy Moth Defoliation Case Study

This report details one of three experiments performed during FY 2007 for the NASA RPC (Rapid Prototyping Capability) at Stennis Space Center. This RPC experiment assesses the potential of VIIRS (Visible/Infrared Imager/Radiometer Suite) and MODIS (Moderate Resolution Imaging Spectroradiometer) data for detecting and monitoring forest defoliation from the non-native Eurasian gypsy moth (Lymantria dispar). The intent of the RPC experiment was to assess the degree to which VIIRS data can provide forest disturbance monitoring information as an input to a forest threat EWS (Early Warning System) as compared to the level of information that can be obtained from MODIS data. The USDA Forest Service (USFS) plans to use MODIS products for generating broad-scaled, regional monitoring products as input to an EWS for forest health threat assessment. NASA SSC is helping the USFS to evaluate and integrate currently available satellite remote sensing technologies and data products for the EWS, including the use of MODIS products for regional monitoring of forest disturbance. Gypsy moth defoliation of the mid-Appalachian highland region was selected as a case study. Gypsy moth is one of eight major forest insect threats listed in the Healthy Forest Restoration Act (HFRA) of 2003; the gypsy moth threatens eastern U.S. hardwood forests, which are also a concern highlighted in the HFRA of 2003. This region was selected for the project because extensive gypsy moth defoliation occurred there over multiple years during the MODIS operational period. This RPC experiment is relevant to several nationally important mapping applications, including agricultural efficiency, coastal management, ecological forecasting, disaster management, and carbon management. In this experiment, MODIS data and VIIRS data simulated from MODIS were assessed for their ability to contribute broad, regional geospatial information on gypsy moth defoliation. Landsat and ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) data were used to assess the quality of gypsy moth defoliation mapping products derived from MODIS data and from simulated VIIRS data. The project focused on use of data from MODIS Terra as opposed to MODIS Aqua mainly because only MODIS Terra data was collected during 2000 and 2001-years with comparatively high amounts of gypsy moth defoliation within the study area. The project assessed the quality of VIIRS data simulation products. Hyperion data was employed to assess the quality of MODIS-based VIIRS simulation datasets using image correlation analysis techniques. The ART (Application Research Toolbox) software was used for data simulation. Correlation analysis between MODIS-simulated VIIRS data and Hyperion-simulated VIIRS data for red, NIR (near-infrared), and NDVI (Normalized Difference Vegetation Index) image data products collectively indicate that useful, effective VIIRS simulations can be produced using Hyperion and MODIS data sources. The r(exp 2) for red, NIR, and NDVI products were 0.56, 0.63, and 0.62, respectively, indicating a moderately high correlation between the 2 data sources. Temporal decorrelation from different data acquisition times and image misregistration may have lowered correlation results. The RPC experiment also generated MODIS-based time series data products using the TSPT (Time Series Product Tool) software. Time series of simulated VIIRS NDVI products were produced at approximately 400-meter resolution GSD (Ground Sampling Distance) at nadir for comparison to MODIS NDVI products at either 250- or 500-meter GSD. The project also computed MODIS (MOD02) NDMI (Normalized Difference Moisture Index) products at 500-meter GSD for comparison to NDVI-based products. For each year during 2000-2006, MODIS and VIIRS (simulated from MOD02) time series were computed during the peak gypsy moth defoliation time frame in the study area (approximately June 10 through July 27). Gypsy moth defoliation mapping products from simated VIIRS and MOD02 time series were produced using multiple methods, including image classification and change detection via image differencing. The latter enabled an automated defoliation detection product computed using percent change in maximum NDVI for a peak defoliation period during 2001 compared to maximum NDVI across the entire 2000-2006 time frame. Final gypsy moth defoliation mapping products were assessed for accuracy using randomly sampled locations found on available geospatial reference data (Landsat and ASTER data in conjunction with defoliation map data from the USFS). Extensive gypsy moth defoliation patches were evident on screen displays of multitemporal color composites derived from MODIS data and from simulated VIIRS vegetation index data. Such defoliation was particularly evident for 2001, although widespread denuded forests were also seen for 2000 and 2003. These visualizations were validated using aforementioned reference data. Defoliation patches were visible on displays of MODIS-based NDVI and NDMI data. The viewing of apparent defoliation patches on all of these products necessitated adoption of a specialized temporal data processing method (e.g., maximum NDVI during the peak defoliation time frame). The frequency of cloud cover necessitated this approach. Multitemporal simulated VIIRS and MODIS Terra data both produced effective general classifications of defoliated forest versus other land cover. For 2001, the MOD02-simulated VIIRS 400-meter NDVI classification produced a similar yet slightly lower overall accuracy (87.28 percent with 0.72 Kappa) than the MOD02 250-meter NDVI classification (88.44 percent with 0.75 Kappa). The MOD13 250-meter NDVI classification had a lower overall accuracy (79.13 percent) and a much lower Kappa (0.46). The report discusses accuracy assessment results in much more detail, comparing overall classification and individual class accuracy statistics for simulated VIIRS 400-meter NDVI, MOD02 250-meter NDVI, MOD02-500 meter NDVI, MOD13 250-meter NDVI, and MOD02 500-meter NDMI classifications. Automated defoliation detection products from simulated VIIRS and MOD02 data for 2001 also yielded similar, relatively high overall classification accuracy (85.55 percent for the VIIRS 400-meter NDVI versus 87.28 percent for the MOD02 250-meter NDVI). In contrast, the USFS aerial sketch map of gypsy moth defoliation showed a lower overall classification accuracy at 73.64 percent. The overall classification Kappa values were also similar for the VIIRS (approximately 0.67 Kappa) versus the MOD02 (approximately 0.72 Kappa) automated defoliation detection product, which were much higher than the values exhibited by the USFS sketch map product (overall Kappa of approximately 0.47). The report provides additional details on the accuracy of automated gypsy moth defoliation detection products compared with USFS sketch maps. The results suggest that VIIRS data can be effectively simulated from MODIS data and that VIIRS data will produce gypsy moth defoliation mapping products that are similar to MODIS-based products. The results of the RPC experiment indicate that VIIRS and MODIS data products have good potential for integration into the forest threat EWS. The accuracy assessment was performed only for 2001 because of time constraints and a relative scarcity of cloud-free Landsat and ASTER data for the peak defoliation period of the other years in the 2000-2006 time series. Additional work should be performed to assess the accuracy of gypsy moth defoliation detection products for additional years.The study area (mid-Appalachian highlands) and application (gypsy moth forest defoliation) are not necessarily representative of all forested regions and of all forest threat disturbance agents. Additional work should be performed on other inland and coastal regions as well as for other major forest threats.

Spruce, Joseph P.↗

A Machine Learning Approach to Improve Air Traffic Management Initiatives

Collaborating closely with commercial air carriers and related organizations, the Federal Aviation Administration(FAA) regulates air traffic and ensures the safety and efficiency of air operations. Air traffic controllers make strategic decisions, such as delaying, rerouting, or canceling flights, partly based on guidance provided by the FAA’s Air TrafficControl System Command Center (ATCSCC). The guidance includes, among other things, control measures known asTraffic Management Initiatives (TMIs) designed to enhance safety and improve operational efficiency. TMIs play a crucial role in managing the demand and capacity within the U.S. National Airspace System (NAS). Two major TMIs that are routinely used (primarily to mitigate the adverse effects of bad weather) are Ground Delay Programs (GDPs) andGround Stops (GSs). In a GDP, flights destined for airports facing thunderstorm activity experience delays at their origin airports. This proactive approach minimizes the risk of routing aircraft through hazardous weather conditions and also replaces (fuel burning) airborne delays with ground delays. In a GS, a temporary restriction is imposed on the departure or arrival of aircraft at a specific airport or within a designated airspace. Although other TMIs (e.g., miles-in-trail) are also implemented as part of (air) traffic flow management in the NAS, the focus of this work is on GDPs and GSs. Since TMIs, by design, lead to flight delays or cancellations, it is crucial to put in place the right set of parameters(e.g., scope and duration of the GDP). For example, when the end time of a GDP extends beyond what is necessary, it imposes unnecessary delays on departing flights. This situation could occur as a result of inaccurate prediction of the(required) duration of the GDP based on the weather forecast. On the other hand, if a GDP ends prematurely before the underlying capacity constraints are resolved at the destination airport, it may result in airborne holding. The delicate balance lies in matching the termination of the GDP precisely with the resolution of capacity constraints, avoiding both the imposition of unnecessary ground delays and the need for airborne holding due to premature program termination.Failing to specify the right parameters for TMIs also leads to flight delays, creating a significant obstacle in managing the increasing traffic volumes causing increased work load for the controllers. To address this issue, we propose the integration of Machine Learning (ML) models in the traffic flow management(TFM) pipeline. In current operations, decisions are made by human experts based on extensive training, historical patterns, available traffic and weather data. Since we have an abundance of data from past events that tell us the likely impact of various TMIs, by ingesting historical data, properly trained ML models can offer valuable insights and aid human decision-making. With the FAA increasingly exploring advanced analytics, ML emerges as a focal point for enhancing TFM within the National Airspace System (NAS). As a first step, this study aims to provide traffic controllers with decision-making support for the issuance and adjustment of TMIs. Data analytics and machine learning have been previously employed to address some of the challenges associated with TMIs. Numerous studies have concentrated on various facets of TMI issuance, exploring factors influencing TMI parameters, including arrival rate, airport capacity, and delay prediction. For example, using weather forecasts, several statistical methods were used to produce probabilistic capacity profiles which in conjunction with deterministic models provided insights into the GDP planning process [1–4]. The downside of using deterministic models is that they rely on fixed inputs and predetermined rules, which lack the ability to account for the inherent uncertainty and variability present in real-world scenarios. In a separate series of studies, researchers aimed to predict the occurrences of GDPs and GSs. The majority of these studies utilized various supervised learning methods, including Decision Trees, Naive Bayes, Support VectorMachines, and Random Forests to analyze the influence of weather conditions and arrival demand on TMI incidents[5–8]. However, these studies primarily focused on predicting the incidence of TMIs without explicitly addressing the scope of TMIs, including their duration and their geographical coverage. Furthermore, the emphasis of these studies was largely on GDPs, given their higher frequency and longer duration when compared to GSs. A limited number of studies focused on predicting the parameters of TMIs, specifically addressing their duration and extent. In one such study focusing on optimizing the TMI parameters at San Francisco International Airport (SFO),the authors utilized a probabilistic forecast of fog [9]. They simulated various capacity scenarios based on the (fog)burn-off forecasts, selecting GDP parameters that minimized airborne and overall ground delays. However, this approach exclusively emphasizes stratus (fog) burn-off as the primary determinant of GDP and GS, neglecting other influential factors like severe weather events, runway closures, lower capacity than traffic demand, and other important variables. Given the complexity of predicting the TMI and determining its scope, we seek a more holistic approach. We aim to consider all significant factors that could impact TMIs and their parameters. What sets this research apart is the fusion of all data sources relevant to the issuance and adjustment of TMIs and it represents the first comprehensive attempt to optimize TMIs in this manner. Since this comprehensive solution involves various aspects, we break down the problem into smaller components and input all parameters into a unified model called the “TMI Adjuster”. Figure 1 shows the overall framework and the list of datasets used in each model. The objective of the TMI Adjuster module is to deliver reliable, consistent and expedited recommendations for the progression, adjustment, and termination of TMIs. The ML solution entails developing a pipeline capable of predicting the necessity of a TMI (e.g., GS or GDP) along with its various parameters. For example, in the case of a GS, this includes the scope of the GS either in terms of distance from the destination airport or based on pre-defined airspace sectors. Here, scope refers to those regions and departing airports that are subject to the GS. In this paper, we concentrate on the issuance of GSs in the three major airports in the New York area — LaGuardia(LGA), John F. Kennedy International (JFK), and Newark Liberty International (EWR). We fuse traffic, weather and other relevant aviation data from years 2017 to 2019 to train and validate the ML models. In particular, we use the following datasets: •Terminal Aerodrome Forecast (TAF): meteorological forecasts specific to each airport, issued four times a day, covering predefined time periods. •TMI data: includes all GSs and GDPs along with their respective parameters. •Aviation System Performance Metrics (ASPM): includes traffic related data such as aircraft delays, arrival, and departure rates. •Notices to Airmen (NOTAMs): utilized to extract runway closure data and manage interdependencies between terminals in close proximity. •Flight cancellation data •Airspace Flow Programs (AFP): includes information on flight airborne holdings caused by TMIs. The data preprocessing entails transforming ASPM, TMI, AFP, NOTAMs, and weather data into an hourly format and consolidating all datasets by merging them based on date and time as the primary key. The TMI Adjuster framework comprises two parallel models: one dedicated to GS and a second model focused on GDP. As previously mentioned, our specific focus is on the GS model as a multi-classification problem. In this framework, each data point of the GS model input summarizes ten hours of data. Specifically, the data loader for the GS model generates the input and output of the model as follows: at a given time step, the input includes the actual traffic, weather, and TMI data from the two-hour window before the time step, alongside the weather forecast and scheduled traffic for the next 8 hours starting from the time step. Based on this information, the output of the GS model for each time interval consists of three dimensions. The first dimension represents a binary decision on whether there should be a GS in place for the next hour or not. The second dimension is related to the scope of the GS in the United States, and the third dimension is related to the scope of the GS in Canada (i.e., to determine if the GS impacts airports in Canada).One of the challenges with TMI modeling is the sparsity of TMI events, particularly regarding its scope. To address this challenge in the scope of the GS model output, we implement grouping. The GS scope for the US region is defined based on a list of centers that should be included when the GS is in place. With 20 centers in the US, we utilized historical data to group them into 4 categories. In particular, we summarized our historical data in a graph format where nodes represent centers, and link weights are defined based on the co-occurrence of centers in the scope parameter ofTMIs. By identified strongly connected components in this graph, we were able to partition the centers into four groups. We consider two model structures for the GS Model. Firstly, a hierarchical classification model [10], where the human decision-making for a GS is of hierarchical nature. The decision-maker first decides whether there is a need fora GS, and if the answer is yes, determines the scope. A hierarchical classification model organizes the problem into a class hierarchy, typically a tree or a Directed Acyclic Graph (DAG) structure, and considers the dependency of the decision in the previous step to the next component [10]. Here, we employ the local classifier per level approach, which involves training one multi-class classifier for each level of the class hierarchy. The second structure is the independent structure. In this setting, as the name suggests, we do not consider the dependency of the decisions in the different dimensions of the output of the model. Instead, for each dimension, we train a multi-class classifier independently. Table 1 summarizes GS model statistics for training, validation and testing. The table documents the effect of limiting data to the time steps when there was actually a TMI in place or when a TMI had just terminated. This resulted in a more balanced distribution of the GS class(GS positive class)versus “No GS”(GS negative class), which might help the training process. While JFK and LGA follow very similar distributions, with 40% and 42% GS positive class respectively, EWR has proportionally fewer GS incidents at 28%. Our subsequent phase involves evaluating the performance of both hierarchical structure and independent structure using different state-of-the-art multi-class classifier models such as Random Forest, Decision Trees, K-nearest Neighbors, and Logistic Regression and forecast the duration and scope of the GSs.

Farzan Masrour Shalmani↗

Characterizing Detailed Grain Shape and Size Distribution Properties of Lunar Regolith

Introduction: As the nation prepares to return to the Moon, there is an increasing need for testing tools, instruments, and equipment in simulated environments on Earth to ensure successful operations during lunar missions. Regolith will affect all aspects of future lunar missions, from plume interactions during landing to space suit and tool design [1]. Because of this, it is important to understand the grain shape and size properties of lunar regolith and how those influence regolith behavior in order to prepare for these missions. This knowledge is also vital to create more accurate lunar regolith simulants for testing equipment in a lunar environment. While particle size analyses have been performed on most Apollo soils using simple sieving, shape has only been crudely addressed [2]. This work analyzes 4 lunar regolith samples to provide a better understanding of these size and shape parameters and will provide a more accurate baseline of data to create high-fidelity lunar regolith simulants. Methods: New technologies exist today that are capable of measuring size and shape simultaneously for hundreds of thousands of particles in a single measurement. We conducted a rigorous analysis of the particle size distribution (PSD) as well as the size-dependent 2D and 3D shape parameters of lunar regolith samples of different compositions and maturity levels. This analysis was done using a Microtrac SYNC which provides a unique combination of tri-laser diffraction and Dynamic Image Analysis (DIA). Sample Selection. Four regolith samples were selected for analysis based on maturity level and lunar terrain type: • 10084 – Mature high-Ti mare regolith • 15601 – Immature low-Ti mare regolith • 64501 – Mature highland regolith • 67461 – Immature highland regolith Sample Analysis. After receiving the samples, each sample was imaged with an optical microscope (Figure 1). To obtain 2D and 3D particle size and shape, 0.1 g of each sample was then added to the SYNC for analysis, which outputs more than 30 size and shape parameters for each individual grain as well as the complete size distribution from 0.01-2000 μm by blending laser diffraction and DIA together. For the 0.1 g sample masses requested, ~100,000 grains per sample were captured by DIA. Results: From the PSD analysis, it can be seen that the average particle size of 10084 is ~24.5 µm which is much smaller than the average particle size of the Apollo sample collection (~72 µm) [3], while the average particle size of samples 15601, 64501, and 67461 are larger than the Apollo sample average at ~106 µm, ~103 µm, and ~118.5 µm respectively. The size and shape measurements for the samples output ~30 parameters, four of which were focused on for this study: sphericity, aspect ratio, roundness, and concavity (Figure 3). However, after analyzing the plots it was found that only sphericity and aspect ratio showed differences between the samples. These two parameters are measured on a scale from 0 to 1, with 1 being a perfect sphere with equal dimensions. The results show that the sphericity values are slightly high-er for the mature samples 10084 and 64501 (~0.96) than they are for samples 15601 and 67461 (~0.95) (Figure 4). The aspect ratio values are slightly lower for samples 64501 (~0.7) and 67461 (~0.74) than they are for samples 10084 (~0.76) and 15601 (~0.8) (Figure 5). Discussion: Mature regoliths are those that have been exposed to micrometeorite and solar wind bombardment for long periods of time, breaking up particles and causing them to become more rounded [4]. Therefore, the smaller particle sizes and higher sphericity values for samples 10084 and 64501 are expected due to their higher maturity level compared to 15601 and 67461. However, the aspect ratio values are not dependent on maturity and are instead dependent on terrain type. The lower aspect ratio values for the high-land samples is potentially due to the higher plagioclase content, which occurs in elongated particles and does not break down as easily as the pyroxenes or olivines that are present in the mare. Conclusion and Future Work: The results of this work provide a baseline of high-quality data that will contribute to the creation of high-fidelity lunar simulants and will greatly benefit NASA’s efforts of establishing a human presence on the Moon. Future work includes performing unsupervised image classification on the ~105 particle images per sample in order to identify different classes of grains. These grain classes can then be linked to detailed shape properties, and the relative abundance of each class in the samples can be compared. Acknowledgments: We would like to thank the Extraterrestrial Materials Analysis Group (ExMAG) and the Astromaterials Allocation Review Board (AARB) for allocating lunar regolith samples 10084,9010, 15601,365, 64501,249, and 67461,171 for this work. References: [1] Taylor, L. A. et al. (2005) AIAA #2510. [2] Katagiri et al. (2015) ASCE. [3] Carrier III, W. D. (2005) Lunar Geotech Institute Tech Report. [4] McKay, D. S. et al. (1991) The Lunar Sourcebook, Chapter 7.

S R Deitrick↗

Methodology and Results of the Near-Earth Object (NEO) Human Space Flight (HSF) Accessible Targets Study (NHATS)

Near-Earth Asteroids (NEAs) have been identified by the current administration as potential destinations for human explorers during the mid-2020s. While the close proximity of these objects' orbits to Earth's orbit creates a risk of highly damaging or catastrophic impacts, it also makes some of these objects particularly accessible to spacecraft departing Earth, and this presents unique opportunities for solar system science and humanity's first ventures beyond cislunar space. Planning such ambitious missions first requires the selection of potentially accessible targets from the growing population of nearly 7,800 NEAs. To accomplish this, NASA is conducting the Near-Earth Object (NEO) Human Space Flight (HSF) Accessible Targets Study (NHATS). Phase I of the NHATS was executed during September of 2010, and Phase II was completed by early March of 2011. The study is ongoing because previously undetected NEAs are being discovered constantly, which has motivated an effort to automate the analysis algorithms in order to provide continuous monitoring of NEA accessibility. The NHATS analysis process consists of a trajectory filter and a minimum maximum estimated size criterion. The trajectory filter employs the method of embedded trajectory grids to compute all possible ballistic round-trip mission trajectories to every NEA in the Jet Propulsion Laboratory (JPL) Small-Body Database (SBDB) and stores all solutions that satisfy the trajectory filter criteria. An NEA must offer at least one qualifying trajectory solution to pass the trajectory filter. The Phase II NHATS filter criteria were purposely chosen to be highly inclusive, requiring Earth departure date between January 1st, 2015 and December 31st, 2040, total round-trip flight time <= 450 days, stay time at the NEA >= 8 days, Earth departure C(sub 3) energy <= 60 km(exp 2)/s(exp 2), total mission delta-v <= 12 km/s (including an Earth departure maneuver from a 400 km altitude circular parking orbit), and a maximum atmospheric re-entry speed of 12 km/s. After determining which NEAs offer at least one trajectory solution meeting the criteria, the estimated size constraint is then imposed whereby those NEAs may only be considered NHATS- qualifying NEAs if their maximum estimated size is >= 30 m. This corresponds to an absolute magnitude H <= 26.5 with an assumed albedo p = 0:05. The following is a brief high-level summary of the Phase II study results. Of the 7,665 NEAs in the SBDB as of February 3rd, 2011, 765 NEAs passed the trajectory filter and yielded a total of 79,157,604 trajectory solutions. The trajectory solutions for each NEA are post-processed into Pork Chop Contour (PCC) plots which show total mission delta-v as a function of Earth departure date and total mission duration. Although the PCC plots necessarily compress a very multi-dimensional design space into a two-dimensional plot, they permit rapid assessment of the breadth and quality of an NEA's available Earth departure season and clearly indicate the regions of the trajectory design space which warrant further analysis and optimization. The PCC plot for the NEA with the greatest number of NHATS-qualifying trajectory solutions, 2000 SG-344, is shown. Of the 765 NEAs which passed the Phase II trajectory filter, a total of 590 NEAs also satisfied the further constraint of maximum estimated size >= 30 m. The distributions of osculating heliocentric orbital semi-major axis (a), eccentricity (e), and inclination (i), for those 590 NEAs are shown. Note that the semi-latus rectum used is equal to alpha (1-e(exp 2)). To further our understanding of round-trip trajectory accessibility dynamics, it is instructive to examine the distribution of the NHATS-Qualifying NEAs according to orbit classification. NEAs are grouped into four orbit families: Atiras (aphelion < 0.983 AU), Atens (aphelion > 0.983 AU, alpha < 1.0 AU), Apollos (perihelion < 1.017 AU, alpha > 1.0 AU), and Amors (1.017 < perihelion < 1.3 AU). Of the 765 NEAhich satisfied the NHATS trajectory criteria, none are Atiras, 193 are Atens (31% of known Atens), 456 are Apollos (11% of known Apollos), and 116 are Amors (4% of known Amors). While Apollos comprise 60% of the NEAs which pass the NHATS trajectory filter and Atens comprise only 25%, the percentages according to orbit family are perhaps more relevant. Note that only 11% of known Apollos passed the trajectory filter while 31% of known Atens passed. These simple statistics alone strongly suggest that Aten orbits possess features which tend to enhance their round-trip trajectory accessibility as compared to Apollos or Amors. This is significant because Atens' orbits cause them to spend considerable time in Earth's daytime sky, making them difficult to discover and track using ground-based observing assets. In this paper we will detail the NHATS analysis algorithms, present and analyze all NHATS results to date, and discuss aspects of HSF mission architecture design for future NEA missions.

Barbee, Brent↗

Classifying Multi-Model Wheat Yield Impact Response Surfaces Showing Sensitivity to Temperature and Precipitation Change

Crop growth simulation models can differ greatly in their treatment of key processes and hence in their response to environmental conditions. Here, we used an ensemble of 26 process-based wheat models applied at sites across a European transect to compare their sensitivity to changes in temperature (minus 2 to plus 9 degrees Centigrade) and precipitation (minus 50 to plus 50 percent). Model results were analysed by plotting them as impact response surfaces (IRSs), classifying the IRS patterns of individual model simulations, describing these classes and analysing factors that may explain the major differences in model responses. The model ensemble was used to simulate yields of winter and spring wheat at four sites in Finland, Germany and Spain. Results were plotted as IRSs that show changes in yields relative to the baseline with respect to temperature and precipitation. IRSs of 30-year means and selected extreme years were classified using two approaches describing their pattern. The expert diagnostic approach (EDA) combines two aspects of IRS patterns: location of the maximum yield (nine classes) and strength of the yield response with respect to climate (four classes), resulting in a total of 36 combined classes defined using criteria pre-specified by experts. The statistical diagnostic approach (SDA) groups IRSs by comparing their pattern and magnitude, without attempting to interpret these features. It applies a hierarchical clustering method, grouping response patterns using a distance metric that combines the spatial correlation and Euclidian distance between IRS pairs. The two approaches were used to investigate whether different patterns of yield response could be related to different properties of the crop models, specifically their genealogy, calibration and process description. Although no single model property across a large model ensemble was found to explain the integrated yield response to temperature and precipitation perturbations, the application of the EDA and SDA approaches revealed their capability to distinguish: (i) stronger yield responses to precipitation for winter wheat than spring wheat; (ii) differing strengths of response to climate changes for years with anomalous weather conditions compared to period-average conditions; (iii) the influence of site conditions on yield patterns; (iv) similarities in IRS patterns among models with related genealogy; (v) similarities in IRS patterns for models with simpler process descriptions of root growth and water uptake compared to those with more complex descriptions; and (vi) a closer correspondence of IRS patterns in models using partitioning schemes to represent yield formation than in those using a harvest index. Such results can inform future crop modelling studies that seek to exploit the diversity of multi-model ensembles, by distinguishing ensemble members that span a wide range of responses as well as those that display implausible behaviour or strong mutual similarities.

Classification; Climate change; Crop model; Ensemb↗