Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61

A Geometric Approach to Modeling Microstructurally Small Fatigue Crack Formation: Simulation and Prediction of Crack Nucleation in AA 7075-T651 - 2

The objective of this paper is to develop further a framework for computationally modeling microstructurally small fatigue crack growth in AA 7075-T651 [1]. The focus is on the nucleation event, when a crack extends from within a second-phase particle into a surrounding grain, since this has been observed to be an initiating mechanism for fatigue crack growth in this alloy. It is hypothesized that nucleation can be predicted by computing a non-local nucleation metric near the crack front. The hypothesis is tested by employing a combination of experimentation and nite element modeling in which various slip-based and energy-based nucleation metrics are tested for validity, where each metric is derived from a continuum crystal plasticity formulation. To investigate each metric, a non-local procedure is developed for the calculation of nucleation metrics in the neighborhood of a crack front. Initially, an idealized baseline model consisting of a single grain containing a semi-ellipsoidal surface particle is studied to investigate the dependence of each nucleation metric on lattice orientation, number of load cycles, and non-local regularization method. This is followed by a comparison of experimental observations and computational results for microstructural models constructed by replicating the observed microstructural geometry near second-phase particles in fatigue specimens. It is found that orientation strongly influences the direction of slip localization and, as a result, in uences the nucleation mechanism. Also, the baseline models, replication models, and past experimental observation consistently suggest that a set of particular grain orientations is most likely to nucleate fatigue cracks. It is found that a continuum crystal plasticity model and a non-local nucleation metric can be used to predict the nucleation event in AA 7075-T651. However, nucleation metric threshold values that correspond to various nucleation governing mechanisms must be calibrated.

Hochhalter, Jake D.↗

Monitoring Airspace Complexity and Determining Contributing Factors

The national airspace has evolved over many years to accommodate increased traffic demand [1] while simultaneously maintaining one of the safest forms of transportation [2], [3]. One of the reasons for this success is the ability of the system and the operators to adapt and accommodate to situations that routinely disrupt optimal operations. These situations may include: adverse weather, delays, early arrivals, equipment outages, and other factors that are outside the operators ability to control. These factors can lead to states where automation is unable to properly handle these issues and therefore air traffic controllers and pilots have to intervene, ultimately increasing communication between operators resulting in higher workload. As controller workload increases to handle sub-optimal operating conditions this can be viewed as an increase in complexity. The reasoning for this is because humans are now required to make tactical decisions in response to external factors, resulting in a departure from the strategic plan where operations would be more efficiently managed. Human operators control airspace complexity under rigid regulations that are constantly changing. The airspace is divided into sectors and the number of aircraft assigned to each controller is limited for safe handling. There has been past work that devised airspace complexity metrics in commercial aviation and related these metrics to controller workload (e.g., [4],[5]). The upper bounds on the system load are pre-determined. Such bounds on complexity make for a safe system, but the system cannot scale and adapt to autonomous, dense, and heterogeneous traffic, including the many types of Unmanned Aerial Vehicles (UAVs) envisioned to be added to the operations. We hypothesize that, as traffic density and heterogeneity grow, and other key metrics change, there will be phase transitions at which the way traffic should be managed changes significantly [6]. We offer a method for in-time detection of contributing factors that lead to phase transitions, characterized by increased complexity. To the best of our knowledge, there is no tool similar to our proposed effort that identifies such contributing factors or precursor patterns. To define the scope we are proposing to measure complexity from the viewpoint of the Terminal Radar Approach Control Facilities (TRACON) controller’s perspective. In particular we are analyzing arrivals into KSFO. With safety as the top concern for airspace operators, it is important to recognize that as density and heterogeneity grow, the focus of the system will change. Times of the day when the airspace has low density and heterogeneity, the flights will follow more efficient paths where the aircraft move on established routes that are more or less directly to the destination. However, when density and heterogeneity increases, the system will begin changing focus to avoiding conflicts and collisions and route the flights in a more flexible way. Higher flexibility requires more communication and coordination between controllers and pilots which the current automation is unable to handle. This paper proposes a novel approach that monitors airspace complexity at multiple scales, uses a Machine Learning-based tool that predicts when operations will transition to a regime of greater complexity, and identifies actions that can reduce the complexity while still maintaining efficient and safe operations. We demonstrate our proposed approach using data from multiple complementary sources. This includes, but is not limited to: historical aircraft surveillance data from NASA’s Sherlock Data Warehouse [7], METAR weather data, and airport configuration data from Aviation System Performance Metrics (ASPM). The surveillance data flight paths are sampled at a variable sample rate — increasing as the aircraft approaches the airport. This is due to how Sherlock manages flight track stitching between different radar facilities which have different sampling rates. The weather and performance data are logged at defined intervals throughout the day at a courser refresh rate. In addition to the logged data and metrics, we leverage pre-defined Standard Terminal Arrival Routes (STARs) procedures to characterize the path of each flight. Each flight files for one of these routes in the flight plan well before entering the terminal airspace, and approximately follows the route until it leaves the STAR, typically on the final fix of a runway transition. However, most flights do not always fly the full STAR procedure to completion [8], but the majority do adhere to the fixes within the common route of the procedure. Our approach leverages fixes in the common route of each of the STARs to build a reference path to the airport. This allows us to characterize the flight paths in what we are defining as the “maneuvering area” (the airspace between the STAR and before the flight is lined up on the runway’s final approach) to determine how off nominal the flights are to calculate its complexity score. Determining airspace complexity is a concept that does not have a concrete answer. In designing this metric, we consider what increases the workload for the air traffic controllers. Consequently more specialized vectoring maneuvers results in higher workload. Accordingly, we start with a theory: each flight has a direct path it takes from the STAR’s common route to the final approach’s outer marker fix for the flight’s landing runway. It is important to note that the direct path is only used as a reference. If the majority of the flights have a large consistent offset as compared to other routes it does not necessarily mean that those flights have higher complexity. We are merely building a distribution based on this direct path for that particular STAR and runway pair to determine the normal mode of operations for that route. Flights that are in the upper tail of these distributions will result in higher complexity scores and flights that fly in the median will represent the normal mode of operations and therefore will have lower complexity scores. Since flights following each STAR route take different paths to the airport, we have a different distribution for each STAR route and therefore can model these distributions to compute a complexity score from their respective normalized distributions. To evaluate the effectiveness of our proposed airspace complexity metric we will compare against an established approach based on trajectory clustering [9]. This unsupervised learning technique consists of the following steps: (1) identify the general maneuvering areas (waypoints) by performing $\kappa$-means or DBSCAN clustering on locations where aircraft frequently turn based on the surveillance radar track data, (2) map flight trajectories onto sequences of waypoints, and (3) cluster the sequences based on their common subsequences. From a high-level perspective, this baseline model learns nominal operations in the airspace through the sequence of waypoints that are representative of where aircraft change direction and defines deviations from the nominal operations as “complex.” Therefore, more deviations from the nominal operations correspond to higher complexity values. For our validation, we re-implemented this technique and tune model hyper-parameters to correctly detect waypoints for the arrival traffic into the San Francisco bay area. We will compute the complexity measure over a one-year period using our proposed technique as well as the baseline. Our validation will be based on each technique’s ability to detect a set of undesirable outcomes (e.g., go-arounds, holding patterns, average time in the airspace, etc.). Since our current complexity metric is derived from the offset from the direct reference path, it’s important to understand what causes these offsets. In many of the flights with high offset distance, flights performing holding patterns and S turns can be observed. These maneuvering tactics are utilized to add distance between the aircraft and the destination runway to prevent multiple flights from having conflicting arrival times. In order to predict a rise in complexity (or the precursor to complexity), it’s necessary to be able to identify these potential conflicts (which in turn, result in higher offsets). To do this, we define a “representative flight” for each STAR route and runway pair. This flight is approximately the path the flight would take if there was a clear path with no other flights in the airspace — including the time remaining to the airport. We first identify the flights for a given STAR runway pair using the offset to the reference path distributions that fall between the 44-55 percentiles. This yields the flights that conform to the most normal mode of operation. Each of these flights is partitioned based on the percent complete from the entry point into the maneuvering areas from 0\% – 100\% complete. Then for each percent “bin”, we take the median value of the flight’s latitude/longitude coordinates, airspeed, and (non causal) time remaining to the airport to construct a lookup table for each percent complete bin on a given route. As a flight enters the maneuvering area, we can find the estimated arrival time of a flight to the airport by finding the closest point to the representative path’s percent complete bin (relative to the flight’s current position at any snapshot in the airspace) and therefore retrieve the corresponding remaining time left on the “representative path”. We assume that the flight will follow the representative path to completion when deriving these estimates. We can then compare these estimated arrival times against other flights for the same snapshot in time to identify potential conflicts. If more flights are estimated to arrive within a tolerance window than there are runways available, then we have a potential conflict. We can use this derived measure along with other factors expected to add disruption to the operation such as weather and runway configuration changes as an input to machine learning tools to detect precursors that increases in our complexity measure. This novel method will assist in uncovering insights into the contributing factors that lead to increased complexity that may allow for in-time responses to avoid reaching a high complexity state in the airspace.

complexity↗

Use of Business Intelligence Tools in the DSN

JPL has operated the Deep Space Network (DSN) on behalf of NASA since the 1960's. Over the last two decades, the DSN budget has generally declined in real-year dollars while the aging assets required more attention, and the missions became more complex. As a result, the DSN budget has been increasingly consumed by Operations and Maintenance (O&M), significantly reducing the funding wedge available for technology investment and for enhancing the DSN capability and capacity. Responding to this budget squeeze, the DSN launched an effort to improve the cost-efficiency of the O&M. In this paper we: elaborate on the methodology adopted to understand "where the time and money are used"-surprisingly, most of the data required for metrics development was readily available in existing databases-we have used commercial Business Intelligence (BI) tools to mine the databases and automatically extract the metrics (including trends) and distribute them weekly to interested parties; describe the DSN-specific effort to convert the intuitive understanding of "where the time is spent" into meaningful and actionable metrics that quantify use of resources, highlight candidate areas of improvement, and establish trends; and discuss the use of the BI-derived metrics-one of the most fascinating processes was the dramatic improvement in some areas of operations when the metrics were shared with the operators-the visibility of the metrics, and a self-induced competition, caused almost immediate improvement in some areas. While the near-term use of the metrics is to quantify the processes and track the improvement, these techniques will be just as useful in monitoring the process, e.g. as an input to a lean-six-sigma process.

Metrics↗

On the Integration of Loudness Over Time for the Prediction of Single-Event Annoyance

Loudness is the quality of human perception that is most related to acoustic intensity and energy, however there is no agreed-upon way of integrating loudness over time for the prediction of noise-induced annoyance. Most noise metrics used today for aircraft certification and regulation employ the “Equal-Energy Hypothesis” (EEH) and sum the acoustical energy of one or more noise events over time to create a single metric value that represents the entire exposure. Using the EEH creates a natural tradeoff between the peak energy and the duration of a single noise event, the “hypothesis” being that this tradeoff optimally predicts annoyance in a wide variety of situations. This work proposes a strategy for integrating a loudness-like time series which is flexible in two ways: First, it includes a parameter b ∈ [0,1]. When b = 0, the metric will return the peak of the time series. At b = .5, the metric will behave in accordance with the EEH. At b = 1, the metric will penalize the duration of the sound more than the EEH would. Second, transformations are given so that the strategy can be computed from, or generate quantities in units analogous to, decibels (or phon), physical units (acoustic pressure/power), or perceptually-scaled units (sone). This approach is demonstrated on a dataset of annoyance responses to single events of UAV and road vehicle noise that is fit with an augmented linear regression. Analyses based on this integration of A-weighted level and the output of the “Zwicker” loudness model yield similar results: that subjects may have been slightly more sensitive to the durations of the events than the EEH would suppose, but that the EEH cannot be disproven using these data. The time-integrated metrics outperform both time-averaged and centile-based metrics.

Psychoacoustics↗

On the Integration of Loudness Over Time for the Prediction of Single-Event Annoyance

Loudness is the quality of human perception that is most related to acoustic intensity and energy, however there is no agreed-upon way of integrating loudness over time for the prediction of noise-induced annoyance. Most noise metrics used today for aircraft certification and regulation employ the “Equal-Energy Hypothesis” (EEH) and sum the acoustical energy of one or more noise events over time to create a single metric value that represents the entire exposure. Using the EEH creates a natural tradeoff between the peak energy and the duration of a single noise event, the “hypothesis” being that this tradeoff optimally predicts annoyance in a wide variety of situations. This work proposes a strategy for integrating a loudness-like time series which is flexible in two ways: First, it includes a parameter b ∈ [0,1]. When b = 0, the metric will return the peak of the time series. At b = .5, the metric will behave in accordance with the EEH. At b = 1, the metric will penalize the duration of the sound more than the EEH would. Second, transformations are given so that the strategy can be computed from, or generate quantities in units analogous to, decibels (or phon), physical units (acoustic pressure/power), or perceptually-scaled units (sone). This approach is demonstrated on a dataset of annoyance responses to single events of UAV and road vehicle noise that is fit with an augmented linear regression. Analyses based on this integration of A-weighted level and the output of the “Zwicker” loudness model yield similar results: that subjects may have been slightly more sensitive to the durations of the events than the EEH would suppose, but that the EEH cannot be disproven using these data. The time-integrated metrics outperform both time-averaged and centile-based metrics.

Psychoacoustics↗

Multiscale assessment of land surface phenology from harmonized Landsat 8 and Sentinel-2, PlanetScope, and PhenoCam imagery

As the spatial and temporal resolution of remotely sensed imagery has improved over the last four decades, algorithms for monitoring and mapping seasonal changes in surface properties have evolved rapidly. Most recently, the availability of daily PlanetScope imagery has created new opportunities for monitoring the land surface phenology (LSP) of terrestrial ecosystems at high spatial resolution. However, the quality and value of LSP information from PlanetScope imagery have not been systematically examined. In this paper, we evaluate the character and quality of LSP information derived from PlanetScope by comparing time series of vegetation indices and LSP metrics from PlanetScope to corresponding time series and LSP metrics derived from Harmonized Landsat 8 and Sentinel-2 (HLS) imagery and PhenoCams at six sites that span a diverse range of land cover types and climate. Results show that vegetation index time series from all three data sources show high temporal correlation, and LSP metrics derived from HLS, PlanetScope, and PhenoCam show high agreement with negligible bias. Semi-variograms for phenometrics estimated from PlanetScope imagery indicate that the majority of spatial variance captured in PlanetScope phenometrics occurs well below the spatial resolution HLS imagery. At the same time, LSP metrics from HLS are most strongly correlated with the 50–75% quantiles of 3 m LSP metrics from PlanetScope. This indicates that HLS captures the average phenology at sub-pixel scale captured in PlanetScope imagery. Here, our results represent the first comprehensive comparison of LSP metrics estimated from PlanetScope and publicly available moderate spatial resolution imagery, and provide insights regarding: (1) the quality and character of LSP metrics derived from HLS and PlanetScope; and (2) the relative merits and trade-offs associated with the use of each data source for LSP studies.

54 ENVIRONMENTAL SCIENCES↗

Visualization Quality Assessment

Understanding how inaccuracies in visualizations affect users’ perception and understanding of scientific data is hard. Inaccuracies in visualizations are quite common and could arise from a range of sources such as errors in the original dataset arising from compression artifacts, errors in the capturing device, noise during transmission of the data, effects due to the algorithm being used to convert data to visualization images, images generated from neural networks, and sources we have yet to discover. Many image quality assessment metrics have been developed to quantify image errors. However, these are usually focused on “natural images” rather than visualizations of scientific data. Common image quality assessment metrics (IQAs) include MSE, PSNR, perceptual metrics such SSIM, FSIM as well as perceptual metrics using deep learning approaches. However, a critical part of understanding how errors are perceived by humans, and subsequently developing more accurate quality assessment metrics, is through user evaluation studies. The goal of this software is to develop a visualization quality assessment (VQA) process that will enable the generation of VQAs that can be used to quantify errors in scientific data visualizations. The VQA development process will include software to support user evaluation experimental design, analysis of visualization differences against standard quality metrics, and the ability to develop additional VQA metrics specific to scientific visualization images.

Grosset, Andre↗

Austin Sustainable and Holistic Integration of Energy Storage and Solar PV [Austin SHINES]. Final Report, Version 2

The Austin SHINES project and solution is a software management platform, for an electric grid with a high penetration of dispersed photovoltaic (PV) solar generation sites, which maintains the traditional power quality and reliability associated with grid service. This project developed and deployed the platform as a Distributed Energy Resource Management System (DERMS), engaging multiple advanced controls, to evaluate operation and optimization of a fleet of diverse DER assets, installed at several locations among Austin Energy’s customers and distribution system. The project also produced a methodology to create a replicable DERMS template, adaptable to other regions and market structures. Last, Austin SHINES aimed to demonstrate the solution’s methodology would enable the DER grid ecosystem to serve load at a technical cost (System Levelized Cost of Electricity, or System LCOE) of less than the U.S. Department of Energy SHINES program metric of $0.14/kWh, in a defined boundary, while enabling a high penetration of distributed PV. Research was categorized in 6 reports (Final Deliverables = FD) listed below, with titles and descriptions indicating which area of understanding was investigated: FD-1: System Levelized Cost of Electricity (System LCOE) Methodology The creation and use of the System LCOE to Serve Load metric that encompasses the holistic, system-level costs and benefits of all resources, and enables them to be evaluated based on their ability to support an efficient and low-cost integrated grid ecosystem. FD-2: Software Platform Product Description The creation of new DER control methodologies deployable within a utility-grade software platform that enable DER's to maximize their benefit within a grid, that is capable of serving load enabling a high penetration of distributed PV generation. FD-3: Optimal Design Methodology Optimal design methodologies for individual DER installations that enable utilities to determine the optimal combinations and sizing for individual DER sites. FD-4: Austin SHINES Ownership and Operation Models for DER System Performance A comparison of multiple DER aggregation and ownership methodologies including direct utility control, third-party aggregator, and autonomous. FD-5: Economic Modeling & Optimization A comparison of multiple DER technology mixes and configurations within the distribution system, providing insight into an optimal blend of technologies that best enable the distribution system to serve load at the lowest cost at high penetrations of solar. FD-6: Fielded Assets Deployed DER assets within the Austin Energy SHINES circuits. Austin SHINES provided an opening for state-of-the-art technology products to be deployed, providing a rich opportunity for improving how each of the products perform as stand-alone products, and in concert with other complementary products. The Austin SHINES project comprised of two key metrics for System LCOE: SystemLCOE_SHINES<$0.14/kWh Modeled ΔSystemLCOE_SHINES/ΔSystemLCOE_Base≥20% at same solar penetration The System LCOE calculation uses the costs of the utility-owned infrastructure as it exists today, the cost of the DERs that exist in the system today, and the cost of the purchase of energy from ERCOT wholesale markets over the course of the calendar year. All costs are on an annualized basis. The capital and operating costs are derived from the rate case, which produces a yearly cost. The net cost of energy and services imported to the system is integrated over the test year, as is the load served and solar penetration. The first metric was easily achieved by every scenario considered. The goal was set when the Department of Energy’s SHINES Funding Opportunity Announcement was written in 2015 and was a more difficult target at the time. Due mostly to rapidly declining costs for DERs and the significant decrease in the Electric Reliability Council of Texas (ERCOT) energy market prices, which results in lower net cost of energy purchases, the System LCOE is well below this target for all scenarios considered. A fleet of DERs can assume different mixtures, each of which serves the load at a different LCOE. The optimal mixture of DERs serves load at the smallest System LCOE. The second metric (hereinafter %delta metric) asks that the holistic DERMS controls reduce the incremental cost above the baseline of going to a high solar penetration future by at least 20% as compared to the case of a DER deployment with no sophisticated controls (autonomous). Many comparison sets were created throughout this project. Physical technology was installed for informing utility engineering and testing several types of operational control schemes, through the DERMS. The types of operational control which were compared for valuation of the System LCOE Metric were: Holistic control = using the full suite of the DERMS platform to decide and optimize how/why the systems operate depending on weather, market, and reliability signal input. Autonomous control = a local mode at the asset site, wherein a schedule operates the asset, with visibility into performance only No control = the baseline for comparing value against the other two types of control The types of ownership control included: Direct Utility control = the utility dispatches a signal to each asset Third-Party Aggregator = a third party aggregates a fleet of assets and the utility dispatches one signal for all Autonomous = a local mode is set for operation at the asset site, wherein a schedule operates the asset, with visibility into performance only The types of control methodologies deployable within a utility-grade software platform included: Utility Peak Load Reduction = Lower transmission cost obligation Day-Ahead Energy Arbitrage = Realize economic value through price differential Real-Time Price Dispatch = Realize economic value from real-time price spikes Voltage support = Reduce losses and increase solar generation Distribution Congestion Management = Increase local grid reliability Demand Charge Reduction = Lower customer bills and realize system benefit The fielded assets deployed for the project were: Utility Scale Kingsbery Energy Storage System: 1.5 MW / 3 MWh Li-Ion battery storage Mueller Energy Storage System: 1.75 MW / 3.2 MWh Li-Ion battery storage, 7 Energy Storage Units (250 kW each) La Loma Community Solar: 2.6 MW Commercial Scale Aggregated storage installations at 3 sites, with existing solar (300+ kW): One 18 kW / 36 kWh Li-Ion battery storage Two 72 kW / 144 kWh Li-Ion battery storage Residential Scale Aggregated storage installations: -Six stationary battery storage systems (10 kWh each) at homes with existing solar -One Electric Vehicle installed as Vehicle-to-Grid (V2G) Utility-Controlled Solar via Smart Inverters at 12 homes Autonomously-Controlled Smart Inverters at 6 homes Over the course of the project, Austin SHINES undertook installing more than 3 MW of distributed battery energy storage, smart PV inverters, a DER control platform, and other enabling technologies utilizing customer and utility locations and aggregation models. All of these resources were to be integrated and optimized at the utility level. DER assets and control methodologies were designed to achieve a credible pathway to a System LCOE for energy delivered to load of $0.14//kWh or less by 2020, while maximizing distributed solar generation and maintaining acceptable standards of power quality. The project also established a template for other regions to follow, to maximize the adoption of distributed solar PV in support of an economic and efficient grid. In total, the Austin SHINES project added value to the DER subject area in each layer of integration. From utility, to commercial to residential scales, the sheer hierarchy of communication and coordination was a significant accomplishment in addition to learnings from what these communications revealed was unique to each. Economically, the most effective method demonstrated was the criticality of planning phases. Contingencies and multiple projection scenarios helped guide the project to deploy optimal design as close as feasible, in real world conditions. The project and reports will serve public benefit by outlining specific areas of DER strategy and installation where many stakeholders and needs can be addressed with improved efficiency. Overall, communities and utilities should use the results to guide the increasing options available for powering the grid with DER, renewables, and carbon considerate energy.

14 SOLAR ENERGY↗

Initial Ada components evaluation

The SAIC has the responsibility for independent test and validation of the SSE. They have been using a mathematical functions library package implemented in Ada to test the SSE IV and V process. The library package consists of elementary mathematical functions and is both machine and accuracy independent. The SSE Ada components evaluation includes code complexity metrics based on Halstead's software science metrics and McCabe's measure of cyclomatic complexity. Halstead's metrics are based on the number of operators and operands on a logical unit of code and are compiled from the number of distinct operators, distinct operands, and total number of occurrences of operators and operands. These metrics give an indication of the physical size of a program in terms of operators and operands and are used diagnostically to point to potential problems. McCabe's Cyclomatic Complexity Metrics (CCM) are compiled from flow charts transformed to equivalent directed graphs. The CCM is a measure of the total number of linearly independent paths through the code's control structure. These metrics were computed for the Ada mathematical functions library using Software Automated Verification and Validation (SAVVAS), the SSE IV and V tool. A table with selected results was shown, indicating that most of these routines are of good quality. Thresholds for the Halstead measures indicate poor quality if the length metric exceeds 260 or difficulty is greater than 190. The McCabe CCM indicated a high quality of software products.

Moebes, Travis↗

Solar Type II Radio Bursts and IP Type II Events

We have examined radio data from the WAVES experiment on the Wind spacecraft in conjunction with ground-based data in order to investigate the relationship between the shocks responsible for metric type II radio bursts and the shocks in front of coronal mass ejections (CMEs). The bow shocks of fast, large CMEs are strong interplanetary (IP) shocks, and the associated radio emissions often consist of single broad bands starting below approx. 4 MHz; such emissions were previously called IP type II events. In contrast, metric type II bursts are usually narrowbanded and display two harmonically related bands. In addition to displaying complete dynamic spectra for a number of events, we also analyze the 135 WAVES 1 - 14 MHz slow-drift time periods in 2001-2003. We find that most of the periods contain multiple phenomena, which we divide into three groups: metric type II extensions, IP type II events, and blobs and bands. About half of the WAVES listings include probable extensions of metric type II radio bursts, but in more than half of these events, there were also other slow-drift features. In the 3 yr study period, there were 31 IP type II events; these were associated with the very fastest CMEs. The most common form of activity in the WAVES events, blobs and bands in the frequency range between 1 and 8 MHz, fall below an envelope consistent with the early signatures of an IP type II event. However, most of this activity lasts only a few tens of minutes, whereas IP type II events last for many hours. In this study we find many examples in the radio data of two shock-like phenomena with different characteristics that occur simultaneously in the metric and decametric/hectometric bands, and no clear example of a metric type II burst that extends continuously down in frequency to become an IP type II event. The simplest interpretation is that metric type II bursts, unlike IP type II events, are not caused by shocks driven in front of CMEs.

Cane, H. V.↗

A Predictive Approach to Eliminating Errors in Software Code

NASA s Metrics Data Program Data Repository is a database that stores problem, product, and metrics data. The primary goal of this data repository is to provide project data to the software community. In doing so, the Metrics Data Program collects artifacts from a large NASA dataset, generates metrics on the artifacts, and then generates reports that are made available to the public at no cost. The data that are made available to general users have been sanitized and authorized for publication through the Metrics Data Program Web site by officials representing the projects from which the data originated. The data repository is operated by NASA s Independent Verification and Validation (IV&V) Facility, which is located in Fairmont, West Virginia, a high-tech hub for emerging innovation in the Mountain State. The IV&V Facility was founded in 1993, under the NASA Office of Safety and Mission Assurance, as a direct result of recommendations made by the National Research Council and the Report of the Presidential Commission on the Space Shuttle Challenger Accident. Today, under the direction of Goddard Space Flight Center, the IV&V Facility continues its mission to provide the highest achievable levels of safety and cost-effectiveness for mission-critical software. By extending its data to public users, the facility has helped improve the safety, reliability, and quality of complex software systems throughout private industry and other government agencies. Integrated Software Metrics, Inc., is one of the organizations that has benefited from studying the metrics data. As a result, the company has evolved into a leading developer of innovative software-error prediction tools that help organizations deliver better software, on time and on budget.

Source record↗

Geospace Environment Modeling 2008-2009 Challenge: Ground Magnetic Field Perturbations

Acquiring quantitative metrics!based knowledge about the performance of various space physics modeling approaches is central for the space weather community. Quantification of the performance helps the users of the modeling products to better understand the capabilities of the models and to choose the approach that best suits their specific needs. Further, metrics!based analyses are important for addressing the differences between various modeling approaches and for measuring and guiding the progress in the field. In this paper, the metrics!based results of the ground magnetic field perturbation part of the Geospace Environment Modeling 2008 2009 Challenge are reported. Predictions made by 14 different models, including an ensemble model, are compared to geomagnetic observatory recordings from 12 different northern hemispheric locations. Five different metrics are used to quantify the model performances for four storm events. It is shown that the ranking of the models is strongly dependent on the type of metric used to evaluate the model performance. None of the models rank near or at the top systematically for all used metrics. Consequently, one cannot pick the absolute winner : the choice for the best model depends on the characteristics of the signal one is interested in. Model performances vary also from event to event. This is particularly clear for root!mean!square difference and utility metric!based analyses. Further, analyses indicate that for some of the models, increasing the global magnetohydrodynamic model spatial resolution and the inclusion of the ring current dynamics improve the models capability to generate more realistic ground magnetic field fluctuations.

Pulkkinen, A.↗

Examining Perceptual Luminance Uniformity of Simulated Luminaire Patterns

Luminaire luminance uniformity is an important aspect that can affect perceived lighting quality, discomfort glare, and efficacy. While several metrics have been proposed to characterize luminance uniformity, previous studies have shown that current metrics such as Max:Min or Avg:Min luminance ratios can be insensitive to important differences in luminance gradient that may affect perceived uniformity. In an attempt to resolve this issue, previous studies incorporated a contrast sensitivity function for the human eye based on spatial frequency, such as in the UHVS metric; however, this metric has not been comprehensively studied in relation to perceived uniformity ratings. The study presented in this paper aimed to examine the relationship between the UHVS metric and perceived uniformity ratings. Specifically, the study used a web-based questionnaire that presented simulated luminance patterns and asked participants to assess uniformity using a two-alternative forced-choice procedure. Responses were collected and analyzed from 94 participants. The results showed a significant correlation and relationship between the UHVS metric and perceived uniformity. However, comparisons between patterns that had similar UHVS and between patterns that had larger differences in UHVS did not yield consistent results. These results suggest that the UHVS metric might be used for general guidance but may warrant further studies to better understand its sensitivity and improve its alignment with perceived uniformity ratings.

Abboushi, Belal K.↗

Resilience assessment and planning in power distribution systems: Past and future considerations

High impact low probability (HILP) events such as hurricanes, heat waves, and floods have instigated widespread power outages and blackouts around the globe in the past decade. With the increasing challenges concerning the threats to the power distribution systems and the growing need to mitigate the impacts of the HILP events, resilience has become a crucial requirement for the power grid infrastructures. Numerous efforts have been made to define, measure, and characterize the resilience of power distribution systems. This study thoroughly reviews the state-of-the-art methods on the existing resilience evaluation framework and metrics. Here, the desirable characteristics of resilience metrics are highlighted, and the challenges associated with formulating, developing, and calculating such metrics are discussed. Next, we detail the state-of-the-art literature on planning solutions to ensure distribution system resilience. This paper aims to extract a deep insight into this challenging and critical research area and envision future opportunities that can guide the power distribution system operators/planners to formulate more effective mitigation strategies to enhance the resilience of power distribution systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Combining synchrotron X-ray diffraction, mechanistic modeling and machine learning for in situ subsurface temperature quantification during laser melting

Laser melting, such as that encountered during additive manufacturing, produces extreme gradients of temperature in both space and time, which in turn influence microstructural development in the material. Qualification and model validation of the process itself and the resulting material necessitate the ability to characterize these temperature fields. However, well established means to directly probe the material temperature below the surface of an alloy while it is being processed are limited. To address this gap in characterization capabilities, a novel means is presented to extract subsurface temperature-distribution metrics, with uncertainty, from in situ synchrotron X-ray diffraction measurements to provide quantitative temperature evolution data during laser melting. Temperature-distribution metrics are determined using Gaussian process regression supervised machine-learning surrogate models trained with a combination of mechanistic modeling (heat transfer and fluid flow) and X-ray diffraction simulation. The trained surrogate model uncertainties are found to range from 5 to 15% depending on the metric and current temperature. The surrogate models are then applied to experimental data to extract temperature metrics from an Inconel 625 nickel superalloy wall specimen during laser melting. The maximum temperatures of the solid phase in the diffraction volume through melting and cooling are found to reach the solidus temperature as expected, with the mean and minimum temperatures found to be several hundred degrees less. The extracted temperature metrics near melting are determined to be more accurate because of the lower relative levels of mechanical elastic strains. However, uncertainties for temperature metrics during cooling are increased due to the effects of thermomechanical stress.

36 MATERIALS SCIENCE↗

Comprehensive Approach to Measure the Mobility Energy Productivity of Freight Transport

Freight travel accounts for a major share of the energy consumed in the transportation sector in any country, and the United States is no exception. Understanding and modeling freight movement are critical, particularly in the context of capturing the impact of emerging technologies on freight travel and its externalities. The domain of freight modeling and forecasting has been gaining pace in recent years, but advancement in comprehensive freight performance metrics is still lagging. Conventional freight performance metrics such as truck-miles, ton-miles, or value-miles are unidimensional and aggregate in nature, making them unsuitable to accurately capture the impact of emerging transportation trends on the performance or productivity of freight systems. Addressing the research need, this paper presents the “Freight Mobility Energy Productivity” metric to quantify freight productivity of current as well as future freight systems, accounting for various costs associated with freight transport. The proposed metric was implemented using data from the Freight Analysis Framework along with other published sources, and shows intuitive results in quantifying freight productivity. Further, a scenario analysis exercise was conducted to test the capability of the metric in tracking improvements in system-level freight productivity as a result of vehicle electrification. The relative differences in Freight Mobility Energy Productivity scores help identify which zones benefit from the vehicle powertrain technology improvement. The results of the scenario analysis reinforce confidence that the proposed metric can be used as a decision support tool in assessing the productivity of existing as well as future freight trends and technologies.

47 OTHER INSTRUMENTATION↗

Adapting Existing Energy Planning, Simulation, and Operational Models for Resilience Analysis

Resilience is a topic receiving much attention in relation to energy systems, with particular attention being paid to the supply of electricity. As a result, research communities have proposed a plethora of candidate indicators and metrics for resilience, most of which remain immature at different scales and segments within the energy system. Given the complexity of resilience analyses and mitigation strategies, there is limited value in attempting to identify a single resilience metric, as no one metric can quantify resilience or its associated value for all stakeholders. Instead, a necessary focus of the research community should lie in implementing, testing, and validating resilience metrics and analysis approaches in energy sector models, which will be invaluable for informing resilience planning and investment decisions. Recognizing that implementing, testing, and validating resilience metrics are challenges that need to be addressed, the National Renewable Energy Laboratory (NREL) dedicated staff and time to researching how to effectively integrate resilience considerations into energy sector models and tools, as part of the Laboratory Directed Research and Development (LDRD) program. The overarching goal of the effort was to evaluate the data needs, methodologies, and outcomes - including consequences and/or changes in investment or operational decisions due to avoided consequences - based on resilience analysis in a range of existing tools. In particular, we selected five models originally built at NREL to explore non-resilience energy research questions to implement and exercise resilience metrics and analysis approaches.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Characterizing Families of Spectral Similarity Scores and Their Use Cases for Gas Chromatography–Mass Spectrometry Small Molecule Identification

Metabolomics provides a unique snapshot into the world of small molecules and the complex biological processes that govern the human, animal, plant, and environmental ecosystems encapsulated by the One Health modeling framework. However, this “molecular snapshot” is only as informative as the number of metabolites confidently identified within it. The spectral similarity (SS) score is traditionally used to identify compound(s) in mass spectrometry approaches to metabolomics, where spectra are matched to reference libraries of candidate spectra. Unfortunately, there is little consensus on which of the dozens of available SS metrics should be used. This lack of standard SS score creates analytic uncertainty and potentially leads to issues in reproducibility, especially as these data are integrated across other domains. In this work, we use metabolomic spectral similarity as a case study to showcase the challenges in consistency within just one piece of the One Health framework that must be addressed to enable data science approaches for One Health problems. Here, using a large cohort of datasets comprising both standard and complex datasets with expert-verified truth annotations, we evaluated the effectiveness of 66 similarity metrics to delineate between correct matches (true positives) and incorrect matches (true negatives). We additionally characterize the families of these metrics to make informed recommendations for their use. Our results indicate that specific families of metrics (the Inner Product, Correlative, and Intersection families of scores) tend to perform better than others, with no single similarity metric performing optimally for all queried spectra. This work and its findings provide an empirically-based resource for researchers to use in their selection of similarity metrics for GC-MS identification, increasing scientific reproducibility through taking steps towards standardizing identification workflows.

59 BASIC BIOLOGICAL SCIENCES↗