Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “metric learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS2007

An Evaluation of Training Interventions and Computed Scoring Techniques for Grading a Level Turn Task and a Straight In Landing Approach on a PC-Based Flight Simulator

One result of the relatively recent advances in computing technology has been the decreasing cost of computers and increasing computational power. This has allowed high fidelity airplane simulations to be run on personal computers (PC). Thus, simulators are now used routinely by pilots to substitute real flight hours for simulated flight hours for training for an aircraft type rating thereby reducing the cost of flight training. However, FAA regulations require that such substitution training must be supervised by Certified Flight Instructors (CFI). If the CFI presence could be reduced or eliminated for certain tasks this would mean a further cost savings to the pilot. This would require that the flight simulator have a certain level of 'intelligence' in order to provide feedback on pilot perfolmance similar to that of a CFI. The 'intelligent' flight sinlulator would have at least the capability to use data gathered from the flight to create a measure for the performance of the student pilot. Also, to fully utilize the advances in computational power, the sinlulator would be capable of interacting with the student pilot using the best possible training interventions. This thesis reposts on the two studies conducted at Tuskegee University investigating the effects of interventions on the learning of two flight maneuvers on a flight sinlulator and the robustness and accuracy of calculated perfornlance indices as compared to CFI evaluations of performance. The intent of these studies is to take a step in the direction of creating an 'intelligent' flight simulator. The first study deals with the comparisons of novice pilot performance trained at different levels of above real-time to execute a level S-turn. The second study examined the effect of out-of-the-window (OTW) visual cues in the form of hoops on the performance of novice pilots learning to fly a landing approach on the flight simulator. The reliability/robustness of the computed performance metrics was assessed by comparing them with the evaluations of the landing approach maneuver by a number of CFIs.

Heath, Bruce E.↗
DOE OSTI2026

Adaptive X-ray imaging with reinforcement learning

X-ray imaging is a powerful technique to scan samples in a variety of contexts including biological, environmental and materials science, but commonly requires a synchrotron light source to produce X-rays at sufficient intensity. As these facilities are expensive to operate, the available beam time is limited and always in high demand. Particularly if the illuminated samples are sparse, standard raster scanning methods can be time-consuming, with a majority of that time being spent on areas of the image that carry little information. To increase the efficiency and maximize the information gain for a given time budget, we split the scanning process into a series of steps where previous measurements are used to inform the decision making and adapt the exposure distribution at later stages of the sequence. We formulate this task as a reinforcement learning problem where the goal is to produce a sequence of exposure maps that maximize a predefined scalar metric. We demonstrate the potential of this approach in simulations where the adaptive illumination can accelerate the measurement process by up to an order of magnitude compared with standard raster scanning. Finally, we present the first results from deploying the trained agents on an X-ray fluorescence beamline at the Stanford Synchrotron Radiation Lightsource.

Reinforcement Learning↗
DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗
DOE OSTI2025

The search for high-entropy fuel-cell catalysts using disorder descriptors

The transition to a hydrogen economy depends on efficient, affordable catalysts for fuel cells. Platinum—the industry standard for fuel-cell electrodes—is costly and scarce, highlighting the need for practical alternatives. High-entropy alloys offer vast compositional diversity and tunable properties that can mitigate these issues, yet their chemical complexity and configurational disorder have hindered rational discovery. Here, we introduce a data-driven framework that couples machine learning with first-principles disorder descriptors—including the entropy forming ability, disordered enthalpy-entropy descriptor, and electronic-structure similarity metrics to platinum—to predict alloy synthesizability and catalytic performance. These descriptors are applied for the first time in the context of fuel-cell catalyst discovery. The workflow rapidly screens more than 20 000 compositions and identifies several platinum-free candidates that are economically viable, readily scalable, and exhibit promising predicted activity. These results demonstrate that disorder descriptors are reliably predicted by machine learning models and can be effectively integrated into materials-discovery pipelines, accelerating innovation across complex compositional spaces.

fuel-cell catalysts↗
DOE OSTI2024

Comparison of ML-Based Proxy Modeling Strategies: Lessons Learned from the SMART Initiative

Teams of researchers on Task 5 of the SMART project have developed a variety of modeling architectures to predict subsurface behavior during carbon injection and post-injection periods. One important part of this task was to compare the candidate approaches in terms of accuracy, reliability, speed, and memory use, all using a common set of metrics and visualizations for an “apples to apples” comparison. Dr. Jared Schuetter will share the details of this task, the results that were obtained, and the lessons learned.

Schuetter, Jared↗
NASA NTRS2020

Remote monitoring of agricultural systems using NDVI time series and machine learning methods: a tool for an adaptive agricultural policy

This study aims to provide accurate information about changes in agricultural systems (AS) using phenological metrics derived from the NDVI time series. Use of such information could help land managers optimize land use choices and monitor the status of agricultural lands, under a variety of environmental and socioeconomic conditions. For this purpose, the Moderate Resolution Imaging Spectroradiometer (MODIS) NDVI data were used to derive phenological metrics over the Oum Er-Rbia basin (central Morocco). Random forest (RF), support vector machine (SVM), and K-nearest neighbor (KNN) classifiers were explored and compared on their ability to classify AS classes over the study area. Four main AS classes have been considered: (1) irrigated annual crop (IAC), (2) irrigated perennial crop (IPC), (3) rainfed area (RA), and (4) fallow (FA). By comparing the accuracy of the three classifiers, the RF method showed the best performance with an overall accuracy of 0.97 and kappa coefficient of 0.96.The RF method was then chosen to examine time variations in AS over a 16-year period (2000–2016). The AS main variations were detected and evaluated for the four AS classes. These variations have been found to be linked well with other indicators of local agricultural land management, as well as the historical agricultural drought changes over the study area. Overall, the results present a tool for decision makers to improve agricultural management and provide a different perspective in understanding the spatiotemporal dynamics of agricultural systems.

Youssef Lebrini↗
NASA NTRS2003

Managing a project's legacy: implications for organizations and project management

Organizations that rely on projects to implement their products must find effective mechanisms for propagating lessons learned on one project throughout the organization. A broad view of what constitutes a project's 'legacy' is presented that includes not just the design products and leftover parts, but new processes, relationships, technology, skills, planning data, and performance metrics. Based on research evaluating knowledge reuse in innovative contexts, this paper presents an approach to project legacy management that focuses on collecting and using legacy knowledge to promote organizational learning and effective reuse, while addressing factors of post-project responsibility, information obsolescence, and the importance of ancillary contextual information. .

risk management↗
DOE OSTI2024

An extensible perturbed parameter ensemble for the Community Atmosphere Model version 6

This paper documents the methodology and preliminary results from a perturbed parameter ensemble (PPE) technique, where multiple parameters are varied simultaneously and the parameter values are determined with Latin hypercube sampling. This is done with the Community Atmosphere Model version 6 (CAM6), the atmospheric component of the Community Earth System Model version 2 (CESM2). We apply the PPE method to CESM2–CAM6 to understand climate sensitivity to atmospheric physics parameters. The initial simulations vary 45 parameters in the microphysics, convection, turbulence and aerosol schemes with 263 ensemble members. These atmospheric parameters are typically the most uncertain in many climate models. Control simulations and targeted simulations to understand climate forcing due to aerosols and fast climate feedbacks are analyzed. The use of various emulators is explored in the multi-dimensional space mapping input parameters to output metrics. Parameter impacts on various model outputs, such as radiation, cloud and aerosol properties, are evaluated. Machine learning is also used to probe optimal parameter values against observations. Our findings show that PPE is a valuable tool for climate uncertainty analysis. Furthermore, by varying many parameters simultaneously, we find that many different combinations of parameter values can produce results consistent with observations, and thus careful analysis of tuning is important. The CESM2–CAM6 PPE is publicly available and extensible to other configurations to address questions of other model processes in the atmosphere and other model components (e.g., coupling to the land surface).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2024

An Extensible Perturbed Parameter Ensemble for the Community Atmosphere Model Version 6

This paper documents the methodology and preliminary results from a Perturbed Parameter Ensemble (PPE) technique, where multiple parameters are varied simultaneously and the parameter values are determined with Latin hypercube sampling. This is done with the Community Atmosphere Model version 6 (CAM6), the atmospheric component of the Community Earth System Model version 2 (CESM2). We apply the PPE method to CESM2-CAM6 to understand climate sensitivity to atmospheric physics parameters. The initial simulations vary 45 parameters in the microphysics, convection, turbulence and aerosol schemes with 263 ensemble members. These atmospheric parameters are typically the most uncertain in many climate models. Control simulations are analyzed and targeted simulations to understand climate forcing due to aerosols and fast climate feedbacks. The use of various emulators is explored in the multi- dimensional space mapping input parameters to output metrics. Parameter impacts on various model outputs, such as radiation, cloud and aerosol properties are evaluated. Machine learning is also used to probe optimal parameter values against observations. Our findings show that using PPE is a valuable tool for climate uncertainty analysis. Furthermore, by varying many parameters simultaneously, we find that many different combinations of parameter values can produce results consistent with observations, and thus careful analysis of tuning is important. The CESM2-CAM6 PPE is publicly available, and extensible to other configurations to address questions of other model processes in the atmosphere and other model components (e.g. coupling to the land surface).

Machine learning↗
NASA NTRSDate not supplied

Open-Source Data Engineering at NASA: CCMC's Approach to Managing Petabyte-Scale Heliophysics Data

The Community Coordinated Modeling Center (CCMC) at NASA Goddard Space Flight Center (GSFC) leads heliophysics research by providing open access to numerous models and their outputs. Our resources are available on-demand and continuously updated with real-time data, covering sun-earth interactions across multiple domains. These domains include coronal, heliosphere, inner and global magnetosphere, ionosphere, thermosphere, and lower atmosphere interactions. Operating in a hybrid environment, CCMC utilizes both self-owned hardware and Amazon Web Services (AWS) cloud infrastructure. Managing petabytes of data across multiple locations necessitates robust data engineering solutions. To address this challenge, CCMC has adopted industry-standard and open-source tools. We use Apache Airflow as our primary data engineering platform, Python for scripting and data processing, and GitLab for version control and CI/CD. Additionally, we employ Kubernetes for containerized services, Grafana and Prometheus for metrics and monitoring, and Terraform and Puppet for reproducible infrastructure as code. This presentation will discuss lessons learned from our data engineering experiences, platforms evaluated but found unsuitable for our scientific data requirements, and specific techniques developed to enhance data transfer speed and reliability. By using these technologies effectively, CCMC continues to advance heliophysics research through efficient data management and open-access modeling.

space weather↗
DOE OSTI2026

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗
NASA NTRSDate not supplied

Analyzing Natural Language Context in Human-Machine Teaming using Supervised Machine Learning

Building a foundation for trustworthiness and trust verification in multi-asset teaming is the research challenge of Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR). The Design Reference Mission (DRM) for ATTRACTOR is a search and rescue mission objective governed by a multi-member team consisting of human and machine operators. A crucial component to the effort is the communication between humans and autonomous agents throughout both planning and execution stages of the mission. Intuitive communication methods and modalities are posited as critical enablers for certifying trust and trustworthiness. This paper reports on the data collection and analysis conducted in support of the Human Informed Natural-language GANs Evaluation (HINGE)project to attain explainable and trusted communication between human-machine assets. Two identically curated image description datasets were acquired for HINGE, both consisting of two unique input modalities (typed vs. verbal) and retrieved in two distinct contexts (general vs. specific). The gathered datasets were assessed and compared using Parts-of-Speech (POS)features, sentence similarity metrics, and linguistic analysis. Then, the datasets were modeled and tested separately and in combination with one another using machine learning algorithms. The comparison and testing results reveal a superior dataset, by which a preferred context and input is understood, for generating image representations of missing persons using a Generative Adversarial Network (GAN).

Bryan A Barrows↗
DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

High‐Asymmetry Metasurface: A New Solution for Terahertz Resonance via Active Learning‐Augmented Diffusion Model

Terahertz (THz) metamaterials with high‐figure‐of‐merit (high‐FoM) performance resonance are essential for advancing sensors, detectors, and imagers. Conventional designs focus on symmetric or low‐asymmetry geometric structures, leaving high‐asymmetry designs largely unexplored due to the inefficiency of trial‐and‐error‐based rational design. Recent deep learning techniques offer automation and acceleration but are constrained by the need for large datasets inherent to their data‐driven nature. Here, a novel prior knowledge‐guided generative model augmented by a physics‐constrained active learning mechanism to design high‐asymmetry metamaterials. An advanced diffusion model learns features from a small set of classical structures with high‐FoM THz resonance and generates new high‐asymmetry structures. To mitigate the limited number of classical structures, the generated high‐asymmetry structures are actively selected and integrated into the initial training dataset based on their physical characteristics. Experimental results demonstrate the superior resonance performance of the generated high‐asymmetry metamaterials over classical designs, exhibiting improvements exceeding 30% in key resonance metrics. Remarkably, this performance is attained using only 68 classical structures as the initial training dataset, significantly reducing the data requirements for deep learning‐based metamaterial design. The proposed scheme for generating high‐asymmetry structures provides a new effective and efficient solution for high‐FoM resonance, expanding applications in high‐sensitivity THz metadevices.

diffusion model↗
DOE OSTI2026

Assessing Metal Ion Assignment Accuracy in Protein Data Bank Models via Elemental Spectroscopy

Accurate representation of metal ions in macromolecular structures is critical for chemical interpretation, computational modeling, and machine-learning methods that rely on Protein Data Bank (PDB) entries. However, the elemental identity of metals modeled in crystallographic structures is often inferred indirectly and rarely validated experimentally. Here, we combine Particle Induced X-ray Emission (PIXE) and X-ray Fluorescence Spectroscopy (XRFS) to determine the elemental composition of protein samples used to generate 70 deposited metalloprotein crystal structures. By analyzing the original protein material employed for crystallization, but before the addition of crystallization buffer solutions, we assess whether the modeled metal ions in deposited structures are consistent with experimentally detectable elemental content. We find that in a majority of cases, the metals modeled in the corresponding PDB entries are inconsistent with the metals present in the protein samples before crystallization, or that additional metals are present but not represented in the structural models. Spectroscopic results were integrated with automated crystallographic validation metrics, including real-space Z-difference (RSZD) analysis and systematic rerefinement, to evaluate atomic-number mismatch at metal sites. PIXE and XRFS show strong agreement for dominant elemental signals and provide complementary, scalable approaches for identifying suspect metal assignments. This work does not address physiological or functional metalation but instead highlights a widespread data integrity issue in deposited macromolecular structures, PDB-wide. These results establish an experimentally corroborated link between elemental identity and crystallographic validation metrics, enabling the large-scale detection of chemically inconsistent annotations in structural databases used for computational modeling and machine learning.

Crystallization↗
DOE OSTI2026

A variational framework for residual-based adaptivity in neural PDE solvers and operator learning

Residual-based adaptive strategies are widely used in scientific machine learning yet remain largely heuristic. We introduce a variational framework that formalizes these methods through convex transformations of the residual, where different transformations correspond to distinct objective functionals. For instance, exponential weights target uniform error minimization, while linear weights recover quadratic error minimization. This perspective reveals adaptive weighting as a means of selecting sampling distributions that optimize a primal objective, directly linking discretization choices to error metrics. This principled approach yields three key benefits: it enables systematic design of adaptive schemes, reduces discretization error by lowering estimator variance, and enhances learning dynamics by improving gradient signal-to-noise ratio. Extending the framework to operator learning, we demonstrate substantial performance gains across diverse optimizers and architectures. Our results provide a theoretical perspective for residual-based adaptivity and establish a foundation for principled discretization and training.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Lessons Learned From Smallsat Microwave Sounders Data Assimilation in the NASA Goddard Earth Observing System (GEOS)

The NASA Time-Resolved Observations of Precipitation structure and storm Intensity with a Constellation of Smallsats (TROPICS) mission is a constellation of small satellites carrying state-of-art microwave temperature and humidity sounders with 12 channels between 91 GHz and 205 GHz frequency. Including TROPICS-pathfinder, launched on 30 June 2021, five TROPICS satellites operate and provide temperature and humidity data to NWP and atmospheric retrieval communities. This study evaluates the quality of radiance data from these five satellites. It assesses data impacts on NASA Goddard Earth Observing System (GEOS) global NWP analysis and forecasts while seeking answers to the following key questions: 1. What information do TROPICS satellites bring to NWP analysis? 2. How will TROPICS data assimilation affect NWP forecasts, including tropical cyclone analysis and forecasts? 3. What are the benefits of having multiple TROPICS satellites in NWP forecasts? 4. What are the weaknesses regarding instrument stability, consistency, and data quality of these CubeSats? 5. Can multiple TROPICS make similar impacts as conventional MW sensors such as ATMS or GPM Microwave Imager (GMI)? The sensitivities of each TROPICS channel to temperature, water vapor, cloud, and precipitation are examined in various weather conditions. Results from observing system experiments assimilating TROPICS radiance are used for evaluation metrics, including forecast skills and fit to other observations such as radiosondes and microwave and infrared sounders. Finally, the lessons learned from using these cube satellite radiance data in the GEOS atmospheric data assimilation system are shared.

Min-Jeong Kim↗
DOE OSTI2025

DOE FAIR Surrogate Benchmarks Supporting AI and Simulation Research (SBI Surrogate Benchmark Initiative) (Final Report)

Computational Science is being revolutionized by integrating AI and simulation and, in particular, by deep learning surrogate models that can replace all or part of traditional large‐scale HPC computations. Such surrogates can achieve remarkable performance improvements, as much as several orders of magnitude, and save both compute time and energy. The Surrogate Benchmark Initiative (SBI) project creates a community repository and FAIR (Findable, Accessible, Interoperable, and Reusable) data ecosystem for HPC application surrogate benchmarks. The SBI team comes from Argonne National Laboratory (ANL), Indiana University (IU), Rutgers University, the University of Tennessee, Knoxville (UTK), and the University of Virginia(UVA). SBI repositories include data, code, and all relevant collateral artifacts, that the science and engineering community needs to use and reuse these data sets and surrogates. SBI repositories generate active research from both participants in SBI and the broader AI and domain science communities. This project develops surrogates that use several different neural nets to learn and quickly infer the results of simulations and data systems and capture them as surrogate benchmarks with a rich set of metadata, covering. Data; Model; Metrics specification; Machine specification; Science, Speed, Power Results, We research FAIR metadata for these benchmarks. We develop application surrogate examples as benchmarks across many fields (ANL, UTK, IU, UVA). We also study non Surrogate benchmarks that have many common features and similar issues regarding FAIRness. We work with MLCommons (UVA, UTK), which is a major machine learning benchmarking activity where we get metadata ontologies, software, and benchmarks, benchmarks have datasets, models, and metadata, and they need a technical framework developed by UTK and Rutgers and deployed by UVA. We study features of Surrogates, including performance, training set size, and uncertainty quantification (Rutgers, UVA and IU).

97 MATHEMATICS AND COMPUTING↗