Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “task tuning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

HydraGNN_OPF_GFM_2026 - Ensemble of predictive graph foundation models for power grid applications

This dataset supports research on graph foundation models for optimal power flow (OPF) on electric grids using HydraGNN. It contains heterogeneous graph representations of PGLib-OPF cases spanning systems from 14 to 13,659 buses, together with packed HDF5 datasets for pretraining, feasibility classification, and N-1 contingency analysis. The release includes OPF solution data, downstream fine-tuning datasets, pretrained HeteroSAGE and HeteroHEAT model checkpoints, hyperparameter-optimization summaries across multiple heterogeneous GNN architectures, and aggregated fine-tuning results for sample-efficiency studies. The dataset is designed to enable scalable training, evaluation, and transfer-learning studies for OPF surrogate modeling, including node-level AC-OPF solution prediction, graph-level prediction, feasibility classification, operating-condition generalization, and contingency-response tasks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Generative large language models for predictive maintenance planning

Maintenance planning and the generation of necessary components for tasks can prove time-consuming and complex. Automating the creation of recurring or similar tasks by leveraging previous planning packages and data, while uncovering insights to automate planning package generation, presents an opportunity to conserve valuable time and resources. This work aims to harness the textual and probabilistic capabilities of large language models (LLMs) to automate the generation of planning packages. Utilizing diverse data sources ranging from raw data to handwritten text, both singular and collaborative LLMs are trained and tested. Results demonstrate their capability to generate essential planning package components, effectively replicating the statistical patterns in the data. This demonstrates the use of these tools inside a digital asset for automated planning. This work outlines a methodology for constructing datasets, a training suite, and evaluation methods for LLM-based textual and conversational planning tools utilized in an asset digital twin. Results indicate that the fine-tuned models generate estimated planning information within the statistical ranges observed in real maintenance data. The models achieve high accuracy (>90%) in document question-answering and instruction generation tasks. Furthermore, the conversational retrieval-augmented generation (RAG) assistant system achieves 100% document retrieval accuracy, while conversational information capture exceeds 98% across the majority of work-package assistant modules.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Capsule network-based semantic segmentation model for thermal anomaly identification on building envelopes

Thermography technology is widely used to inspect thermal anomalies in building façade systems. Computer vision-based techniques provide opportunities to autonomously detect such heat anomalies to significantly improve the efficiency of decision-making for building envelope retrofitting and maintenance. Here, in this work, we propose a novel Capsule Network-based deep learning model – CapsLab – that detects and identifies thermal anomalies by semantic segmentation. CapsLab is built based on our proposed prediction-tuning capsule (PT-Capsule) layer. Different from a traditional capsule layer, which consists of part-whole transformation and capsule-routing process, the proposed layer is composed of a prediction and tuning process, which helps decreasing the number of model parameters significantly. While the applicability of traditional Capsule Networks (CapsNets) has been limited to simpler tasks and smaller datasets due to their scalability issue, we can leverage the lightweight of the proposed PT-Capsule layer, and apply it to the semantic segmentation task. In this work, we also employ our previously presented performance metric, referred to as the Anomaly Identification Metric (AIM) (Kakillioglua et al. 2021), to evaluate the segmentation outputs. Traditional performance metrics do not accurately reflect the true performance of the segmentation models in thermal anomaly identification due to the high subjectivity in the annotation process and higher overlap ratio sensitivity of the standard metrics. AIM, on the other hand, is robust to these drawbacks. Experimental results show, both qualitatively and quantitatively, that our proposed segmentation method can effectively segment the thermal anomalies. Specifically, our model provides 9.38% and 13.53% improvements over the baseline model – DeepLabV3+ – based on traditional mIoU score and the AIM score, respectively, while requiring less model parameters and less computation at the same time. In addition, the scores that the AIM metric generates better align with the scores provided by building performance experts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Machine learning-based surrogate models and transfer learning for derivative free optimization of HT-PEM fuel cells

Widespread adoption of high-temperature polymer electrolyte membrane electrochemical systems, such as fuel cells (HT-PEMFCs), requires models and computational tools for accurate optimization and guiding new materials for enhancing performance and durability. In this contribution, knowledge-based modelling and data-driven modelling are combined using Few-Shot Learning and implementing an Automated Machine Learning framework for the generation of Machine Learning-based surrogate models. Applicability of the resulting model for derivative-free optimization is demonstrated. Additionally, a way of considering extrapolation in the optimization task is presented. Results show that although extrapolation is needed to achieve better solutions during optimization, it can be monitored and managed. As a result, tuning the electrode ionomer binder's properties, such as ionic conductivity, in the fuel cell represents a promising pathway for improving HT-PEMFC performance.

08 HYDROGEN↗
DOE OSTI2024

Battery State of Health Estimator: Cooperative Research and Development Final Report

NREL has developed a software tool to enable Renewance to estimate the degradation of batteries from basic information such as the type of battery and the application of that battery during its first life, so that used batteries may be evaluated for potential repurposing at low cost. This software tool utilizes NREL's BLAST-Lite battery degradation modeling code, which was updated with additional models for commercially produced lithium-ion batteries as a part of this CRADA. The software tool enables users to input details such as battery type and application so that lifetime estimates can be made without any programming or expert battery knowledge. The application input loads in saved values for parameters such as cycles per year, depth-of-discharge, and other battery operating parameters from a file defined by Renewance. These parameters may be modified to refine simulations for specific batteries. The software tool also incorporates a degradation model optimization tool, whereby existing battery degradation models may be tuned according to measured battery health. This ensures that new models still predict degradation behaviors expected from a certain battery chemistry, but with the overall degradation rate tuned to a specific battery make and model. The new model can then be saved for estimating the degradation of other similar batteries. An additional task was planned to utilize machine-learning to enable battery health diagnosis from rapid EIS measurements to accelerate the screening of used batteries. This task was not completed due to lack of available data for training a machine-learning model. CRADA benefit to DOE, Participant, and US Taxpayer: Further development of open-source software tool BLAST-Lite for predicting the lifetime of commercially produced Lithium-ion batteries (NREL SWR-22-69).

25 ENERGY STORAGE↗
DOE OSTI2026

Surrogate Neural Architecture Codesign Package (SNAC-Pack)

Neural architecture search (NAS) is a powerful approach for automating model design, but existing methods often optimize for accuracy alone or rely on proxy metrics such as bit operations (BOPs) that correlate poorly with hardware cost. This gap is particularly large for FPGA deployment, where cost is dominated by a multi-dimensional budget of lookup tables, DSPs, flip-flops, BRAM, and latency. We present the Surrogate Neural Architecture Codesign Package (SNAC-Pack), an open-source AutoML framework for hardware-aware neural architecture codesign and end-to-end FPGA deployment. SNAC-Pack runs a multi-objective global search with Optuna and NSGA-II, loading trials to a shared SQLite store that enables parallel workers across compute nodes. A hardware surrogate model outputs per-trial resource and latency estimates, avoiding the synthesis cost that would otherwise dominate the search loop. A local search stage then applies quantization-aware training (QAT) together with iterative magnitude pruning in a combined compression loop, after which the final model is synthesized to FPGA firmware via the hls4ml Python library. A YAML configuration and an optional agentic frontend let users run the pipeline on new datasets without modifying the framework. We demonstrate SNAC-Pack on jet classification at the Large Hadron Collider and superconducting qubit readout, discovering compact architectures that match or exceed strong baselines on the task metric while reducing FPGA resource utilization and, in the qubit readout case, reducing the design space exploration process from months of manual fine-tuning to hours of automated search.

Weitz, Jason [UC, San Diego]↗
DOE OSTI2026

Neural Scaling Laws for Jet Generation

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

Amram, Oz [Fermilab]↗
DOE OSTI2022

KGML-ag: a modeling framework of knowledge-guided machine learning to simulate agroecosystems: a case study of estimating N<sub>2</sub>O emission using data from mesocosm experiments

Abstract. Agricultural nitrous oxide (N2O) emission accounts for a non-trivial fraction of global greenhouse gas (GHG) budget. To date, estimating N2O fluxes from cropland remains a challenging task because the related microbial processes (e.g., nitrification and denitrification) are controlled by complex interactions among climate, soil, plant and human activities. Existing approaches such as process-based (PB) models have well-known limitations due to insufficient representations of the processes or uncertainties of model parameters, and due to leverage recent advances in machine learning (ML) a new method is needed to unlock the “black box” to overcome its limitations such as low interpretability, out-of-sample failure and massive data demand. In this study, we developed a first-of-its-kind knowledge-guided machine learning model for agroecosystems (KGML-ag) by incorporating biogeophysical and chemical domain knowledge from an advanced PB model, ecosys, and tested it by comparing simulating daily N2O fluxes with real observed data from mesocosm experiments. The gated recurrent unit (GRU) was used as the basis to build the model structure. To optimize the model performance, we have investigated a range of ideas, including (1) using initial values of intermediate variables (IMVs) instead of time series as model input to reduce data demand; (2) building hierarchical structures to explicitly estimate IMVs for further N2O prediction; (3) using multi-task learning to balance the simultaneous training on multiple variables; and (4) pre-training with millions of synthetic data generated from ecosys and fine-tuning with mesocosm observations. Six other pure ML models were developed using the same mesocosm data to serve as the benchmark for the KGML-ag model. Results show that KGML-ag did an excellent job in reproducing the mesocosm N2O fluxes (overall r2=0.81, and RMSE=3.6 mgNm-2d-1 from cross validation). Importantly, KGML-ag always outperforms the PB model and ML models in predicting N2O fluxes, especially for complex temporal dynamics and emission peaks. Besides, KGML-ag goes beyond the pure ML models by providing more interpretable predictions as well as pinpointing desired new knowledge and data to further empower the current KGML-ag. We believe the KGML-ag development in this study will stimulate a new body of research on interpretable ML for biogeochemistry and other related geoscience processes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Comparison of time-frequency-analysis techniques applied in building energy data noise cancellation for building load forecasting: A real-building case study

Time-frequency analysis that disaggregates a signal in both time and frequency domain is an important supporting technique for building energy analysis such as noise cancellation in data-driven building load forecasting. There is a gap in the literature related to comparing various time–frequency-analysis techniques, especially discrete wavelet transform (DWT) and empirical mode decomposition (EMD), to guide the selection and tuning of time–frequency-analysis techniques in data-driven building load forecasting. This article provides a framework to conduct a comprehensive comparison among thirteen DWT/EMD techniques with various parameters in a load forecasting modeling task. A real campus building is used as a case study for illustration. The DWT and EMD techniques are also compared under various data-driven modeling algorithms for building load forecasting. The results in the case study show that the load forecasting models trained with noise-cancelled energy data have increased their accuracy to 9.6% on average tested under unseen data. This study also shows that the effectiveness of DWT/EMD techniques depends on the data-driven algorithms used for load forecasting modeling and the training data. Hence, DWT/EMD-based noise cancellation needs customized selection and tuning to optimize their performance for data-driven building load forecasting modeling.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Developing Accelerated Test Protocols and Tuning Microstructures of the Common Materials to Improve Robustness, Reliability, and Endurance of SOFC Cells

This work is built on our prior work where we found that phase transformation in praseodymium nickelates, e.g. Pr 2 NiO 4 (PNO) and (Pr 1-x Nd x ) 2 NiO 4 (PNNO), can be electrochemically driven, and is substantially faster when compared to thermal annealing studies. The first task aims at an attempt to further accelerate the phase transformation in the oxygen electrode by alternating the current input in the cells, which lead to the development of accelerated test protocols (ATPs). ATPs showed up to 60x faster phase transformation and up to 10x faster performance degradation in (Pr 0.50 Nd 0.50 ) 2 NiO 4 electrodes, when compared to long-term operation under constant current density. Furthermore, the phase stable Nd 2 NiO 4 and (La 0.6 Sr 0.4 )(Co 0.8 Fe 0.2 )O 3 (LSCF6482) electrodes were tested in full cells under ATPs, and showed up to 10x faster performance degradation within 1,100 hours in a comparison with long-term thermal annealing studies and electrochemical operation under constant current density. The second task aims at the quantification of the contributions of cell components to the total impedance of a solid oxide fuel cell (SOFC) using electrochemical impedance spectroscopy (EIS) and distribution of relaxation times (DRT). Specifically, the role of gas composition at both anode and cathode was systematically studied, aiming at deconvoluting, identifying and quantifying the contributions of different electrode processes. This was achieved by first tuning the partial pressure of H 2 at the anode and subsequently varying the partial pressure of O 2 at the cathode. The results suggest that, while DRT offers a viable way of deconvoluting different times distributions, additional attention is needed before assigning a peak to a specific electrode process due to the significant overlap of the contributions from the cathode and the anode. Density function theory studies show that both Pr-vacancies and O-defects play a key role on the activity and stability for nickelates towards oxygen reduction reaction. The resident O-interstitials and oxygen ions in the PrO layer form peroxide (O 2 2- ) nearby Pr vacancies. The O 2 2- limits oxygen-ion transport due to the required additional energy to break its O-O bond. We further calculated the formation and segregation energies for different Ln ions (La, Pr, Nd, Pm, Sm, Gd, Tb, Dy, and Ho) in PNO and CeO 2 (111) surfaces. In addition to Nd, Pm and La are suggested as potential dopants in PNO to enhance it stability without decomposition due to their more negative formation energies, lower diffusion energies, and positive separation energies.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2024

Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research

Abstract Large language models (LLMs) are sophisticated AI-driven models trained on vast sources of natural language data. They are adept at generating responses that closely mimic human conversational patterns. One of the most notable examples is OpenAI's ChatGPT, which has been extensively used across diverse sectors. Despite their flexibility, a significant challenge arises as most users must transmit their data to the servers of companies operating these models. Utilizing ChatGPT or similar models online may inadvertently expose sensitive information to the risk of data breaches. Therefore, implementing LLMs that are open source and smaller in scale within a secure local network becomes a crucial step for organizations where ensuring data privacy and protection has the highest priority, such as regulatory agencies. As a feasibility evaluation, we implemented a series of open-source LLMs within a regulatory agency’s local network and assessed their performance on specific tasks involving extracting relevant clinical pharmacology information from regulatory drug labels. Our research shows that some models work well in the context of few- or zero-shot learning, achieving performance comparable, or even better than, neural network models that needed thousands of training samples. One of the models was selected to address a real-world issue of finding intrinsic factors that affect drugs' clinical exposure without any training or fine-tuning. In a dataset of over 700 000 sentences, the model showed a 78.5% accuracy rate. Our work pointed to the possibility of implementing open-source LLMs within a secure local network and using these models to perform various natural language processing tasks when large numbers of training examples are unavailable.

Biochemistry & Molecular Biology↗
DOE OSTI2020

How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection

Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Tackling the Challenges in Scene Graph Generation With Local-to-Global Interactions

In this work, we seek new insights into the underlying challenges of the scene graph generation (SGG) task. Quantitative and qualitative analysis of the visual genome (VG) dataset implies: 1) ambiguity: even if interobject relationship contains the same object (or predicate), they may not be visually or semantically similar; 2) asymmetry: despite the nature of the relationship that embodied the direction, it was not well addressed in previous studies; and 3) higher-order contexts: leveraging the identities of certain graph elements can help generate accurate scene graphs. Motivated by the analysis, we design a novel SGG framework, Local-to-global interaction networks (LOGINs). Locally, interactions extract the essence between three instances of subject, object, and background, while baking direction awareness into the network by explicitly constraining the input order of subject and object. Globally, interactions encode the contexts between every graph component (i.e., nodes and edges). Finally, Attract and Repel loss is utilized to fine-tune the distribution of predicate embeddings. By design, our framework enables predicting the scene graph in a bottom-up manner, leveraging the possible complementariness. To quantify how much LOGIN is aware of relational direction, a new diagnostic task called Bidirectional Relationship Classification (BRC) is also proposed. Overall, experimental results demonstrate that LOGIN can successfully distinguish relational direction than existing methods (in BRC task), while showing state-of-the-art results on the VG benchmark (in SGG task).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

CSPlib: A performance portable parallel software toolkit for analyzing complex kinetic mechanisms

Computational singular perturbation (CSP) is a method to analyze dynamical systems. It targets the decoupling of fast and slow dynamics using an alternate linear expansion of the right-hand side of the governing equations based on eigenanalysis of the associated Jacobian matrix. This representation facilitates diagnostic analysis, detection and control of stiffness, and the development of simplified models. For this work, we have implemented CSP in a C++ open-source library CSPlib using the Kokkos parallel programming model to address portability across diverse heterogeneous computing platforms, i.e., multi/many-core CPUs and GPUs. We describe the CSPlib implementation and present its computational performance across different computing platforms using several test problems. Specifically, we test the CSPlib performance for a constant pressure ignition reactor model on different architectures, including IBM Power 9, Intel Xeon Skylake, and NVIDIA V100 GPU. The size of the chemical kinetic mechanism is varied in these tests. As expected, the Jacobian matrix evaluation, the eigensolution of the Jacobian matrix, and matrix inversion are the most expensive computational tasks. When considering the higher throughput characteristic of GPUs, GPUs performs better for small matrices with higher occupancy rate. CPUs gain more advantages from the higher performance of well-tuned and optimized linear algebra libraries such as OpenBLAS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Serial2Parallel

In the era of machine learning, we often need to run the same code/script many times with little or no variations (e.g., performance evaluation, data preprocessing, data generation, hyperparameter tuning, etc.). It is not a problem when you just need to do that a few times, but when the number of repetitions becomes very large, it can be a daunting task. The code “Serial2Parallel” provides an easy way for users to be able to run many numbers of any serial code/scripts in a parallel manner across multiple nodes in an message passing interface (MPI) cluster. The code includes the server program that deals with task pool management and client program that processes task. The server gets the tasks ready and waits for clients' connections. The client code pulls tasks from the server and processes them. The client code will run in parallel.

Sangkeun, MattLee↗
DOE OSTI2022

OC6 Phase Ia: CFD Simulations of the Free-Decay Motion of the DeepCwind Semisubmersible

Currently, the design of floating offshore wind systems is primarily based on mid-fidelity models with empirical drag forces. The tuning of the model coefficients requires data from either experiments or high-fidelity simulations. As part of the OC6 (Offshore Code Comparison Collaboration, Continued, with Correlation, and unCertainty (OC6) is a project under the International Energy Agency Wind Task 30 framework) project, the present investigation explores the latter option. A verification and validation study of computational fluid dynamics (CFD) models of the DeepCwind semisubmersible undergoing free-decay motion is performed. Several institutions provided CFD results for validation against the OC6 experimental campaign. The objective is to evaluate whether the CFD setups of the participants can provide valid estimates of the hydrodynamic damping coefficients needed by mid-fidelity models. The linear and quadratic damping coefficients and the equivalent damping ratio are chosen as metrics for validation. Large numerical uncertainties are estimated for the linear and quadratic damping coefficients; however, the equivalent damping ratios are more consistently predicted with lower uncertainty. Some difference is observed between the experimental and CFD surge-decay motion, which is caused by mechanical damping not considered in the simulations that likely originated from the mooring setup, including a Coulomb-friction-type force. Overall, the simulations and the experiment show reasonable agreement, thus demonstrating the feasibility of using CFD simulations to tune mid-fidelity models.

17 WIND ENERGY↗
DOE OSTI2024

Improved particle-flow event reconstruction with scalable neural networks for current and future particle detectors

Abstract Efficient and accurate algorithms are necessary to reconstruct particles in the highly granular detectors anticipated at the High-Luminosity Large Hadron Collider and the Future Circular Collider. We study scalable machine learning models for event reconstruction in electron-positron collisions based on a full detector simulation. Particle-flow reconstruction can be formulated as a supervised learning task using tracks and calorimeter clusters. We compare a graph neural network and kernel-based transformer and demonstrate that we can avoid quadratic operations while achieving realistic reconstruction. We show that hyperparameter tuning significantly improves the performance of the models. The best graph neural network model shows improvement in the jet transverse momentum resolution by up to 50% compared to the rule-based algorithm. The resulting model is portable across Nvidia, AMD and Habana hardware. Accurate and fast machine-learning based reconstruction can significantly improve future measurements at colliders.

Physics↗