Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “language models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Evaluating and Explaining Natural Language Generation with GenX

Current methods for evaluation of natural language generation models focus on measuring text quality but fail to probe the model creativity, i.e., its ability to generate novel but coherent text sequences not seen in the training corpus. We present the GenX tool which is designed to enable interactive exploration and explanation of natural language generation outputs with a focus on the detection of memorization. We demonstrate the utility of the tool on two domain-conditioned generation use cases - phishing emails and ACL abstracts.

Duskin, Kayla R.↗
DOE OSTI2022

Towards a robust out-of-the-box neural network model for genomic data

The accurate prediction of biological features from genomic data is paramount for precision medicine and sustainable agriculture. For decades, neural network models have been widely popular in fields like computer vision, astrophysics and targeted marketing given their prediction accuracy and their robust performance under big data settings. Yet neural network models have not made a successful transition into the medical and biological world due to the ubiquitous characteristics of biological data such as modest sample sizes, sparsity, and extreme heterogeneity. Here, we investigate the robustness, generalization potential and prediction accuracy of widely used convolutional neural network and natural language processing models with a variety of heterogeneous genomic datasets. Mainly, recurrent neural network models outperform convolutional neural network models in terms of prediction accuracy, overfitting and transferability across the datasets under study. While the perspective of a robust out-of-the-box neural network model is out of reach, we identify certain model characteristics that translate well across datasets and could serve as a baseline model for translational researchers.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

QCOR; A Language Extension Specification for the Heterogeneous Quantum-Classical Model of Computation

Quantum computing (QC) is an emerging computational paradigm that leverages the laws of quantum mechanics to perform elementary logic operations. Existing programming models for QC were designed with fault-tolerant hardware in mind, envisioning stand-alone applications. However, the susceptibility of near-term quantum computers to noise limits their stand-alone utility. To better leverage limited computational strengths of noisy quantum devices, hybrid algorithms have been suggested whereby quantum computers are used in tandem with their classical counterparts in a heterogeneous fashion. This modus operandi calls out for a programming model and a high-level programming language that natively and seamlessly supports heterogeneous quantum-classical hardware architectures in a single-source-code paradigm. Motivated by the lack of such a model, we introduce a language extension specification, called QCOR, which enables single-source quantum-classical programming. Programs written using the QCOR library–based language extensions can be compiled to produce functional hybrid binary executables. After defining QCOR’s programming model, memory model, and execution model, we discuss how QCOR enables variational, iterative, and feed-forward QC. Additionally, QCOR approaches quantum-classical computation in a hardware-agnostic heterogeneous fashion and strives to build on best practices of high-performance computing. The high level of abstraction in the language extension is intended to accelerate the adoption of QC by researchers familiar with classical high-performance computing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A fractional calculus framework for open quantum dynamics: From Liouville to Lindblad to memory kernels

Open quantum systems exhibit dynamics ranging from unitary evolution to irreversible dissipation. While the Gorini–Kossakowski–Sudarshan–Lindblad equation uniquely characterizes Markovian completely positive and trace-preserving (CPTP) evolution, many physical platforms display non-Markovian features such as algebraic relaxation and coherence backflow. Fractional calculus provides a natural way to model such long-memory behavior through power-law temporal kernels introduced by fractional time derivatives. Here, we develop a unified framework that embeds fractional master equations within the broader hierarchy of open-system formalisms. The fractional equation forms a structured subclass of memory-kernel models, reduces to the Lindblad form at unit order, and, through Bochner–Phillips subordination, admits a CPTP representation as an average over Lindblad semigroups. Its resolvent structure further connects fractional dynamics to established non-Markovian approaches, including Nakajima–Zwanzig kernels and hierarchical equations of motion, providing a compact surrogate for long-memory effects. This formulation positions fractional calculus as a rigorous and practical language for modeling non-Markovian quantum dynamics in chemical physics and physical chemistry, providing a CPTP-preserving, computationally efficient surrogate for structured condensed-phase environments where long-time memory and dissipation play a central role.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Thermal Adaptation of Enzyme‐Mediated Processes Reduces Simulated Soil CO2 Fluxes Upon Soil Warming

Abstract Understanding factors influencing carbon effluxes from soils to the atmosphere is important in a world experiencing climatic change. Two important uncertainties related to soil organic carbon (SOC) stock responses to a changing climate are (a) whether soil microbial communities acclimate or adapt to changes in soil temperature and (b) how to represent this process in SOC models. To further explore these issues, we included thermal adaptation of enzyme‐mediated processes in a mechanistic SOC model (ReSOM) using the macromolecular rate theory. Thermal adaptation is defined here to encompass all potential responses of soil microbes and microbial communities following a change in temperature. To assess the effects of thermal adaptation of enzyme‐mediated processes on simulated SOC losses, ReSOM was applied to data collected from a 13‐year soil warming experiment. Results show that a model omitting thermal adaptation of enzyme‐mediated processes substantially overestimates observed CO 2 effluxes during the initial years of soil warming. The bias against observed CO 2 effluxes was lower for models including thermal adaptation of enzyme‐mediated processes. In addition, for a simulated linear 3°C soil warming over 100 years, models including thermal adaptation of enzyme‐mediated processes simulated SOC losses of a factor of three smaller than models omitting this process. As thermal adaptation of microbial community characteristics is generally not included in models simulating feedback between the soil, biosphere and atmosphere, we encourage future studies to assess the potential impact that microbial adaptation has on soil carbon – climate feedback representations in models. Plain Language Summary A major uncertainty in projecting how much soil organic carbon (SOC) will be converted to CO 2 as a consequence of climate change is related to how soil microbes may adapt to increasing soil temperatures. While this “microbial thermal adaptation” has been shown to occur in short‐term lab incubation experiments, its effect on SOC cycling on a decadal timescale is not clear. To address this knowledge gap, a mechanistic SOC model was used to simulate data collected from a 13‐year soil warming experiment, to assess how microbial thermal adaptation affects predicted SOC losses upon soil warming. The model results show that incorporating microbial thermal adaptation into the model led to reduced CO 2 effluxes from the soil to the atmosphere compared to the common approach of omitting this mechanism. Our results imply that projected SOC losses for the decades to come may be reduced when this mechanism is incorporated in land models. We therefore advocate for more research on the mechanisms controlling microbial thermal adaptation, and how to implement this mechanism in SOC models. Key Points A crucial aspect of soil organic carbon (SOC) models is the representation of soil microbes Predicted soil CO 2 fluxes upon soil warming are reduced when accounting for microbial thermal adaptation On a centennial time scale, this thermal adaptation results in up to a factor of three lower predicted SOC loss

Van de Broek, Marijn↗
DOE OSTI2023

Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation

We evaluate AI-assisted generative capabilities on fundamental numerical kernels in high-performance computing (HPC), including AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG. We test the generated kernel codes for a variety of language-supported programming models, including (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). We use the GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code as of April 2023 to generate a vast amount of implementations given simple + + prompt variants. To quantify and compare the results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. Results suggest that the OpenAI Codex outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general purpose Python can benefit from adding code keywords, while Julia prompts perform acceptably well for its mature programming models (e.g., Threads and CUDA.jl). We expect for these benchmarks to provide a point of reference for each programming model's community. Overall, understanding the convergence of large language models, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

Godoy, William↗
DOE OSTI2021

Machine Intelligence to Detect, Characterise, and Defend against Influence Operations in the Information Environment

Social media has enabled a new era of manipulation in the information and cognitive domains. Deceptive content—misleading, falsified, and fabricated—is routinely created and spread in the modern social media environment with the intent to create confusion and widen political and social divides, and exploit the societal conflict exacerbated by these divides in the real-world (aka physical domain). Such disinformation campaigns demonstrate a threat to the integrity of economic, political, cultural, public health, and national security institutions around the world. In this work we overview our artificial intelligence (AI) capabilities to detect, describe, and defend against information operations on Twitter as an example social platform to understand the influence of misleading and falsified content diffusion and better enable those charged with defending against such manipulation to enable responsive parties to counter it. We first present novel linguistically-informed deep learning (DL) models for misinformation and disinformation detection, and present an in-depth linguistic analysis of psycho-linguistic markers across broad deception categories. We then demonstrate how our models perform in the multilingual and multimodal setting and categorize falsified and misleading content based on the intent to deceive. We also provide a large-scale analysis to describe user behavior and spread patterns while engaging with deceptive content and report novel findings about the immediate diffusion of deceptive content by characterizing the vulnerable sub-populations and their demographics, and explicitly measuring speed and scale of deception spread to uncover who shares deceptive content, how quickly, how much, and how evenly. In addition, we measure audience reactions to misinformation and disinformation at scale, distinguishing the reactions of users identified as bots versus humans. Finally, we take advantage of deep translation and generation models to create unique solutions for real-time defense against digital deception and discuss how to apply causal inference to prescribe and intervene into strategic communications jointly across information, cognitive, and physical domains.

artificial intelligence, deep learning, neural lan↗
DOE OSTI2025

SWARM: Reimagining scientific workflow management systems in a distributed world

Modern scientific workflows process massive amounts of data from diverse instruments and sensors, leveraging geographically distributed, heterogeneous compute and storage resources—from leadership-class systems to edge devices—connected by high-performance networks. The diversity of resources introduces challenges in harnessing their full potential, with resilience issues arising across applications, system software, networks, storage, and hardware. Today, workflow management systems (WMS) coordinate the execution of computation and data management tasks across target resources. However, WMS’s centralized nature makes them vulnerable to faults and scalability issues that may result in failures of entire computational campaigns. In conclusion, this paper introduces a novel agentic framework for workflow management, fully distributing and decentralizing the WMS functions and modeling them as swarm intelligence agents infused with advanced artificial intelligence solutions and traditional distributed computing algorithms that can make coordinated decisions in the presence of failures of the underlying cyberinfrastructure.

Swarm intelligence↗
DOE OSTI2020

OpenABLext: An automatic code generation framework for agent-based simulations on CPU-GPU-FPGA heterogeneous platforms

The execution of agent-based simulations (ABSs) on hardware accelerator devices such as graphics processing units (GPUs) has been shown to offer great performance potentials. However, in heterogeneous hardware environments, it can become increasingly difficult to find viable partitions of the simulation and provide implementations for different hardware devices. To automate this process, we present OpenABLext, an extension to OpenABL, a model specification language for ABSs. By providing a device-aware OpenCL backend, OpenABLext enables the co-execution of ABS on heterogeneous hardware platforms consisting of central processing units, GPUs, and field programmable gate arrays (FPGAs).We present a novel online dispatching method that efficiently profiles partitions of the simulation during run-time to optimize the hardware assignment while using the profiling results to advance the simulation itself. In addition, OpenABLext features automated conflict resolution based on user-specified rules, supports graph-based simulation spaces, and utilizes an efficient neighbor search algorithm. We show the improved performance of OpenABLext and demonstrate the potential of FPGAs in the context of ABS. We illustrate how co-execution can be used to further lower execution times. OpenABLext can be seen as an enabler to tap the computing power of heterogeneous hardware platforms for ABS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Enhancing building resilience in cold climates: Integrating heat pump technologies with renewable energy

As electrification advances and Cold Climate Heat Pump technology progresses, ensuring grid stability becomes increasingly critical for effective heating in cold climates. However, natural disasters, especially during winter, pose significant threats to grid stability, impacting the reliability of air-source heat pumps. Despite these challenges, the integration of renewable energy sources and storage solutions in heating systems has not been extensively studied within the context of resilience. Here, this paper delves into the literature on renewable-powered heat pumps to assess their potential in enhancing building resilience in U.S. cold climate zones, which are particularly susceptible to extreme weather and grid disruptions. By leveraging renewable sources—solar, geothermal, and water—in conjunction with heat pump technology and supported by thermal or battery storage, this approach aims to provide a dependable solution for maintaining indoor heating during grid failures. Our analysis begins with a review of various renewable energy sources suitable for heat pumps, followed by an exploration of their application in cold climate regions across the U.S., and discussions on potential integration strategies with heat pump systems. This study highlights the advantages and suitability of solar irradiance and geothermal resources, emphasizing the importance of tailored, site-specific assessments to maximize energy efficiency and resilience. Additionally, it outlines the economic and environmental considerations necessary for implementing such systems and identifies potential challenges and areas for future research to facilitate the broader integration of renewable energy in heating solutions for enhanced resilience.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

The Road Ahead for Metal–Organic Frameworks: Current Landscape, Challenges and Future Prospects

This perspective highlights the transformative potential of Metal–Organic Frameworks (MOFs) in environmental and healthcare sectors. It discusses work that has advanced beyond technology readiness levels of >4 including applications in capture, storage, and conversion of gases to value added products. This work showcases efforts in the most salient applications of MOFs which have been performed at a great cadence, enabled by the federal government, large companies, and startups to commercialize these technologies despite facing significant challenges. This article also forecasts the role of nanoscale MOFs in healthcare, including strides toward personalized medicine, advocating for their use in custom-tailored drug delivery systems. Lastly, we underscore the potential acceleration in MOF research and development through the integration of machine learning and AI, positioning MOFs as versatile tools poised to address global sustainability and health challenges.

atmospheric chemistry↗
DOE OSTI2025

Autonomous Synthesis and Inverse Design of Electrochromic Polymers with High Efficiency and Accuracy

Here, the design and synthesis of functional polymers, aimed at targeted properties through specific structures, have long been challenged by their complex and often nonlinear structure–property relationships. Key processes, including knowledge accumulation for predictive design and experimental refinement and validation, are traditionally labor-insensitive and time-consuming, making it difficult to balance accuracy and efficiency. Here, we introduce an accelerated, autonomous system for the on-demand synthesis of electronic polymers that achieves the desired electrochromic functionality with high accuracy and efficiency. Our approach leverages large language model-assisted data mining, a physics-informed copolymer machine learning model, and an AI-driven autonomous robotic workflow in the Polybot lab. Within 72 h, Polybot autonomously synthesized electrochromic polymers (ECPs) with targeted, previously-unreported color values, including green polymers with specific absorption profiles, precisely fine-tuning copolymer structures with a 5% step size in comonomer composition within a three-monomer system. A publicly accessible ECP informatics database has also been created to foster knowledge exchange.

AI-driven Robotic Lab↗
DOE OSTI2021

EpitopeVec: linear epitope prediction using deep protein sequence embeddings

Motivation: B-cell epitopes (BCEs) play a pivotal role in the development of peptide vaccines, immuno-diagnostic reagents and antibody production, and thus in infectious disease prevention and diagnostics in general. Experimental methods used to determine BCEs are costly and time-consuming. Therefore, it is essential to develop computational methods for the rapid identification of BCEs. Although several computational methods have been developed for this task, generalizability is still a major concern, where cross-testing of the classifiers trained and tested on different datasets has revealed accuracies of 51–53%. Results: We describe a new method called EpitopeVec, which uses a combination of residue properties, modified antigenicity scales, and protein language model-based representations (protein vectors) as features of peptides for linear BCE predictions. Extensive benchmarking of EpitopeVec and other state-of-the-art methods for linear BCE prediction on several large and small datasets, as well as cross-testing, demonstrated an improvement in the performance of EpitopeVec over other methods in terms of accuracy and area under the curve. As the predictive performance depended on the species origin of the respective antigens (viral, bacterial and eukaryotic), we also trained our method on a large viral dataset to create a dedicated linear viral BCE predictor with improved cross-testing performance.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

N2A

N2A includes an object-oriented language for modeling neural systems as well as a software tool for editing and simulating models. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-2695 O

Rothganger, Frederick↗
DOE OSTI2023

pnnl/projection_ntk

This is a repository for work conducted in our research paper we would like to make open source. The main features are that we forked another open source library and added functionality that enabled us to compute neural-kernel objects using random vector projection software for both computer vision and large language text models. The new features are specifically code that enables the calculation of objects we define and refer to as the projection pseudo neural tangent kernel and the projection trace neural tangent kernel; both of which are approximations of the empirical neural tangent kernel. The software that computes the projections themselves are not part of the invention, but are necessarily tied to our software and licensed via MIT license, and BSD 2 license.

Georgiev, Kristian↗
DOE OSTI2024

pnnl/chemreasoner

ChemReasoner - Catalyst Discovery via Large Language Model-driven Reasoning

Choudhury, Sutanay↗