Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Language”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Autogenerating a Domain-Specific Question-Answering Data Set from a Thermoelectric Materials Database to Enable High-Performing BERT Models

We present a method for autogenerating a large domain-specific question-answering (QA) dataset from a thermoelectric materials database. We show that a small language model, BERT, once fine-tuned on this automatically generated dataset of 99,757 QA pairs about thermoelectric materials, affords better performance in the field of thermoelectric materials compared to a BERT model fine-tuned on the generic English-language QA data set, SQuAD-v2. We further show that mixing the two data sets (ours and SQuAD-v2), which have significantly different syntactic and semantic scopes, allows the BERT model to achieve even better performance. The best-performing BERT model fine-tuned on the mixed data set outperforms the models fine-tuned on the other two data sets by scoring an exact match of 67.93% and an F1 score of 72.29% when evaluated on our test data set. This has important implications as it demonstrates the ability to realize high-performing small language models, with modest computational resources, empowered by domain-specific materials data sets which can be generated according to our method.

biological databases↗
DOE OSTI2026

Bayesian Optimization of Catalysis with In-Context Learning

Large language models (LLMs) can perform accurate classification with zero or few examples through in-context learning (ICL), allowing the model to observe query-relevant examples at inference time and eliminating the need for additional weight updates to generalize beyond its original training data. We extend this capability to regression with uncertainty estimation using frozen LLMs (e.g., GPT-4o, Gemini), enabling Bayesian optimization (BO) in natural language without explicit model training or feature engineering. We apply this to materials discovery by representing materials as synthesis and testing procedures for use in natural language prompts. This Bayesian, design-first approach prioritizes optimization toward target material properties before detailed characterization, in contrast to conventional experimental workflows that often emphasize characterization of suboptimal materials. On benchmarks like aqueous solubility and oxidative coupling of methane (OCM), BO-ICL matches or outperforms Gaussian processes. In live experiments on the reverse water–gas shift (RWGS) reaction, BO-ICL identifies multimetallic catalysts that approach equilibrium CO yield within 6 and 10 iterations from a pool of 3,700 and 360,000 candidates, respectively. Our method redefines materials representation and accelerates discovery, with broad applications across catalysis, materials science, and AI.

Calibration↗
DOE OSTI2026

ToPolyAgent: AI agents for coarse-grained bead-spring topological polymer simulations

We introduce ToPolyAgent, a multi-agent AI framework for performing coarse-grained molecular dynamics (MD) simulations of topological polymers through natural language instructions. By integrating large language models (LLMs) with domain-specific computational tools, ToPolyAgent supports both interactive and autonomous simulation workflows across diverse polymer architectures, including linear, ring, brush, and star polymers, as well as dendrimers. The system consists of four LLM-powered agents: a Config Agent for generating initial polymer–solvent configurations, a Simulation Agent for executing LAMMPS-based MD simulations and conformational analyses, a Report Agent for compiling markdown reports, and a Workflow Agent for streamlined autonomous operations. Interactive mode incorporates user feedback loops for iterative refinements, while autonomous mode enables end-to-end task execution from detailed prompts. We demonstrate ToPolyAgent's versatility through case studies involving diverse polymer architectures under varying solvent conditions, thermostats, and simulation lengths. Furthermore, we highlight its potential as a research assistant by directing it to investigate the effect of interaction parameters on the linear polymer conformation, and the influence of grafting density on the persistence length of the brush polymer. By coupling natural language interfaces with rigorous simulation tools, ToPolyAgent lowers barriers to complex computational workflows and advances AI-driven materials discovery in polymer science. It lays the foundation for autonomous and extensible multi-agent scientific research ecosystems.

Ding, Lijie [Oak Ridge National Laboratory (ORNL),↗
DOE OSTI2018

Automated Approach for Construction of Long-Term, Data-Intensive Watershed Models

Watershed models such as the Hydrological Simulation Program in FORTRAN (HSPF) are frequently used to analyze large-scale water quantity and water quality issues. The construction of HSPF models is a difficult and time-consuming process, because it requires compilation of extensive quantities of data into formatted files followed by a simultaneous fitting of many uncertain parameters. Because of these complications, studies are often limited to calibration periods of only a few years even when analyzing long time-scale issues such as climate and land-use changes that are difficult or impossible to reproduce. High-level, open source programming languages provide an environment for automating the extraction and processing of various sources of data in addition to the calibration process required by HSPF. Recently developed software can be used to build HSPF input files, run simulations, and postprocess simulation results using the Python programming language. The combination of tools in Python, public data sets on the Internet, and software extensions enables rapid development of long-term, reproducible, and sophisticated models for new hydrologic insight. Herein, the utility of these tools is illustrated by developing an automated 30-year HSPF model with a Nash-Sutcliffe efficiency of 0.88 for monthly flows and 0.75 for daily flows in a simulation time of approximately 3h. The integration of HSPF with a high-level programming language creates opportunities to more rigorously explore model assumptions, calibration methods, and alternative data sets.

Lampert, Dave J↗
DOE OSTI2016

Neural speech recognition: continuous phoneme decoding using spatiotemporal representations of human cortical activity

The superior temporal gyrus (STG) and neighboring brain regions play a key role in human language processing. Previous studies have attempted to reconstruct speech information from brain activity in the STG, but few of them incorporate the probabilistic framework and engineering methodology used in modern speech recognition systems. Here, in this work, we describe the initial efforts toward the design of a neural speech recognition (NSR) system that performs continuous phoneme recognition on English stimuli with arbitrary vocabulary sizes using the high gamma band power of local field potentials in the STG and neighboring cortical areas obtained via electrocorticography. The system implements a Viterbi decoder that incorporates phoneme likelihood estimates from a linear discriminant analysis model and transition probabilities from an n-gram phonemic language model. Grid searches were used in an attempt to determine optimal parameterizations of the feature vectors and Viterbi decoder. The performance of the system was significantly improved by using spatiotemporal representations of the neural activity (as opposed to purely spatial representations) and by including language modeling and Viterbi decoding in the NSR system. These results emphasize the importance of modeling the temporal dynamics of neural responses when analyzing their variations with respect to varying stimuli and demonstrate that speech recognition techniques can be successfully leveraged when decoding speech from neural signals. Guided by the results detailed in this work, further development of the NSR system could have applications in the fields of automatic speech recognition and neural prosthetics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

Abstract Motivation Millions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. Results We developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementation The source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

VWC-BERT: Scaling Vulnerability–Weakness–Exploit Mapping on Modern AI Accelerators

Defending cybersystems needs accurate mapping of software and hardware vulnerabilities to generalized descriptions of weaknesses, and weaknesses to exploits. These mappings enable cyber defenders to build plans for effective defense and assessment of potential risks to a cybersystem. With close to 170k vulnerabilities, manual mapping is not a feasible option. However, automated mapping is challenging due to limited training data, computational intractability, and limitations in computational natural language processing. Tools based on breakthroughs in Transformer-based language models have been demonstrated to classify vulnerabilities with high accuracy. We make three key contributions in this paper: (1) We present a new framework, \VWCBERT, that augments the Transformer-based hierarchical multi-class classification framework of Das et al. (\textsc{V2W-BERT}) with the ability to map weaknesses to exploits. (2) We implement \VWCBERT~ on modern AI accelerator platforms using two data parallel techniques for the pre-training phase and demonstrate nearly linear speedups across NVIDIA and Graphcore accelerator platforms. We observe nearly linear speedups for up to 16 V100 and 8 A100 GPUs, and about 3.4$\times$ speedup for A100 relative to V100 GPUs. We also observe excellent speedups on Graphcore, with $5.7\times$ speedup on 128 IPUs relative to 16 IPUs. Enabled by scaling, we also demonstrate higher accuracy using a larger language model, RoBERTa-Large. We show up to 87\% accuracy for strict and up to 98\% accuracy for relaxed classification. (3) We develop a novel parallel link manager for the link prediction phase and demonstrate up to 21$\times$ speedup with 16 V100 GPUs relative to one V100 GPU, and thus reducing the runtime from 2.5 hours to 10 minutes. We believe that generalizability and scalability of \VWCBERT~ will benefit both the theoretical development and practical deployment of novel cyberdefense solutions and vulnerability classification.

Das, Siddhartha Shankar↗
DOE OSTI2024

IRIS: Exploring Performance Scaling of the Intelligent Runtime System and its Dynamic Scheduling Policies

High-Performance Computing is becoming increasingly heterogeneous, relying on a diverse mix of hardware to achieve good performance. Paradoxically, current drivers and frameworks for these devices typically require separate languages and implementations for each vendor. Furthermore, there are few tools and little support to schedule codes between these devices in a truly heterogeneous manner-partly because of this fragmentation between vendors and the languages each supports. To overcome both limitations, the Intelligent Runtime System (IRIS) was developed. It allows a common task abstraction to automatically be shared among contemporary vendors and is run from a single host-side API. At runtime, IRIS queries the host system and registers which frameworks and drivers are available, these determine which kernels can be used by the scheduler-CPUs via OpenMP, Nvidia GPUs (CUDA), AMD GPUs (HIP), and Intel and Xilinx FPGAs with OpenCL. IRIS enables tasks to be scheduled to any heterogeneous device and resolves to the appropriate kernel binary at runtimeit only uses the devices supported by the system on which it is run. IRIS supports single-task and graph-based expressions of dependencies of tasks. Additionally, IRIS features a range of dynamic scheduling policies, allowing complex chains of tasks and interactions to be executed, relieving the programmer/user from considering the system to assign tasks to devices optimally. This paper presents the peak performance attainable by IRIS over a range of systems-each with different numbers and types of accelerator devices, it highlights the flexibility of IRIS since these devices are truly heterogeneous, relying on different backends (drivers, frameworks, and languages) which historically required unique implementations to utilize them. We then use this peak performance as a baseline to compare increasingly complex chains of tasks (with increasingly complex task dependencies) and evaluate how IRIS copes. Finally, we consider the performance of different IRIS scheduling policies on this range of task graphs.

Johnston, Beau↗
DOE OSTI2026

Dual Context: Leveraging Structured Application Context for Code Generation and Runtime Feature Activation via Chat Interfaces

Integrating artificial intelligence (AI) capabilities into software applications typically involves two common paths. For developers, AI assists in generating and documenting source code and other related software engineering efforts. For users, AI assists them through question-and-answer exchanges via chatbots. Both approaches have their value, but neither effectively leverages the modularity of component-based architectures that modern web application frameworks offer. We implement a proof of concept within a centralized suite of applications used for the Atmospheric Radiation Measurement (ARM) Data Center Operational Tools, where we introduce a third integration path through the ARM Context Engine (ACE). ACE is a context driven system that uses structured contextual specifications to enable Large Language Models (LLMs) to render interactive and feature-rich user interface (UI) components directly within chat responses, alongside or in place of conventional text outputs. These specifications serve two important purposes across what we call code context and UI context. Code context provides AI-assisted development tools with structured application knowledge beyond raw code, including component relationships, architectural patterns and schematic information, enabling the generation of consistent, well-structured code. UI context defines the rules for enabling and rendering component features at runtime based on the user's natural language input, allowing end users to activate capabilities such as data export, filtering, and pagination within chat responses, without requiring code changes or redeployment. We demonstrate, through a comparative evaluation against general-purpose AI chatbots, that context-driven component rendering provides interactive capabilities that text-based responses cannot replicate, including deterministic component behavior, application-consistent design language, and on-demand feature activation. A development effort comparison further shows that features that traditionally require multi-step development cycles can be activated with a single naturallanguage request. In this ongoing work, we present ACE as an emerging approach to AI integration that positions modular, well-documented software architecture as the foundation for AI-ready applications. ACE treats context as a shared resource across both development and user-facing AI, bringing cohesion to conventionally disconnected efforts, bridging developer tooling and end-user capabilities within a single framework.

Tadimeti, Vijay [ORNL]↗
DOE OSTI2015

NSTX-U Advances in Real-Time C++11 on Linux

Programming languages like C and Ada combined with proprietary embedded operating systems have dominated the real-time application space for decades. The new C++11standard includes native, language-level support for concurrency, a required feature for any nontrivial event-oriented real-time software. Threads, Locks, and Atomics now exist to provide the necessary tools to build the structures that make up the foundation of a complex real-time system. The National Spherical Torus Experiment Upgrade (NSTX-U) at the Princeton Plasma Physics Laboratory (PPPL) is breaking new ground with the language as applied to the needs of fusion devices. A new Digital Coil Protection System (DCPS) will serve as the main protection mechanism for the magnetic coils, and it is written entirely in C++11 running on Concurrent Computer Corporation's real-time operating system, RedHawk Linux. It runs over 600 algorithms in a 5 kHz control loop that determine whether or not to shut down operations before physical damage occurs. To accomplish this, NSTX-U engineers developed software tools that do not currently exist elsewhere, including real-time atomic synchronization, real-time containers, and a real-time logging framework. Together with a recent (and carefully configured) version of the GCC compiler, these tools enable data acquisition, processing, and output using a conventional operating system to meet a hard real-time deadline (that is, missing one periodic is a failure) of 200 microseconds.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2016

Distributed Halide

Many image processing tasks are naturally expressed as a pipeline of small computational kernels known as stencils. Halide is a popular domain-specific language and compiler designed to implement image processing algorithms. Halide uses simple language constructs to express what to compute and a separate scheduling co-language for expressing when and where to perform the computation. This approach has demonstrated performance comparable to or better than hand-optimized code. Until now, however, Halide has been restricted to parallel shared memory execution, limiting its performance for memory-bandwidth-bound pipelines or large-scale image processing tasks. We present an extension to Halide to support distributed-memory parallel execution of complex stencil pipelines. These extensions compose with the existing scheduling constructs in Halide, allowing expression of complex computation and communication strategies. Existing Halide applications can be distributed with minimal changes, allowing programmers to explore the tradeoff between recomputation and communication with little effort. Approximately 10 new of lines code are needed even for a 200 line, 99 stage application. On nine image processing benchmarks, our extensions give up to a 1.4× speedup on a single node over regular multithreaded execution with the same number of cores, by mitigating the effects of non-uniform memory access. The distributed benchmarks achieve up to 18× speedup on a 16 node testing machine and up to 57× speedup on 64 nodes of the NERSC Cori supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources

Apache Calcite is a foundational software framework that provides query processing, optimization, and query language support to many popular open-source data processing systems such as Apache Hive, Apache Storm, Apache Flink, Druid, and MapD. The goal of this paper is to formally introduce Calcite to the broader research community, brie y present its history, and describe its architecture, features, functionality, and patterns for adoption. Calcite's architecture consists of a modular and extensible query optimizer with hundreds of built-in optimization rules, a query processor capable of processing a variety of query languages, an adapter architecture designed for extensibility, and support for heterogeneous data models and stores (relational, semi-structured, streaming, and geospatial). This exible, embeddable, and extensible architecture is what makes Calcite an attractive choice for adoption in big-data frameworks. It is an active project that continues to introduce support for the new types of data sources, query languages, and approaches to query processing and optimization.

Begoli, Edmon↗
DOE OSTI2023

Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation

We evaluate AI-assisted generative capabilities on fundamental numerical kernels in high-performance computing (HPC), including AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG. We test the generated kernel codes for a variety of language-supported programming models, including (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). We use the GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code as of April 2023 to generate a vast amount of implementations given simple + + prompt variants. To quantify and compare the results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. Results suggest that the OpenAI Codex outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general purpose Python can benefit from adding code keywords, while Julia prompts perform acceptably well for its mature programming models (e.g., Threads and CUDA.jl). We expect for these benchmarks to provide a point of reference for each programming model's community. Overall, understanding the convergence of large language models, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

Godoy, William↗
DOE OSTI2016

Arion

Arion is a library and tool set that enables researchers to holistically define test system models. To define a complex system for testing an algorithm or control requires expertise across multiple domains. Simulating a complex system requires the integration of multiple simulators and test hardware, each with their own specification languages and concepts. This requires extensive set of knowledge and capabilities. Arion was developed to alleviate this challenge. Arion is a library of Java libraries that abstracts the concepts from supported simulators into a cohesive model language that allows someone to build models to their needed level of fidelity and expertise. Arion is also a software tool that translates the users model back into the specification languages of the simulators and test hardware needed for execution.

Fisher, Andy↗
DOE OSTI2019

statusMessageReporting

Unlike computer languages like C++ which has throw/catch, Python which has try/except and other newer computer languages, C has no built-in mechanism for handling exceptions (e.g., memory allocate failure or a divide by zero exceptions). While simple exception handling in the form of an integer parameter (e.g, errno) or an enum are often implemented in C programs, the richer types of exception reporting possible with newer computer languages are rarely implemented in C. The statusMessageReporting package was created to facilitate rich exception reporting by C functions to their calling functions. The statusMessageReporting (smr) package allows a function, called the generator, to communicate an exception to its calling function using a statusMessageReporting report. The main parts of a report are: 1)the level of the report called the status (valid levels are smr_status_Ok, smr_status_Info, smr_statusWarning and smr_status_Error) 2) a string called the report's message and 3) two integer values, one representing the generator's ID, called the libraryID, and the other is the library's specific integer, called the code, that represents the type of exception the generator is reporting. The code is similar to the errno for the C standard library. Since libraries are often developed independent of one another, they often use the same code values to represent different exceptions (e.g., one library may define the code 3 to be an error opening a file while another library may define the code 3 to be a divide-by-zero error) which is why the libraryID is needed. That is, the equivalent of errno is the combination libraryID:code. To insure that each library's libraryID is unique, libraryIDs must be obtained from the function smr_registerLibrary.

Beck, Bret↗
DOE OSTI2022

WORM (Write One, Read Many)

WORM (Write One, Run Many) is an easy to use, cross platform, embedded and extensible, functional programming language designed to facilitate the creation of input-decks for computer codes that use standard ASCII text files for input. WORM makes it easy to create generic (yet, complex and powerful) reusable models. Additionally its nature allows for complex calculations and routines to be coded once and easily reused, further simplifying the creation of input decks. WORM (Write One, Run Many) is a powerful and versatile tool designed to improve the efficiency of today’s criticality safety analyst by allowing: + input decks for parametric studies to be created quickly and easily, + calculations and variables to be imbedded into any input deck, thus allowing for meaningful parameter specifications, + problems to be specified using any combination of units, and + complex mathematically defined models to be created. A very simple syntax is employed, and therefore the WORM is easy to learn. A WORM model is essentially a standard input deck with some of its numerical values replaced by WORM code. WORM code may include and evaluate the following mathematical operators and functions: addition, subtraction, multiplication, division, exponentiation, modulus, sine, cosine, tangent, arcsine, arccosine, arctangent, the natural logarithm, logarithm base 10, integer truncation, absolute value, and random number. Several common constants, e.g., pi, e, and Avogadros’s Number (both as 6.022e23 and 0.6022), are predefined in WORM. Additionally, many unit conversion factors are also predefined: millimeters, meters, inches, feet, yards, and mils to centimeters; kilograms, pounds, and ounces to grams; liters, milliliters, gallons, and fluid ounces to cubic centimeters; and angular degrees to radians. For parametric studies, WORM supports various shorthand list specifications: the explicit step size, linear interpolation, and logarithmic interpolation. The list notation sequentially assigns multiple values to a name. WORM creates an input deck for each value of the name. If multiple lists are used, WORM steps through each list individually, i.e., WORM creates input decks corresponding to each and every permutation of the list values. Additionally, a library of standard material definitions and Perl subroutines are included. Any one of these files can be incorporated into the subject model with a simple WORM read command. WORM is completely written in Perl, the Practical Extraction and Reporting Language. Perl is one of the most portable programming languages available today. As such, the WORM works on practically any computer platform.

Sartor, Raymond↗
DOE OSTI2023

ctrl-flow v0.0.1

ctrl-flow is a software that assists users in selecting a building control sequence from a library of control sequences that are implemented in the Modelica Buildings Library. The software allows selecting, configuring and editing the control sequences. It also allows exporting the sequence for use in building project design documents in Modelica-based simulations. The export will be in a json intermediate format, in the Modelica Language, in the Control Description Language and in English language format, such as in a Word document. The software functions as a web based application that includes a graphical user interface.

Wetter, Michael↗
DOE OSTI2022

Inference-Engine v0.1.0

Given a pre-trained neural network, Inference-Engine performs maps network inputs to outputs by executing the forward pass through the provided network. Although the predominant programming language for machine-learning is Python, most high-performance computing (HPC) applications are written in Fortran, C, or C++. Inference-Engine aims to support HPC programs and is written in Fortran, a language with a large feature set supporting interoperability with C. This software exposes concurrency in a portable way by using standard language features that some modern Fortran compilers can exploit with various optimizations, including offloading computation to a Graphics Processing Unit (GPU). In particular, this software makes extensive use of Fortran's "do concurrent" parallel loop construct, implicitly parallel array statements, and pure procedures that can be invoked inside "do concurrent" blocks. Inference-Engine also supports dynamic choice of inference methods at runtime. Two current options include one method that uses Fortran's "dot_product" intrinsic function inside "do concurrent" blocks and another method that instead uses Fortran' "matmul" array intrinsic function. We plan to investigate automatic compiler offloading of "do concurrent" calculations to GPUs and compile-time substitution of optimized libraries such as the Basic Linear Algebra Library (BLAS) for "matmul" invocations. We also envision the potential for the choice of which method to use could happen at program launch based on in situ performance measurements on any given platform.

Rouson, Damian↗