SEARCH · Engineering Papers

Results for “Language”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2014

Compiler verification meets cross-language linking via data abstraction

Many real programs are written in multiple different programming languages, and supporting this pattern creates challenges for formal compiler verification. We describe our Coq verification of a compiler for a high-level language, such that the compiler correctness theorem allows us to derive partial-correctness Hoare-logic theorems for programs built by linking the assembly code output by our compiler and assembly code produced by other means. Our compiler supports such tricky features as storable cross-language function pointers, without giving up the usual benefits of being able to verify different compiler phases (including, in our case, two classic optimizations) independently. Here, the key technical innovation is a mixed operational and axiomatic semantics for the source language, with a built-in notion of abstract data types, such that compiled code interfaces with other languages only through axiomatically specified methods that mutate encapsulated private data, represented in whatever formats are most natural for those languages.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Predicting Foreign Language Usage from English-Only Social Media Posts

Social media is known for its multicultural and multilingual interactions, a natural product of which is code-mixing. Multilingual speakers mix languages they tweet to address a different audience, express certain feelings, or attract attention. This paper presents a large-scale analysis of 6 million tweets produced by 23 thousand multilingual users speaking 11 other languages besides English. We rely on this multilingual corpus to build predictive models for a novel task – inferring non- English languages that users speak exclusively from their English tweets. We contrast the predictive power of different linguistic signals and report that lexical content and syntactic structure of English tweets are the most predictive of non-English languages that users speak on Twitter. By analyzing cross-lingual transfer – the influence of non-English languages on various levels of linguistic performance in English, we present novel findings on stylistic and syntactic variations across speakers of 11 languages.

code-swicthing↗

DOE OSTI2020

Data-driven materials research enabled by natural language processing and information extraction

Given the emergence of data science and machine learning throughout all aspects of society, but particularly in the scientific domain, there is increased importance placed on obtaining data. Data in materials science are particularly heterogeneous, based on the significant range in materials classes that are explored and the variety of materials properties that are of interest. This leads to data that range many orders of magnitude, and these data may manifest as numerical text or image-based information, which requires quantitative interpretation. The ability to automatically consume and codify the scientific literature across domains - enabled by techniques adapted from the field of natural language processing - therefore has immense potential to unlock and generate the rich datasets necessary for data science and machine learning. This review focuses on the progress and practices of natural language processing and text mining of materials science literature and highlights opportunities for extracting additional information beyond text contained in figures and tables in articles. Here, we discuss and provide examples for several reasons for the pursuit of natural language processing for materials, including data compilation, hypothesis development, and understanding the trends within and across fields. Current and emerging natural language processing methods along with their applications to materials science are detailed. We, then, discuss natural language processing and data challenges within the materials science domain where future directions may prove valuable.

36 MATERIALS SCIENCE↗

DOE OSTI2017

Kinship structures create persistent channels for language transmission

Here, languages are transmitted through channels created by kinship systems. Given sufficient time, these kinship channels can change the genetic and linguistic structure of populations. In traditional societies of eastern Indonesia, finely resolved cophylogenies of languages and genes reveal persistent movements between stable speech communities facilitated by kinship rules. When multiple languages are present in a region and postmarital residence rules encourage sustained directional movement between speech communities, then languages should be channeled along uniparental lines. We find strong evidence for this pattern in 982 individuals from 25 villages on two adjacent islands, where different kinship rules have been followed. Core groups of close relatives have stayed together for generations, while remaining in contact with, and marrying into, surrounding groups. Over time, these kinship systems shaped their gene and language phylogenies: Consistently following a postmarital residence rule turned social communities into speech communities.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2023

Adaptive language model training for molecular design

Abstract The vast size of chemical space necessitates computational approaches to automate and accelerate the design of molecular sequences to guide experimental efforts for drug discovery. Genetic algorithms provide a useful framework to incrementally generate molecules by applying mutations to known chemical structures. Recently, masked language models have been applied to automate the mutation process by leveraging large compound libraries to learn commonly occurring chemical sequences (i.e., using tokenization) and predict rearrangements (i.e., using mask prediction). Here, we consider how language models can be adapted to improve molecule generation for different optimization tasks. We use two different generation strategies for comparison, fixed and adaptive. The fixed strategy uses a pre-trained model to generate mutations; the adaptive strategy trains the language model on each new generation of molecules selected for target properties during optimization. Our results show that the adaptive strategy allows the language model to more closely fit the distribution of molecules in the population. Therefore, for enhanced fitness optimization, we suggest the use of the fixed strategy during an initial phase followed by the use of the adaptive strategy. We demonstrate the impact of adaptive training by searching for molecules that optimize both heuristic metrics, drug-likeness and synthesizability, as well as predicted protein binding affinity from a surrogate model. Our results show that the adaptive strategy provides a significant improvement in fitness optimization compared to the fixed pre-trained model, empowering the application of language models to molecular design tasks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Language Barriers in Organismal Biology: What Can Journals Do Better?

Synopsis In the field of organismal biology, as in much of academia, there is a strong incentive to publish in internationally recognized, highly regarded, English-language journals to promote career advancement. This expectation has created a linguistic hegemony in scientific publishing, whereby scholars for whom English is an additional language face additional barriers to achieving the same scientific recognition as scholars who speak English as a first language. Here, we surveyed the author guidelines of 230 journals in organismal biology with impact factors of 1.5 or greater for linguistically inclusive and equitable practices and policies. We looked for efforts that reflect first steps toward reducing barriers to publication for authors globally, including the presence of statements that encouraged submissions from authors of diverse nationalities and backgrounds, policies regarding manuscript rejection based on perceived inadequacies of the English language, the existence of bias-conscious reviewer practices, whether translation and editing resources or services are available, allowance for non-English abstracts, summaries, or translations, and whether journals offer license options that would permit authors (or other scholars) to translate their work and publish it elsewhere. We also directly contacted a subset of journals to verify whether the information on their author guidelines page accurately reflects their policies and the accommodations they would make. We reveal that journals and publishers have made little progress toward beginning to recognize or reduce language barriers. Counter to our predictions, journals associated with scientific societies did not appear to have more inclusive policies compared to non-society journals. Many policies lacked transparency and clarity, which can generate uncertainty, result in avoidable manuscript rejections, and necessitate additional time and effort from both prospective authors and journal editors. We highlight examples of equitable policies and summarize actions that journals can take to begin to alleviate barriers to scientific publishing.

Nolde-Lopez, B.↗

DOE OSTI2020

Analysis Description Languages for the LHC

An analysis description language is a domain specific language capable of describing the contents of an LHC analysis in a standard and unambiguous way, independent of any computing framework. It is designed for use by anyone with an interest in, and knowledge of, LHC physics, i.e., experimentalists, phenomenologists and other enthusiasts. Adopting analysis description languages would bring numerous benefits for the LHC experimental and phenomenological communities ranging from analysis preservation beyond the lifetimes of experiments or analysis software to facilitating the abstraction, design, visualization, validation, combination, reproduction, interpretation and overall communication of the analysis contents. Here, we introduce the analysis description language concept and summarize the current efforts ongoing to develop such languages and tools to use them in LHC analyses.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2020

QCOR; A Language Extension Specification for the Heterogeneous Quantum-Classical Model of Computation

Quantum computing (QC) is an emerging computational paradigm that leverages the laws of quantum mechanics to perform elementary logic operations. Existing programming models for QC were designed with fault-tolerant hardware in mind, envisioning stand-alone applications. However, the susceptibility of near-term quantum computers to noise limits their stand-alone utility. To better leverage limited computational strengths of noisy quantum devices, hybrid algorithms have been suggested whereby quantum computers are used in tandem with their classical counterparts in a heterogeneous fashion. This modus operandi calls out for a programming model and a high-level programming language that natively and seamlessly supports heterogeneous quantum-classical hardware architectures in a single-source-code paradigm. Motivated by the lack of such a model, we introduce a language extension specification, called QCOR, which enables single-source quantum-classical programming. Programs written using the QCOR library–based language extensions can be compiled to produce functional hybrid binary executables. After defining QCOR’s programming model, memory model, and execution model, we discuss how QCOR enables variational, iterative, and feed-forward QC. Additionally, QCOR approaches quantum-classical computation in a hardware-agnostic heterogeneous fashion and strives to build on best practices of high-performance computing. The high level of abstraction in the language extension is intended to accelerate the adoption of QC by researchers familiar with classical high-performance computing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Domain Specific Language Support for Exascale Application Development

The University of Oregon has produced a Fortran parser based on an SDF (Syntax Definition Formalism) grammar. Primary partners in this effort were Rice University and Lawrence Livermore National Laboratory. The developed Fortran parser incorporates the full Fortran 2008 standard with extensions from the more recent document, TS 29113, “Further Interoperability of Fortran with C,” (to be included in the Fortran 2018 standard). In the final year of the project, the Fortran parser was updated to comply with the complete 2018 standard, as specified in the ISO/IEC document, JTC 1/SC 22/WG 5/N2146. The research explored in this project provided the expression of the Fortran grammar in SDF form for integration with the Rosebud DSL framework developed by Rice University. The Rosebud framework is a cohesive interface allowing modular DSL plugins to be developed independently and then distributed and combined in HPC applications. These plugins implement a form of generalized DSL which includes both concrete languages used to write application source code and abstract languages used within the compilation process. ROSE’s predefined DSL Building Blocks (for example the Fortran base language) are represented as abstract language plugins and DSL developers can use Rosebud to construct additional building blocks in the same way.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Management of Digital Equipment in Preproduction by Means of Uniform Numeric Control Language

The requirements on preproduction and the control software for various digital machining systems (numerically controlled machine tools, robot manipulators, additive manufacturing equipment) in flexible production are considered. By analysis of existing programming languages for digital equipment, UNCL (Uniform Numeric Control Language) has been developed for standard representation of control commands. The use of UNCL in verifying program operation on a machining simulator is considered. Software for the translation of control programs (specified in CLDATA formats and in ISO code) to the invariant format of the universal language is described. The postprocessing of control software from the universal language into programs in commonly used digital control systems is also considered.

42 ENGINEERING↗

DOE OSTI2017

Identifying Effective Signals to Predict Deleted and Suspended Accounts on Twitter across Languages

Social networks have an ephemerality to them where accounts and messages are constantly being edited, deleted, or marked as private. This continuous change comes from concerns around privacy, a potential desire for deception, and spam-like behavior. In this study we analyze multiple large datasets of thousands of active and deleted Twitter accounts to produce a series of predictive features for the removal or shutdown of an account. We have selected these accounts from speakers of three languages -- Russian, Spanish, and English to evaluate if speakers of various languages behave differently with regards to deleting accounts. We find that unlike previously used profile and network features, the discourse of deleted vs. active accounts forms the basis for highly accurate account deletion prediction. More precisely, we observed that the presence of a certain set of terms in user tweets leads to a higher likelihood for that user's account deletion. We show that the predictive power of profile, language, affect, and network features is not consistent across speakers of the three evaluated languages.

account deletion prediction↗

DOE OSTI2017

Identifying Effective Signals to Predict Deleted and Suspended Accounts on Twitter across Languages

social media, machine learning, natural language p↗

DOE OSTI2026

SmileyLlama: modifying large language models for directed chemical space exploration

Here we show that large language models (LLMs) can be transformed via supervised fine-tuning of engineered prompts into SmileyLlama for exploring the chemical space of drug molecules. We benchmark SmileyLlama against pretrained LLMs and chemical language models trained from scratch for generating valid and novel drug-like molecules, and use direct preference optimization to both improve SmileyLlama’s adherence to a prompt and as part of the iMiner reinforcement learning framework to predict molecules with optimized three-dimensional conformations and high binding affinity to drug targets. By training an LLM to speak directly as a chemical language model, while retaining most of its natural language capabilities, we show that SmileyLlama can reliably generate molecules with user-specified properties rather than acting only as a chatbot with knowledge of chemistry or as a virtual assistant. While SmileyLlama is geared toward drug discovery, the supervised fine-tuning/direct preference optimization/LLM framework can be extended to other chemical, biological and materials applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Decoding substance use disorder severity from clinical notes using a large language model

Substance use disorder (SUD) poses a major concern due to its detrimental effects on health and society. SUD identification and treatment depend on a variety of factors such as severity, co-determinants (e.g., withdrawal symptoms), and social determinants of health. Existing diagnostic coding systems used by insurance providers, like the International Classification of Diseases (ICD-10), lack granularity for certain diagnoses, but American clinicians will add this granularity (as that found within the Diagnostic and Statistical Manual of Mental Disorders classification or DSM-5) as supplemental unstructured text in clinical notes. Traditional natural language processing (NLP) methods face limitations in accurately parsing such diverse clinical language. Large language models (LLMs) offer promise in overcoming these challenges by adapting to diverse language patterns. This study investigates the application of LLMs for extracting severity-related information for various SUD diagnoses from clinical notes. We propose a workflow employing zero-shot learning of LLMs with carefully crafted prompts and post-processing techniques. Through experimentation with Flan-T5, an open-source LLM, we demonstrate its superior recall compared to the rule-based approach. Focusing on 11 categories of SUD diagnoses, we show the effectiveness of LLMs in extracting severity information, contributing to improved risk assessment and treatment planning for SUD patients.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2018

FLPR: The Fortran Language Program Remodeling system

The Fortran Language Program Remodeling system (FLPR) is a C++ library designed to manipulate Fortran language source code files. A client can use FLPR to rapidly develop utilities that perform application-specific source code transformations. Examples of transformations are varied, ranging from simple tasks such as reformatting source code, to complex transformations that implement application-specific extensions to the Fortran language. Two unique features of FLPR are that it allows the client to extend the language that it recognizes, and that it allows the client to manipulate all of the text in a source code file, including formatting and comments.

henning, Robert↗

DOE OSTI2025

Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models

Semantic schemas like Haystack 4, Brick and ASHRAE standard 223 enable the structured, standardized, and machine-readable representation of building data, facilitating interoperability, data integration, and advanced analytics. However, extracting information from these models requires specialized expertise in SPARQL and other programming languages, skills that are not commonly found among building professionals. Recent advancements in Large Language Models (LLMs), such as ChatGPT, enable the construction of queries using natural language, making it easier for individuals to interact with these systems in a manner that resembles everyday speech. However, these methods have not yet been tested on building semantic ontologies. This paper introduces a novel workflow and tool for enabling users to ask questions about a specific building's data, using natural language and receive answers automatically generated by GPT-4o. Our approach integrates semantic ontologies with advanced LLM capabilities to automate three critical steps: (1) generating SPARQL queries to retrieve time series references from ontological models, (2) extracting the corresponding time series data from the Building Automation System, and (3) performing computations and visualizations tailored to the user's query. The proposed method simplifies access to BAS data, allowing both domain experts and non-specialists to conduct sophisticated analyses without needing extensive technical knowledge of semantic web technologies. By demonstrating this pipeline, we facilitate more accessible and scalable data-driven decision-making in building operations and management.

Mulayim, Ozan Baris↗

DOE OSTI2021

An Introduction to Word Embeddings and Language Models

Language models have advanced at a phenomenal pace over the past decade. This document provides a short introduction to terminology, word embeddings (aka low-dimensional representations), and popular large-scale language models (LMs). Word embeddings are used to represent words as numerical vectors and are context-independent, meaning a word can only have a single representation (e.g., club can only be club sandwich, not golf club ). Language models can determine the probability of a given sequence of words occurring in a sentence and can provide context to distinguish between words and phrases that sound similar. LMs are context-dependent (e.g., club can be club sandwich or golf club ) and largely fall in two main classes – autoregressive and autoencoding models. Autoregressive models are pretrained on the classic language modeling task: guess the next token having read all the previous ones. Those models can be fine-tuned and achieve great results on many tasks, the most natural application is text generation. A typical example of such models is GPT, but others include GPT-2, GPT-3, CTLR, TRANSFORMER-XL, REFORMER, XLNET. Autoencoding models are pretrained by corrupting the input tokens in some way and trying to reconstruct the original sentence. They can be fine-tuned and achieve great results on many tasks such as text generation, but their most natural application is sentence classification or token classification. A typical example of such models is BERT, but others include ROBERTA, ALBERT, XML, XML-ROBERTA, FLAUBERT AND LONGFORMER.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Predictive Indicators of the Performance of Large Language Models

In several mission contexts, it is desirable to estimate the performance of large language models (LLMs) on tasks that we cannot run directly. In light of published “scaling laws” our hypothesis is that some tasks should be consistently more challenging than others based on characteristics of the task. The goal of this project was to begin quantifying how much information about LLM performance can be gained from the features of a model and a task. Two of our statistical models struggled to converge. Pass/fail test results may provide limited information for inference beyond model quality and task difficulty, but we see no evidence at this time for significant feature interaction effect sizes, arguing for simple models. Future work extending the models to capitalize on perplexity of ground truth answers is suggested. This project also introduces “Depth of Knowledge Variant Testing” as a strategy for more finely assessing language models on open domain question and answer tasks. We developed sets of questions that ask a language model to produce similar information while demonstrating increasing depth of knowledge, and also relabeled existing Q&A test questions with their depth of knowledge. Our results suggest further consideration of Bloom’s taxonomy and further refinement of prompts to properly elicit information at varying depths. In the course of this work, we set up a basic infrastructure for standardizing tasks and testing many language models on these tasks. In addition to testing the predictive quality of model features and performance across test suites, with this project we have introduced two new task features to contextualize each test question: the Dewey Classification main category of information covered, and the Bloom’s taxonomy level that corresponds to the depth of knowledge probed by the question. Splits across these and other features produced over five hundred task subtypes with distinct feature vectors, which we tested on half a dozen models.

97 MATHEMATICS AND COMPUTING↗