SEARCH · Engineering Papers

Results for “Natural Language Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2025

From Machine Learning to Machine Reasoning: A Model-based Approach to Analyze Equipment Reliability Data

In current nuclear power plants (NPPs) a large amount of condition-based data which can be used to assess and monitor component health and performance. Assessing component health from such data can be performed with a large variety of methods. While the analysis of numeric data can be performed with several methods, the extraction of information from textual data remains a challenge. Currently employed natural language processing (NLP) methods do not really provide quantitative information that might be contained in IRs. In addition, the integration of numeric and textual data to identify possible causal relationships between data elements is still an unresolved challenge. This paper presents an approach to extract information from textual (e.g., incident or maintenance reports) and numeric data that relies on model based system engineer (MBSE) models. MBSE are diagrams designed to represent system and component dependencies (from both a form and functional point of view). In our approach, MBSE models emulate system engineer knowledge about component/system architecture. NLP methods are employed to perform syntactic and semantic analyses. Syntactic analysis analyzes the grammatical structure of a sentence while semantic analysis is designed to analyze the logic structure of a sentence. An innovative element of our approach is that semantic analysis uses MBSE models to identify links between textual elements. Similarly, numeric data is directly linked to elements of the MBSE models in order to map which functions are being monitored.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Evaluating Economic Impact: An Investment Tool for Large Language Model Integration in Workweek Management

This paper explores the development and application of an investment tool designed to quantify the costs and potential savings associated with integrating large language models (LLMs) into work week management optimization (WMO) within the nuclear industry. LLMs, with their advanced natural language processing capabilities, can significantly enhance various aspects of work management, such as problem identification, prioritization, planning, scheduling, information retrieval, and information summary. Our investment tool focuses on evaluating the return on investment (ROI) for LLM applications in WMO by considering four pivotal decision factors: model selection, application, user training, and hosting options. This paper details the development and implementation of the ROI model and illustrates its application through multiple case studies, analyzing the impact of different variables, such as work time saved, number of requests, and model performance, on the computed ROI over two years. The computed ROI is also compared over different hosting solutions. Our findings indicate that ROI increases with enhanced work time savings and optimal request load but can decline with high request volumes or increased model costs. This model aids decision-makers in the nuclear industry by providing a structured approach to assessing the economic viability and potential savings from integrating LLMs into WMO processes.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2025

A Model Based Approach to Extract Health Information from Textual Data

In current nuclear power plants (NPPs) a large amount of condition-based data is being generated and stored to assess and monitor component health and performance. The format of this data can be either numeric (e.g., pump vibration data) or textual (e.g., condition report which assess component health). While assessing component health from numeric data can be performed with a large variety of methods, the extraction of information from textual data still remains a challenge. Natural language processing (NLP) methods are starting to be deployed in current NPPs mainly to filter out incident reports (IRs) that are not safety related by employing supervised machine learning methods. However, these methods do not really provide the quantitative information that might be contained in IRs. This paper presents an approach to extract information from textual data (e.g., from IRs, maintenance reports) that is based on NLP data analytics methods coupled with model-based system engineer (MBSE) models. NLP methods are employed to perform syntactic and semantic analyses. Syntactic analysis analyzes the grammatical structure of a sentence; such analysis includes: part of speech (POS) tagging (i.e., identification of grammatic elements of each string - e.g., nouns, verbs), named entity recognition (i.e., identification of text entities - e.g., names, dates, events), and relation extraction (e.g., coreference resolution). On the other hand, semantic analysis is designed to analyze the logic structure of a sentence. Through a specific set of rules, our methods can identify whether a sentence contains health information of a component (e.g., degraded performance, anomaly behavior) or the causal relationship between two events (i.e., a cause-effect pair). An innovative element of our approach is that semantic analysis relies on MBSE models to identify links between textual elements. MBSE are diagrams designed to represent system and component dependencies (from both a form and functional point of view). In our approach, MBSE models emulate system engineer knowledge about component/system architecture. This paper presents in detail how the integration of NLP methods and MBSE models is performed. Few analysis examples focusing on centrifugal pumps are presented.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Evaluating Economic Impact: An Investment Tool for Large Language Model Integration in Workweek Management

99 - GENERAL AND MISCELLANEOUS↗

DOE OSTI2026

Data Analytics Methods to Measure Plant Outage Resilience

Every 18 or 24 months nuclear power plants (depending on plant configuration, pressurized or boiling water reactor respectively) undergo a period of outage where the plant is taken offline and a large number of maintenance and surveillance activities (that cannot be performed while plant is running) are performed in typically 2–3 weeks. Planning of a plant outage is very challenging since all the activities are required to be performed in the shortest amount of time given available resources (typically contractor crews hired for the duration of the outage). Consequently, plant outages can be costly due the actual loss of power generation and crew costs and, because of it, there is a need to maximize resource usage in the outage planning phase and reduce the risk of outage delays. This paper is addressing these needs by providing a set of analytical methods designed to analyze plant outage schedule and identify critical elements based on available resources (time and crews). These methods are based on natural language processing and optimization algorithms. In this respect, two classes of methods have been developed: one that focuses on the time resource and how variability in the time to complete outage tasks may impact outage delays, and one that minimizes the risk of outage delays by integrating available resources to assess when daily activities should be performed.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Privacy-Aware RAG-Enabled LLMs for Collaborative AI in Organizations

Recent advancements in Large Language Models (LLMs) based on Transformer architectures have significantly improved capabilities in natural language processing and generation. However, deploying LLMs for inter-organizational communication poses challenges, in ensuring privacy and facilitating effective collaboration. This paper introduces a novel decentralized inference meta-agent chatbot that leverages privacy-aware Retrieval-Augmented Generation (RAG)-enabled LLMs for collaborative AI communication across organizations. Built on Microsoft’s Autogen, the platform enables LLMs to autonomously refine responses, enhancing accuracy and relevance. It incorporates advanced hallucination mitigation techniques using Uptrain and a privacy-focused RAG framework that employs synthetic document generation to protect sensitive information. Comprehensive evaluations demonstrate the platform’s effectiveness in maintaining contextual relevance and stringent privacy standards, effectively addressing critical challenges in LLM-enhanced collaborative AI communication. This work represents a significant step toward secure and efficient inter-organizational collaboration using advanced generative AI technologies.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Design of a Novel Information System for Semi-Automated Management of Cybersecurity in Industrial Control Systems

There is an urgent need in many critical infrastructure sectors, including the energy sector, for attaining detailed insights into cybersecurity features and compliance with cybersecurity requirements related to their Operational Technology (OT) deployments. Frequent feature changes of OT devices interfere with this need, posing a great risk to customers. One effective way to address this challenge is via a semi-automated cyber-physical security assurance approach, which enables verification and validation of the OT device cybersecurity claims against actual capabilities, both pre- and post-deployment. To realize this approach, this paper presents new methodology and algorithms to automatically identify cybersecurity-related claims expressed in natural language form in ICS device documents. Here, we developed an identification process that employs natural language processing (NLP) techniques with the goal of semi-automated vetting of detected claims against their device implementation. We also present our novel NLP components for verifying feature claims against relevant cybersecurity requirements. The verification pipeline includes components such as automated vendor identification, device document curation, feature claim identification utilizing sentiment analysis for conflict resolution, and reporting of features that are claimed to be supported or indicated as unsupported. Our novel matching engine represents the first automated information system available in the cybersecurity domain that directly aids the generation of ICS compliance reports.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2022

MetaText: Compositional Generalization in Deep Language Models

Compositional generalization, the ability to infer and reason using novel combinations of previously encountered entities and structures, is a trait of great utility across a variety of deep learning tasks; it implies a certain ability to reason using consistent and therefore interpretable rules, to understand the constituent parts of its input at multiple levels of granularity, to be robust to spurious differences between semantically equivalent inputs, and more. We seek to understand the degree to which existing natural language models achieve or fall short of compositional generalization across a variety of tasks, whether there are distinct types of compositional generalization in practice, and to identify avenues of intervention through which we can improve models’ compositional generalization ability. After exploring these questions using an array of models, tasks, and potential interventions we see that large, pretrained language models have encountered sufficient training data to account for a variety of fine-grained compositional behavior but still struggle to reason at the level of phrases or larger language structures. Non-intrusive, data-based interventions in the form of augmenting individual sequences with compressed versions of themselves or deriving new examples from induced grammars prove insufficient to encourage greater levels of compositional reasoning, indicating that future work might benefit most from focusing on changes to a model’s inductive bias at the architectural or loss level, or by integrating compositionality-boosting data interventions into the large-scale pretraining process itself.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Investigating Sociodemographic Disparities in Cancer Risk Using Web-Based Informatics

Cancer health disparities due to demographic and socioeconomic factors are an area of great interest in the epidemiological community. Adjusting for such factors is important when developing cancer risk models. However, for digital epidemiology studies relying on online sources such information is not readily available. This paper presents a novel method for extracting demographic and socioeconomic information from openly available online obituaries. The method relies on tailored language processing rules and a probabilistic scheme to map subjects’ occupation history to the occupation classification codes and related earnings provided by the U.S. Census Bureau. Using this information, a case-control study is executed fully in silico to investigate how age, gender, parity, and income level impact breast and lung cancer risk. Based on 48,368 online obituaries (4,643 for breast cancer, 6,274 for lung cancer, and 37,451 cancer-free) collected automatically and a generalized cancer risk model, our study shows strong association between age, parity, and socioeconomic status and cancer risk. Although for breast cancer the observed trends are very consistent with traditional epidemiological studies, some inconsistency is observed for lung cancer with respect to socioeconomic status.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2023

pvOps: a Python package for empirical analysis of photovoltaic field data

The purpose of pvOps is to support empirical evaluations of data collected in the field related to the operations and maintenance (O&M) of photovoltaic (PV) power plants. pvOps presently contains modules that address the diversity of field data, including text-based maintenance logs, current-voltage (IV) curves, and timeseries of production information. The package functions leverage machine learning, visualization, and other techniques to enable cleaning, processing, and fusion of these datasets. These capabilities are intended to facilitate easier evaluation of field patterns and extraction of relevant insights to support reliability-related decision-making for PV sites. The open-source code, examples, and instructions for installing the package through PyPI can be accessed through the GitHub repository.

14 SOLAR ENERGY↗

DOE OSTI2019

Full Integration of Lipidomics Data into Multi-OMIC Functional Enrichment

Lipids have numerous roles critical to living systems. Many of these vital functions are tightly linked or regulated by proteins. Even though the function of many lipid-related enzymes is known, direct links between most individual lipid species and the enzymes that metabolize them are severely lacking. This missing connection impedes biological progress. A resource that allows users to quickly identify protein-lipid relationships will advance our understanding of lipid homeostasis and metabolic perturbed systems. Databases are readily available for mapping small molecule metabolites (i.e., polar metabolites) and specific oxidized fatty acid lipids (e.g., arachidonic acid derivatives) to proteins/enzymes known to be directly involved in metabolism of these molecules (e.g. SMPDB, Reactome, HMDB, BioCyc). This enables the integration of metabolite and protein (as well as transcript) data when performing functional enrichment and metabolic modeling allowing researchers to rapidly examine metabolic pathways from multiple omics sources. Currently, linking lipidomics data with proteomics data is conducted using a time-intensive, user defined, manual process. Here we propose creating a tool that will link lipidomics and proteomics data and also conduct functional enrichment analysis and visualization allowing for full statistical integration. Using text-mining approaches, we will extract lipid references from uniprot and create a database linking lipid species to enzymes directly related to their metabolism. We will also extract directional annotation, which will allow even greater specificity when linking data types and will help eliminate false positives from functional associations.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

CyBERT: Cybersecurity Claim Classification by Fine-Tuning the BERT Language Model

We introduce CyBERT, a cybersecurity feature claims classifier based on bidirectional encoder representations from transformers and a key component in our semi-automated cybersecurity vetting for industrial control systems (ICS). To train CyBERT, we created a corpus of labeled sequences from ICS device documentation collected across a wide range of vendors and devices. This corpus provides the foundation for fine-tuning BERT’s language model, including a prediction-guided relabeling process. We propose an approach to obtain optimal hyperparameters, including the learning rate, the number of dense layers, and their configuration, to increase the accuracy of our classifier. Fine-tuning all hyperparameters of the resulting model led to an increase in classification accuracy from 76% obtained with BertForSequenceClassification’s original architecture to 94.4% obtained with CyBERT. Furthermore, we evaluated CyBERT for the impact of randomness in the initialization, training, and data-sampling phases. CyBERT demonstrated a standard deviation of ±0.6% during validation across 100 random seed values. Finally, we also compared the performance of CyBERT to other well-established language models including GPT2, ULMFiT, and ELMo, as well as neural network models such as CNN, LSTM, and BiLSTM. The results showed that CyBERT outperforms these models on the validation accuracy and the F1 score, validating CyBERT’s robustness and accuracy as a cybersecurity feature claims classifier.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Automating Bug Report Classification with Few Shot Learning

Orthogonal defect classification (ODC) is a method used to categorize software defects, providing valuable insights into the development process. This study focuses on automating the classification of software bug reports into different ODC defect types using few shot learning, a machine learning approach that requires minimal labeled data. Previous research has manually classified bug reports or used traditional machine learning algorithms like linear support vector machine, achieving limited success. Our approach uses few shot learning to improve classification accuracy and efficiency. The results show a harmonic mean of recall and precision (i.e., the F1 score) of around 0.6 which is a performance improvement over previous methods. The results highlight the potential benefit of few shot learning techniques and their application in enhancing the safety and reliability of nuclear digital instrumentation and control (DI&C) systems. Future work will explore incorporating advanced techniques to supplement the model's training data and achieve better results.

42 - ENGINEERING↗

DOE OSTI2017

Deep Learning for Automated Extraction of Primary Sites from Cancer Pathology Reports

Pathology reports are a primary source of information for cancer registries which process high volumes of free-text reports annually. Information extraction and coding is a manual, labor-intensive process. Here in this study we investigated deep learning and a convolutional neural network (CNN), for extracting ICDO- 3 topographic codes from a corpus of breast and lung cancer pathology reports. We performed two experiments, using a CNN and a more conventional term frequency vector approach, to assess the effects of class prevalence and inter-class transfer learning. The experiments were based on a set of 942 pathology reports with human expert annotations as the gold standard. CNN performance was compared against a more conventional term frequency vector space approach. We observed that the deep learning models consistently outperformed the conventional approaches in the class prevalence experiment, resulting in micro and macro-F score increases of up to 0.132 and 0.226 respectively when class labels were well populated. Specifically, the best performing CNN achieved a micro-F score of 0.722 over 12 ICD-O-3 topography codes. Transfer learning provided a consistent but modest performance boost for the deep learning methods but trends were contingent on CNN method and cancer site. Finally, these encouraging results demonstrate the potential of deep learning for automated abstraction of pathology reports.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗

DOE OSTI2022

Machine Learning Modeling Pipeline for Extracting Nuclear Proliferation Events of Interest from Open Data Sources (U)

In FY2020, the Savannah River National Laboratory (SRNL) and the Sanghani Center for Artificial Intelligence and Data Analytics at Virginia Polytechnic Institute and State University entered a collaboration funded by Department of Energy’s (DOE) Office of Defense Nuclear Nonproliferation Research and Development. The project’s mission was to take the first steps toward developing a demonstration prototype system that uses multiple machine learning and data analytics methods on largescale open data sources to identify new, developing, and/or undeclared nuclear programs. Given the SRNL team’s on-site perspective of events culminating in the DOE’s decision to pursue the Savannah River Plutonium Processing Facility (SRPPF), the team targeted the identification of events and indicators in retrospective datasets that pointed to the activity of “fissile core fabrication at the Savannah River Site” prior to the official announcement in May of 2018. A preliminary modeling pipeline was developed in FY20 that showed the datasets contained adequate signal for continuation of efforts. In FY21, a modular demonstration prototype modeling pipeline has continued in development for two text-based data sources: a broad internet archive (Webhose Ltd.) and a decahose Twitter database (i.e., a global sampling of one in every ten Tweets). The techniques that have been developed rely on graph theory and anomaly detection to identify contextual shifts in key words and phrases at various points in time such that indicators of events of interest could be identified and subsequently, events could be extracted from the corpuses. The foundational concept behind the approaches is that contextual shifts in key words and phrases can act as indicators of events of interest. Both datasets have proven successful in extracting events of interest related to pit production at the Savannah River Site prior to the official announcement. In addition, the pipelines have generated a wide range of events broadly summarized as: the awarding of DOE contracts at major sites, DOE investments in various programs, accidents at DOE national laboratories, speculations about the fate of pit production in the DOE complex, domestic and international shipments and receipts of nuclear materials at DOE sites, termination of non-proliferation agreements with Russia, termination of MOX, new weapons development approvals/testing, nuclear posture reviews, major DOE cleanup/production milestones, political opinions, and nuclear watch groups’ opinions, among many others.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Creating Training Data for Scientific Named Entity Recognition with Minimal Human Effort

Scientific Named Entity Referent Extraction is often more complicated than traditional Named Entity Recognition (NER). For example, in polymer science, chemical structure may be encoded in a variety of nonstandard naming conventions, and authors may refer to polymers with conventional names, commonly used names, labels (in lieu of longer names), synonyms, and acronyms. As a result, accurate scientific NER methods are often based on task-specific rules, which are difficult to develop and maintain, and are not easily generalized to other tasks and fields. Machine learning models require substantial expert-annotated data for training. Here we propose polyNER: a semi-automated system for efficient identification of scientific entities in text. PolyNER applies word embedding models to generate entity-rich corpora for productive expert labeling, and then uses the resulting labeled data to bootstrap a context-based word vector classifier. Evaluation on materials science publications shows that the polyNER approach enables improved precision or recall relative to a state-of-the-art chemical entity extraction system at a dramatically lower cost: it required just two hours of expert time, rather than extensive and expensive rule engineering, to achieve that result. This result highlights the potential for human-computer partnership for constructing domain-specific scientific NER systems.

Crowdsourcing↗

DOE OSTI2019

Classifying Cancer Pathology Reports with Hierarchical Self-Attention Networks

We introduce a deep learning architecture, hierarchical self-attention networks (HiSANs), designed for classifying pathology reports and show how its unique architecture leads to a new state-of-the-art in accuracy, faster training, and clear interpretability. We evaluate performance on a corpus of 374,899 pathology reports obtained from the National Cancer Institute's (NCI) Surveillance, Epidemiology, and End Results (SEER) program. Each pathology report is associated with five clinical classification tasks – site, laterality, behavior, histology, and grade. We compare the performance of the HiSAN against other machine learning and deep learning approaches commonly used on medical text data – Naive Bayes, logistic regression, convolutional neural networks, and hierarchical attention networks (the previous state-of-the-art). We show that HiSANs are superior to other machine learning and deep learning text classifiers in both accuracy and macro F-score across all five classification tasks. Compared to the previous state-of-the-art, hierarchical attention networks, HiSANs not only are an order of magnitude faster to train, but also achieve about 1% better relative accuracy and 5% better relative macro F-score.

60 APPLIED LIFE SCIENCES↗