Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Natural Language Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

labquake_future_prediction

The labquake_future_prediction code is a collection of python modules and scripts that serves as supporting information for the article “Predicting future laboratory fault friction through deep learning” for publication in the journal of “Geophysical Research Letters”. It is designed to predict laboratory fault slips in the immediate future by scanning continuous acoustic emission (AE) waveforms recorded in laboratory biaxial shear experiments. The predictions are made with a deep learning model based on convolutional encoder-decoder (CED) models and the Transformer model primarily developed for Natural Language Processing (NLP). The deep learning model is trained with the tensorflow package using publicly available laboratory data sets in standard binary file format in numpy. The utility functions for reading data files, configuring model hyperparameters, constructing the CED and Transformer models, training and testing of the models are defined in python module files. The workflow of training the models for labquake future predictions and the multiple GPU’s rapid model hyperparameter optimization as described in the journal article, are demonstrated in accompanying python script files and Jupyter notebooks.

Wang, Kun↗
DOE OSTI2023

Hpc Natural Language Understanding (nlu) Dataset

This code provides natural language annotation and labels for multiple activities in high performance. The purpose is to enable machine learning for natural language processing algorithms with high performance computing systems.

Biggs, BrandonS↗
DOE OSTI2026

Cyote-attack Chain Estimator

Attack Chain Estimator (ACE) Application Overview The Attack Chain Estimator (ACE) Application is a sophisticated tool designed for the ingestion, classification, sequencing, and enrichment of cybersecurity threat reports. This application leverages advanced machine learning models and extensive historical data to provide comprehensive insights into cyber threats, specifically targeting Industrial Control Systems (ICS). Purpose The primary functions of the ACE Application include: Ingestion of Cybersecurity Threat Reporting: Capable of ingesting text-based threat reports in markdown or text file format. Supports ingestion of structured data from other sources in STIX/JSON format. Classification of Report’s Text-Based Events: Utilizes a DeBERTa classifier, specifically trained on cybersecurity data, to map the events to MITRE ATT&CK for ICS Tactics and Techniques. Classification is performed using multiple Jupyter notebooks and machine learning workflows hosted as FastAPI microservices: regex_data deberta_base_35_train_hft_classifier_mlflow.ipynb hft_regex_classifier_mlflow.ipynb param_train_hft_classifier_mlflow.ipynb regex_tactic_tech.ipynb Ordering of Tactics, Techniques, and Observable Events: Sequences the identified tactics, techniques, and events to form a coherent attack chain. Enrichment with Historical Attack Chain Details: Enhances the attack chain with details from historical attacks using a Markov model developed from CyOTE Precursor Analysis Report data. The Markov model is available as a FastAPI endpoint for seamless integration. Enrichment with Adversary Emulation Capabilities Data: Integrates adversary emulation capabilities data using MITRE Caldera for OT adversary abilities UUIDs. Export of Output Files: Provides options to export the enriched attack chain in JSON or CSV formats. Routing of Output to Other Applications: Facilitates routing of output to various platforms and applications, including: Threat Intelligence Platforms COREII Scout for Threat Intelligence Analysis COREII Modeling and Simulation for Adversary Emulation Technical Description The ACE Application is an advanced cybersecurity tool designed to provide detailed threat analysis and sequence generation. It is built on a robust architecture that integrates natural language processing, machine learning, and historical data modeling. Key Components: Data Ingestion Module: Handles the input of threat reports and data from various formats, ensuring flexibility in data sources. Classification Engine: Employs DeBERTa-based classifiers hosted as FastAPI microservices to analyze and classify threat report events in accordance with the MITRE ATT&CK framework for ICS. Sequence Generator: Orders the classified events into a logical attack chain, providing clear insight into the sequence of tactics and techniques used in the threat. Enrichment Engine: Integrates historical data and adversary emulation capabilities to enhance the attack chain with valuable context and additional details. The historical data enrichment is powered by a Markov model, which is available as a FastAPI endpoint. Export and Routing Module: Facilitates the export of the enriched attack chain in multiple formats and routes the output to designated applications for further analysis or emulation.

Paul, Tony [Idaho National Laboratory (INL), Idaho↗
DOE OSTI2022

A meta-analysis of semantic classification of citations

The aim of this literature review is to examine the current state of the art in the area of citation classification. In particular, we investigate the approaches for characterizing citations based on their semantic type. We conduct this literature review as a meta-analysis covering 60 scholarly articles in this domain. Although we included some of the manual pioneering works in this review, more emphasis is placed on the later automated methods, which use Machine Learning and Natural Language Processing (NLP) for analyzing the fine-grained linguistic features in the surrounding text of citations. The sections are organized based on the steps involved in the pipeline for citation classification. Specifically, we explore the existing classification schemes, data sets, pre-processing methods, extraction of contextual and non-contextual features, and the different types of classifiers and evaluation approaches. The review highlights the importance of identifying the citation types for research evaluation, the challenges faced by the researchers in the process, and the existing research gaps in this field.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

The role of machine learning in scientific workflows

Machine learning (ML) is being applied in a number of everyday contexts from image recognition, to natural language processing, to autonomous vehicles, to product recommendation. In the science realm, ML is being used for medical diagnosis, new materials development, smart agriculture, DNA classification, and many others. In this article, we describe the opportunities of using ML in the area of scientific workflow management. Scientific workflows are key to today’s computational science, enabling the definition and execution of complex applications in heterogeneous and often distributed environments. We describe the challenges of composing and executing scientific workflows and identify opportunities for applying ML techniques to meet these challenges by enhancing the current workflow management system capabilities. We foresee that as the ML field progresses, the automation provided by workflow management systems will greatly increase and result in significant improvements in scientific productivity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

CAT: computer aided triage improving upon the Bayes risk through ε-refusal triage rules

Manual extraction of information from electronic pathology (epath) reports to populate the Surveillance, Epidemiology, and End Result (SEER) database is labor intensive. Systematizing the data extraction automatically using machine-learning (ML) and natural language processing (NLP) is desirable to reduce the human labor required to populate the SEER database and to improve the timeliness of the data. This enables scaling up registry efficiency and collection of new data elements. To ensure the integrity, quality, and continuity of the SEER data, the misclassification error of ML and NPL algorithms needs to be negligible. Current algorithms fail to achieve the precision of human experts who can bring additional information in their assessments. Differences in registry format and the desire to develop a common information extraction platform further complicate the ML/NLP tasks. The purpose of our study is to develop triage rules to partially automate registry workflow to improve the precision of the auto-extracted information. This paper presents a mathematical framework to improve the precision of a classifier beyond that of the Bayes classifier by selectively classifying item that are most likely to be correct. This results in a triage rule that only classifies a subset of the item. We characterize the optimal triage rule and demonstrate its usefulness in the problem of classifying cancer site from electronic pathology reports to achieve a desired precision. From the mathematical formalism, we propose a heuristic estimate for triage rule based on post-processing the soft-max output from standard machine learning algorithms. We show, in test cases, that the triage rule significantly improve the classification accuracy.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Scalable deep text comprehension for Cancer surveillance on high-performance computing

Background: Deep Learning (DL) has advanced the state-of-the-art capabilities in bioinformatics applications which has resulted in trends of increasingly sophisticated and computationally demanding models trained by larger and larger data sets. This vastly increased computational demand challenges the feasibility of conducting cutting-edge research. One solution is to distribute the vast computational workload across multiple computing cluster nodes with data parallelism algorithms. In this study, we used a High-Performance Computing environment and implemented the Downpour Stochastic Gradient Descent algorithm for data parallelism to train a Convolutional Neural Network (CNN) for the natural language processing task of information extraction from a massive dataset of cancer pathology reports. We evaluated the scalability improvements using data parallelism training and the Titan supercomputer at Oak Ridge Leadership Computing Facility. To evaluate scalability, we used different numbers of worker nodes and performed a set of experiments comparing the effects of different training batch sizes and optimizer functions.Results: We found that Adadelta would consistently converge at a lower validation loss, though requiring over twice as many training epochs as the fastest converging optimizer, RMSProp. The Adam optimizer consistently achieved a close 2nd place minimum validation loss significantly faster; using a batch size of 16 and 32 allowed the network to converge in only 4.5 training epochs.Conclusions: We demonstrated that the networked training process is scalable across multiple compute nodes communicating with message passing interface while achieving higher classification accuracy compared to a traditional machine learning algorithm.

Qiu, John X.↗
DOE OSTI2022

Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients

Mortality prediction for intensive care unit (ICU) patients is crucial for improving outcomes and efficient utilization of resources. Accessibility of electronic health records (EHR) has enabled data-driven predictive modeling using machine learning. However, very few studies rely solely on unstructured clinical notes from the EHR for mortality prediction. In this work, we propose a framework to predict short, mid, and long-term mortality in adult ICU patients using unstructured clinical notes from the MIMIC III database, natural language processing (NLP), and machine learning (ML) models. Depending on the statistical description of the patients’ length of stay, we define the short-term as 48-hour and 4-day period, the mid-term as 7-day and 10-day period, and the long-term as 15-day and 30-day period after admission. We found that by only using clinical notes within the 24 hours of admission, our framework can achieve a high area under the receiver operating characteristics (AU-ROC) score for short, mid and long-term mortality prediction tasks. The test AU-ROC scores are 0.87, 0.83, 0.83, 0.82, 0.82, and 0.82 for 48-hour, 4-day, 7-day, 10-day, 15-day, and 30-day period mortality prediction, respectively. We also provide a comparative study among three types of feature extraction techniques from NLP: frequency-based technique, fixed embedding-based technique, and dynamic embedding-based technique. Lastly, we provide an interpretation of the NLP-based predictive models using feature-importance scores.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

COVIDScholar: An automated COVID-19 research aggregation and analysis platform

The ongoing COVID-19 pandemic produced far-reaching effects throughout society, and science is no exception. The scale, speed, and breadth of the scientific community’s COVID-19 response lead to the emergence of new research at the remarkable rate of more than 250 papers published per day. This posed a challenge for the scientific community as traditional methods of engagement with the literature were strained by the volume of new research being produced. Meanwhile, the urgency of response lead to an increasingly prominent role for preprint servers and a diffusion of relevant research through many channels simultaneously. These factors created a need for new tools to change the way scientific literature is organized and found by researchers. With this challenge in mind, we present an overview of COVIDScholar https://covidscholar.org/, an automated knowledge portal which utilizes natural language processing (NLP) that was built to meet these urgent needs. The search interface for this corpus of more than 260,000 research articles, patents, and clinical trials served more than 33,000 users at an average of 2,000 monthly active users and a peak of more than 8,600 weekly active users in the summer of 2020. Additionally, we include an analysis of trends in COVID-19 research over the course of the pandemic with a particular focus on the first 10 months, which represents a unique period of rapid worldwide shift in scientific attention.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Geospatial and Information Substitution and Anonymization Tool (GISA)

The Geospatial and Information Substitution and Anonymization Tool (GISA) incorporates techniques for obfuscating identifiable information from point data or documents, while simultaneously maintaining chosen variables to enable future use and meaningful analysis. This approach promotes collaboration and data sharing while also reducing the risk of exposure to sensitive information. GISA can be used in a number of different ways, including the anonymization of point spatial data, batch replacement/removal of user-specified terms from file names and from within file content, and aid with the selection and redaction of images and terms based on recommendations using natural language processing. Version 1 of the tool, published here, has updated functionality and enhanced capabilities to the beta version published in 2023. Please see User Documentation for further information on capabilities, as well as a guide for how to download and use the tool. If there are any feedback you would like to provide for the tool, please reach out with your feedback to edxsupport@netl.doe.gov. Disclaimer: This project was funded by the United States Department of Energy, National Energy Technology Laboratory, in part, through a site support contract. Neither the United States Government nor any agency thereof, nor any of their employees, nor the support contractor, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. The Geospatial and Information Substitution and Anonymization Tool (GISA) was developed jointly through the U.S. DOE Office of Fossil Energy and Carbon Management’s EDX4CCS Project, in part, from the Bipartisan Infrastructure Law.

Bipartisan Infrastructure Law↗
DOE OSTI2025

The Carbon Storage Technical Viability Approach (CS TVA)

The Carbon Storage Technical Viability Approach (CS TVA) StoryMap provides an in-depth overview of the products created during the CS TVA research effort. In detail, the StoryMap addresses the CS TVA Matrix, Database Version 2.0, Database Catalog, Workflow, and Data Availability Result Database, discussing how each was developed and implemented. Information on how the matrix, database, and database catalog are interconnected, and their usage is also explained. The workflow section provides information on the CS TVA product development from the data-gathering stage to the final data availability results. A section on an expansion of the CS TVA workflow that utilizes Natural Language processing (NLP) section was included. Finally, the Data Availability Results Database is discussed. These results provide data science-informed insights into potential data gaps when assessing the viability of carbon storage in a given area or region.

Carbon Storage↗
DOE OSTI2022

Foundation Models of Scientific Knowledge for Chemistry: Opportunities, Challenges and Lessons Learned

Foundation models pre-trained on large corpora demonstrate significant gains across many natural language processing tasks and domains e.g., law, healthcare, education, etc. However, only limited efforts have investigated the opportunities and limitations of applying these powerful models to science and security applications. In this work we develop foundation models of scientific knowledge for chemistry to augment scientists with the advanced ability to perceive and reason at scale previously unimagined. Specifically, we build large-scale (1.47B parameter) general-purpose models for chemistry that can be effectively used to perform a wide range of in-domain and out-of-domain tasks. Evaluating these models in a zero-shot setting, we analyze the effect of model and data scaling, knowledge depth, and temporality on model performance in context of model training efficiency. Our novel findings demonstrate that (1) model size significantly contributes to the task performance when evaluated in a zero-shot setting; (2) data quality (aka diversity) affects model performance more than data quantity; (3) similarly, unlike previous work (Luu et al., 2021) temporal order of the documents in the corpus boosts model performance only for specific tasks, e.g., SciQ; and (4) models pre-trained from scratch perform better on in-domain tasks than those tuned from general-purpose models like Open AI’s GPT-2.

Foundation Models, Chemistry↗
DOE OSTI2014

Trend Analysis in Biomedical Texts via Vector Space Model Synonym Extraction

Accurate synonym extraction is a highly desired tool in computational semantics and natural language processing (NLP). Intrinsic to many current language models is the Vector Space Model (VSM) whereby linguistic properties are modeled by real vectors of small length compared to the document size. Here, several vector models are discussed. Utilizing the open source Word2Vec tool, temporal trends in biomedical abstracts from MEDLINE have been analyzed via a unique transformation matrix method. Additionally, accuracy of the word vector method and clustering of temporal trends are discussed.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Exploiting Social Media Sensor Networks through Novel Data Fusion Techniques

Unprecedented amounts of data are continuously being generated by sensors (“hard” data) and by humans (“soft” data), and this data needs to be exploited to its full potential. The first step in exploiting this data is determine how the hard and soft data are related to each other. In this project we fuse hard and soft data, using the attributes of each (e.g., time and space), to gain more information about interesting events. Next, we attempt to use social networking textual data to predict the present (i.e., predict that an interesting event is occurring and details about the event) using data mining, machine learning, natural language processing, and text analysis techniques.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Evolution of Water Narratives in Local US Newspapers: A Case Study of Utah and Georgia

Narratives about water resources have evolved, transitioning from a sole focus on physical and biological dimensions to incorporate social dynamics Recently, the importance of understanding the visibility of water resources through media coverage has gained attention. This study leverages recent advancements in natural language processing (NLP) methods to characterize and understand patterns in water narratives, specifically in 4 local newspapers in Utah and Georgia. Analysis of the corpus identified coherent topics on a variety of water resources issues, including weather and pollution. Closer inspection of the topics revealed temporal and spatial variations in coverage, with a topic on hurricanes exhibiting cyclical patterns whereas a topic on tribal issues showed coverage predominantly in the western newspapers. We also analyzed the dataset for sentiments, identifying similar categories of words on trust and fear emerging in the narratives across newspaper sources. An analysis of novelty, transience, and resonance using Kullback-Leibler Divergence techniques revealed that topics with high novelty generally contained high transience and marginally high resonance over time. Although additional analysis needs to be conducted, the methods explored in this analysis demonstrate the potential of NLP methods to characterize water narratives in media coverage.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Enhancing Clean Energy Innovation Ecosystem Discovery Tool

During March 2016, the DOE’s Office of Energy Policy and Systems Analysis (EPSA) sought a methodology to identify and quantify the strength of the existing clean energy innovation ecosystems (IE) in the U.S. and their characteristics. Oak Ridge National Lab (ORNL) did a pilot study (Phase 1) and demonstrated the feasibility of an application comprised of natural language processing, link analysis, and other computational techniques to transform text and numerical data into metrics on clean energy innovation activity and geography. The data-collection, ingest and analytics pipelines were combined with an advanced user interface, together known as the Ecosystem Discovery Tool, to enhance DOE’s understanding of existing geographic innovation clusters. During Phase 2 of this project, this tool has been further enhanced by integrating new data sets and through backend software architecture changes, validation and bug fixing, making it a much more robust and powerful application. The tool’s user interface is a lot more intuitive, which enables a user to visualize the IE rankings for various geographic regions(CBSA, state level) for different clean energy technologies and seek automated insights. ORNL also created a DOE-specific dashboard that allows the user to visualize and analyze federal funding from various offices including Energy Efficiency and Renewable Energy(EERE), Office of Electricity(OE), Fossil Energy(FE), and Nuclear Energy(NE). During Phase 1 of this project, EPSA defined a clean energy innovation ecosystem as the overlap of five Ecosystem Components: 1) nascent clean energy Indicators, 2) investors, 3) enabling environment, 4) networking assets and 5) large companies. EPSA and ORNL worked together to collect data for each component: 1) small and medium companies, ARPA-E awardees, SBIR awardees, patents, publications, ERCs for Nascent Clean Energy Indicators; 2) qualified investors; 3) the Clean Edge Policy Index/DSIRE for the Enabling Environment; 4) universities, national laboratories, ERCs and incubators for Networking Assets; and 5) large companies and a subset of the Russell 1000 list for Large Companies. The ORNL team created a visual tool based on Tableau that integrated ecosystem component data to score, rank and map IEs. The tool was created with the flexibility to allow the user to choose the weights of each of the five ecosystem components and the subcomponents. This flexibility allows the user to visualize different subsets and to use the underlying data for different types of analysis. During Phase 2 of this project, the backend database and computation process have undergone major changes to provide a much more refined user-interface and added new features into the scoring algorithm. This phase also involved addition of newer databases such as a list of Energy Research Centers (ERC) ,DSIRE, and funding data from EERE, FE, NE, and OE to provide more granular information and more accurate/better quantification.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2019

Data and Models: A Framework for Advancing AI in Science

On June 5, 2019, the Office of Science (SC) organized a one-day roundtable to focus on enhancing access to high-quality and fully traceable research data, models, and computing resources to increase the value of such resources for artificial intelligence (AI) research and development and the SC mission.1 In this report, we consider AI to be inclusive of, for example, machine learning (ML), deep learning (DL), neural networks (NN), computer vision, and natural language processing (NLP). We consider “data for AI” to mean the digital artifacts used to generate AI models and/or employed in combination with AI models during inference. In part, this roundtable was motivated by the recognition that a large portion of science data currently are not well suited for AI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Rapid Response Data Science for COVID-19

This report describes the results of a seven day effort to assist subject matter experts address a problem related to COVID-19. In the course of this effort, we analyzed the 29K documents provided as part of the White House's call to action. This involved applying a variety of natural language processing techniques and compression-based analytics in combination with visualization techniques and assessment with subject matter experts to pursue answers to a specific question. In this paper, we will describe the algorithms, the software, the study performed, and availability of the software developed during the effort.

60 APPLIED LIFE SCIENCES↗