Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Natural Language Understanding”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

SWAN: An expert system with natural language interface for tactical air capability assessment

SWAN is an expert system and natural language interface for assessing the war fighting capability of Air Force units in Europe. The expert system is an object oriented knowledge based simulation with an alternate worlds facility for performing what-if excursions. Responses from the system take the form of generated text, tables, or graphs. The natural language interface is an expert system in its own right, with a knowledge base and rules which understand how to access external databases, models, or expert systems. The distinguishing feature of the Air Force expert system is its use of meta-knowledge to generate explanations in the frame and procedure based environment.

Simmons, Robert M.↗

From the Origins of Life to Intelligence: The Emergence of Symbolic Constructs

Intelligence cannot be understood - and cannot be 'artificially recreated' - without also understanding how it fits as a phenomenon in the evolution of matter. I say 'matter' instead of 'life' because my thesis is that the distinction between matter, with its associated interactions, and life, is simply in the complexity of these interactions and in the number of 'symbolic levels' that are defined by these interactions. Most of us think of symbols only in the context of language. This is understandable, since it is at this level where it is easiest to draw a distinction between natural objects and their 'names', i.e., the different sets of 'abstract' objects that can be manipulated to produce models of the 'real' world. Of course sets of abstract objects can also acquire names and be manipulated at higher and higher conceptual levels. When we use the words 'philosophy' or 'the Declaration of Independence' we use abstract constructs that will only make sense in other specific abstract contexts. Additional information is contained in the original extended abstract.

Colombano, Silvano P.↗

Toward particle accelerator machine state embeddings as a modality for large language models

Understanding and diagnosing the state of a particle accelerator requires navigating high-dimensional control system data, often involving hundreds of interdependent parameters. We propose a novel multimodal embedding framework that jointly learns representations of machine states from both numerical control system readouts and natural language descriptions. This enables the translation of complex machine conditions into human-readable summaries while maintaining fidelity to the underlying physical system. The obtained embeddings are subsequently adapted to an open-weights large language model via cross-attention conditioning. We demonstrate a first implementation trained on European XFEL machine state data. This work covers the embedding model architecture, training methodology, and presents initial examples demonstrating the model's capabilities in action. Due to the general concept of machine state, the model can be easily adapted to other facilities and control system environments.

Accelerator Physics↗

What Is the Agent Doing? Visualizing Agentic AI Querying Workflows

We explore how visualizations can help users understand what an AI agent is doing as it builds and runs queries over data. As part of the LinkQ system, a natural language interface for querying knowledge graphs with a large language model (LLM), we designed two complementary views: A State Diagram that shows where the agent is within a larger workflow, and a Live Action Display that gives real-time updates about the agent's current task. In a study with 14 practitioners, we found that these visuals helped participants build stronger mental models of the agent's behavior while also increasing their confidence in the system. However, we also observed that users sometimes trusted incorrect outputs simply because the agent appeared to be doing the "right" thing. Our findings point to both the value and risk of visualizing agent behavior in interactive AI systems.

97 MATHEMATICS AND COMPUTING↗

Understanding Event Trajectories Across Massive Temporal Datasets with Word Embeddings and Visualization

In collaboration with researchers from Virginia Tech, Savannah River National Laboratory has continued development of a natural language processing pipeline to identify and extract events of interest from massive open data sources in the domain of worldwide state-sponsored civil nuclear energy. The foundation of the pipeline is built on compass aligned temporal word embedding models, whereby contextual shifts are automatically identified by comparing keyword embedding vectors across successive time windows. Within the approach, a contextual shift indicates the occurrence of a potential event of interest. However, in such a broad topical domain that captures events at a global scale, across various life cycle stages, and across numerous different technology types, a user that is monitoring events may have broad interests in capturing many different event types with varying degrees of signal. As such, the quantity of information that may be returned from an automated event extraction pipeline can be substantial, requiring manual effort to sift through the information to identify any relevant bits of information. Therefore, a more streamlined workflow that aids in directing a user toward specific information at different points in time is necessary. The workflow presented here has been developed with this concept in mind, built on top of the initial prototype event extraction pipeline, whereby a user can analyze temporal text-based data sources at multiple different contextual levels to isolate key points in time and key subdomains captured within a data corpus. Using multiple corpuses that consist of approximately 7 million Tweets and 7 million news articles, the team has extended compass aligned temporal word embedding models to establish an interconnected and hierarchical structure that relates known key words of interest to documents, local topics (i.e., within a time window), and global topics across the corpuses. All of this information is packaged into a visual analytics system that is linked to the information extraction pipeline and enables a user to identify contextual information that describes the evolution of a high dimensional embedding space across time to isolate changes of interest and explore associated events. This report demonstrates the use of these analytics and a means to fuse information across multiple datasets.

97 MATHEMATICS AND COMPUTING↗

A Model-Based Systems Engineering Journey to Developing a Concept of Operations

Starting in 2017, NASA’s Human Research Program (HRP) Exploration Medical Capability (ExMC) element began a systems engineering transition from traditional, document-centric development to model-centric development when defining its foundation medical systems. These foundation medical systems define a Concept of Operations (ConOps) and identify the generic requirements for a medical system based on assumptions about a generic crew and mission environments and guidance from NASA standards (e.g., Medical “Levels of Care”). By making the transition, ExMC intends to improve communication among stakeholders about foundation medical system requirements and content. In addition, this transition will enable ExMC to lower both development and crew treatment risks for future, mission-specific medical systems. ExMC followed a Model Based Systems Engineering (MBSE) paradigm when developing the foundation medical systems. A model-based approach provides several advantages over a traditional, document-centric approach. First, when Systems Engineers (SE) develop diagrams in a model using a standard modeling language, they produce information dense pictures that facilitate understanding much more efficiently with less room for misinterpretation than text. Second, due to the evolving nature of projects, documentation becomes out of date the minute it is published. This can result in people making decisions based on information that is no longer current, especially if they are referencing a locally-stored copy of a document. A model, on the other hand, is always up to date with the latest approved changes and information. It serves as a single point of truth. Third, a model-centric approach centralizes all important information in one place. Rather than having to flip through separate ConOps documents, design specifications, requirements specifications, and the like to coordinate information, a model captures the content in one, integrated spot. This integration makes tracing information from end-to-end easier with greater reliability. The ExMC Systems Engineering Lifecycle follows a well-defined process. ExMC Systems Engineers perform all major steps of the process, regardless of the development methodology. One of the first steps in the process is developing the ConOps that describes the operation of the system from the point of view of the users. It includes a list of the users and their needs, the goals of the medical system, key assumptions about the system, and definitions of the medical system’s operational environments. For this development effort, ExMC chose to replace the traditional text-based ConOps document with a model. While the decision to change the development workflow was not difficult, implementing the structural and organizational workflows were. It required showing ExMC’s users, most of whom are not Systems Engineers, how the information they require would be presented in the model and to gain their acceptance of this approach. This paper documents key lessons learned during the ConOps transformation by focusing on how the model represents information, the agile workflow used by SEs when developing the model and how it integrates into a project plan, how leadership influenced key users to accept the transformation, and how the users interact with the model information.

Jeffrey Robert Cohen↗

Learning Global Proliferation Expertise Evolution Using AI-Driven Analytics and Public Information

Detecting and anticipating global proliferation expertise and capability evolution from unstructured, noisy, and incomplete public data streams is a highly desired, but extremely challenging task. Here, in this article, we present our pioneering data-driven approach to support the non-proliferation mission to detect and explain the evolution of proliferation expertise and capability development globally from terabytes of publicly available information (PAI), focusing on our knowledge extraction pipeline and descriptive analytics. We first discuss how we fuse nine open-source data streams, including multilingual data, to convert 4 TB of unstructured data to structured knowledge and encode dynamically evolving proliferation expertise representations—content and context graphs. For this, we rely on natural language processing (NLP) and deep learning (DL) models to perform information extraction, topic modeling, and distributed text representation (aka embedding) learning. We then present interactive, usable, and explainable descriptive analytics to refine domain knowledge and present it in a human-understandable form. Finally, we introduce future work avenues that will leverage our dynamic knowledge representations and descriptive analytics to enable predictive and prescriptive inferences to achieve real-time domain understanding and contextual reasoning about global proliferation expertise and capability evolution.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Auto Procedure Parsing: A Natural Language Processing Approach

Nuclear Power Plant (NPP) operating procedure is “a set of rules that describes how actions on the plant should be made if a certain system goal should be accomplished. U.S. NPPs use paper-based procedures (PBPs). PBPs are difficult to use. Common errors with PBPs are: following the wrong procedure, omit a step etc. Computer based procedures (CPBs) offer great improvement in ensuring plant safety. Some studies rely on experienced operators to understand the procedure content and then reorganize the procedure with digitally executable capabilities. Other studies utilize the procedure format to design rules to extract information from the operating procedures. Existing studies in procedure parsing are manual, laborious and extract limited information. This study aims to automatically extract critical information from operating procedures for generating computer interpretable representation of procedures. Such representation can further be used for automatic dynamic human reliability analysis and CPB design etc.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The role of peer review and science for advancing public understanding

We live in a time of rapid change. The Covid-19 pandemic has united the world to understand and develop solutions to protect the health and wellbeing of society. At the same time, access to information on the internet and social media has exploded. People are exposed to information with many claims. Some of the information is relevant and accurate. A significant fraction may be inaccurate, biased, and potentially dangerous. With Covid-19, the information might point to cures that are toxic or risks that are not relevant. The scientific community has developed a process to develop and convey knowledge. The validation of results and their interpretation is critical to advancing knowledge. The core of the scientific process is based on developing a hypothesis-driven concept, designing and executing an experimental approach to prove or disprove the hypothesis, conducting a statistical analysis of the results to confirm their validity, and interpreting their meaning and impact. When their work is completed the scientist prepares to publish the work. Publication requires peer review. Hypotheses, experimental approaches, results, and interpretation of results are independently and anonymously reviewed by a scientific peer. The reviewers are selected based on their expertise in the field. The peer review process is essential for the integrity of the scientific process. Without a robust scientific process, society cannot discriminate between validated results and interpretations and biased, inaccurate, and potentially dangerous conjecture. With inaccurate science society and policy makers may make poor decisions that could hurt people and the world around them. While scientists should not censor information available to the public, scientists have a responsibility to inform the public. We must articulate what the scientific process is and why it is essential when considering the validity of information. Scientist are responsible for pointing out when public information has not been vetted through a scientific process and therefore cannot be considered conclusive. Scientists are responsible for highlighting information that has been generated and vetted with the scientific process. Over time, our goal is for the public to value the scientific process based on peer review. The public could become enlightened to the necessity for peer review to support decisions and action. Scientists need to contribute to advancing public knowledge. When invited to serve as a peer reviewer, scientists have a responsibility to contribute to the process. We all are extremely busy making peer review a distraction. While peer review does not directly benefit our own work and career, it benefits our chosen field of science. This is very important for the research area that our journal, Water-Energy Nexus, is focusing on. We cover the convergence of science and technology integrating across large spatial ranges (micro to global) in environmental pollution and the balance of remediation issues with energy use. Water-Energy Nexus wants to provide scientists with a platform for scientific debate and discussion on this rich and complicated topic. With the manuscripts submitted to WEN cover integrated research topics across a broad range of subjects from environment sciences through energy engineering, the role of peer review is essential. Peer review on a new submission to WEN should not be biased towards a specific science field. It should be done considering the complex nature of issues in the water-energy nexus. Scientists need to contribute to the public dialog. To accomplish these goals, we need to be honest, accurate, articulate, and use understandable language. The impact of WEN is very diverse affecting society in many ways. The scientific audience for WEN is diverse and readers may not fully understand the content of the papers outside their research expertise. Therefore, scientists, especially those who want to contribute to WEN, should work with communications specialists to convey message to benefit society. The audience includes the general public and researchers outside the author’s core specialty. While the peer-reviewed journal article is the cornerstone of our careers and critical for advancing scientific knowledge, a general interest story could benefit society even more. The author cannot lose sight that our work has two audiences, the science community, and more importantly, our family, friends, neighbors, and policy makers.

54 ENVIRONMENTAL SCIENCES↗

Large language model evaluation for high–performance computing software development

We apply AI-assisted large language model (LLM) capabilities of GPT-3 targeting high-performance computing (HPC) kernels for (i) code generation, and (ii) auto-parallelization of serial code in C ++, Fortran, Python and Julia. Our scope includes the following fundamental numerical kernels: AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG, and language/programming models: (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). Kernel implementations are generated using GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code given simple + + prompt variants. To quantify and compare the generated results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. For auto-parallelization, we use ChatGPT interactively giving simple prompts as in a dialogue with another human including simple “prompt engineering” follow ups. Results suggest that correct outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general-purpose Python can benefit from adding language keywords, while Julia prompts perform acceptably well for its Threads and CUDA.jl programming models. Finally, we expect to provide an initial quantifiable point of reference for code generation in each programming model using a state-of-the-art LLM. Overall, understanding the convergence of LLMs, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

97 MATHEMATICS AND COMPUTING↗

Improving Cyber Situational Understanding

Effective cybersecurity operations require the ability to analyze large amounts of information to assess security risks and formulate defensive strategies against adversaries. This has become more complex in recent years as the sprawl and interconnectivity of devices grows through implementation of virtualization, cloud computing, and Internet of Things (IoT). The amount of data and analysis required for effective cybersecurity command and control decisions far exceeds humans’ capacity to perform manually. We characterize the analysis problem as cyber situational understanding. The research presented to improve cyber situational understanding focuses on vulnerability analysis and threat intelligence. Regarding vulnerabilities, entities must analyze and plan work for between thousands and tens of thousands of software vulnerabilities annually. Entities heavily use network firewalls to limit vulnerability exposure. As a result, some of these vulnerabilities permit exposure to adversarial exploitation, whereas others are inaccessible and therefore present negligible risk of exploitation. Distinguishing between high and low risk software vulnerabilities requires a deep understanding of the vulnerability, network firewall protection, and characteristics of the targeted device. This problem is solved by extracting network service features from vulnerability data features using both machine-learning and natural language processing. Then, the network firewall topology is parsed to determine which vulnerabilities are reachable by adversaries. Ultimately, a state-based safety analysis ascertains which vulnerabilities are unsafe. A related vulnerability analysis problem occurs in cybersecurity operations when associating an entity’s hardware and software assets to public vulnerability databases. Assets often reveal hardware and software through installation artifacts and network service identification, and entities store these artifacts in inventory databases. However, software and hardware vendors apply a standard Common Platform Enumeration (CPE) naming convention when publicly reporting vulnerabilities. Associating these two datasets often requires many hours to days of manual inspection. The proposed solution automates the mapping approach of human analysts using fuzzy matching techniques, natural language processing, and, ultimately, machine learning to present a small set of recommendations for mapping the two datasets. The result significantly reduces human analysis time and reduces the occurrence of false positives in vulnerability notifications. Finally, cyber threat intelligence (CTI) requires associating cyber observable artifacts, such as IP addresses, URIs, and file hashes, with cyber threat tactics, techniques, and procedures. Unfortunately, most CTI data is compartmentalized across multiple organizations and cannot be shared due to the legal and reputational risk with cyber threat being associated with the entity. The approach to solving this problem inovlves using a distributed ledger with anonymous token spending and authentication. This allows a consortium of semi-trusted entities to share the workload of curating CTI for a threat sharing community’s cooperative benefit.

Huff, Philip↗

Transforming Science Prioritization Processes Using Artificial Intelligence

Artificial Intelligence (AI) and Machine Learning (ML) have potential to augment significantly the current labor-intensive processes of science prioritization, specifically by the National Academies’ Decadal Survey on behalf of NASA and NSF. Here we summarize what we believe to be the first exploratory demonstration-of-concept results from an application of AI/ML to Survey science prioritization. Specifically, we applied Latent Dirichlet Allocation (LDA) and Natural Language Processing (NLP) to reveal trends in published astrophysics research that may indicate science priorities and which could be applied to strategic planning. For the purpose of the work that we summarize here, AI/ML is able to analyze – that is, to “understand,” in a manner of speaking – a vast amount of text to reveal complex relationships among research topics, including the growth or decline of science community activities in those topics over time. We trained ourselves and AI/ML algorithms by using ~400,000 abstracts in the period 1998 to 2010 to “forecast” the Academies’ Astro2010 recommendations and compare with the solicited white papers. Comparing our results with actual Astro2010 recommendations allowed us to identify candidate metrics that better predicted the actual results of the Survey. We found, for example, that Compound Annual Growth Rate (CAGR) of papers published in a topic area is a good proxy measure for importance of this topic area of research. With this training complete, we identified candidate astrophysics astrophysics science priorities for the 2021+ period using the research during 2007 - 2019 . We conclude that appropriate application of AI can potentially significantly reduce the current workload of the Decadal Survey processes and reveal otherwise unrecognized characteristics in the body of astronomical research. We emphasize throughout the exploratory nature of our work, encouraging colleagues to pursue promising results further. Our most critical governing assumption was that increased (or decreased) research activity can be used to identify scientific or technology topic areas worthy of increased (or decreased) future emphasis. We discuss advantages, limitations, and recognize the “black box” nature of our technique. We note ethics issues associated, for example, with using AI/ML to reveal “hidden” meanings and biases in published work. Furthermore, inevitable improvements in AI may soon enable widespread and welcome identification of and advocacy for science and technology priorities by disparate and diverse groups and organizations. Consequently, we continue to urge a near-term, in-depth evaluation of appropriate applications of AI, including implications and consequences, as well as support for multiple follow-on assessments, of which ours is only a beginning.

Artificial Intelligence↗

Intelligent interfaces for expert systems

Vital to the success of an expert system is an interface to the user which performs intelligently. A generic intelligent interface is being developed for expert systems. This intelligent interface was developed around the in-house developed Expert System for the Flight Analysis System (ESFAS). The Flight Analysis System (FAS) is comprised of 84 configuration controlled FORTRAN subroutines that are used in the preflight analysis of the space shuttle. In order to use FAS proficiently, a person must be knowledgeable in the areas of flight mechanics, the procedures involved in deploying a certain payload, and an overall understanding of the FAS. ESFAS, still in its developmental stage, is taking into account much of this knowledge. The generic intelligent interface involves the integration of a speech recognizer and synthesizer, a preparser, and a natural language parser to ESFAS. The speech recognizer being used is capable of recognizing 1000 words of connected speech. The natural language parser is a commercial software package which uses caseframe instantiation in processing the streams of words from the speech recognizer or the keyboard. The systems configuration is described along with capabilities and drawbacks.

Villarreal, James A.↗

Large Language Model Integration for Knowledge Retrieval and Interaction for the DUNE Experiment

The Deep Underground Neutrino Experiment (DUNE) is a next-generation neutrino experiment that will generate an unprecedented volume of heterogeneous information-from documentation and technical notes to experimental data and reconstruction pipelines. Efficient knowledge retrieval and contextual understanding are increasingly critical for collaboration-wide productivity and onboarding. In this work, we present DUNE-GPT, a prototype framework that leverages large language models (LLMs) and retrieval-augmented generation (RAG) to enable natural-language querying of DUNE's internal documentation and technical resources. The system provides an intelligent interface for DUNE collaborators to interact with experiment-specific knowledge while maintaining data privacy and infrastructure compliance within Fermilab computing resources.

Rafique, A. [Argonne (main)]↗

Natural Language Processing to Inform Agent-Based Modeling: With Application to Modeling Adoption of Medium-Duty Electric Vehicles

Agent-based socio-technical modeling of medium- and heavy-duty (MDHD) electric vehicle (EV) adoption has the potential to provide analysis, prediction, and gui. This paper describes new applications of text analysis developed through machine learning (ML) to build and understand relevant topics and their saliency in the published discourse on adoption of MDHD EVs. This work contributes to the state of the art in topic mining models by defining a new metric of topic ranking (START) that quantifies the importance of predefined topics within the corpus using weighted results for predefined topics from two topic modeling approaches: Latent Dirichlet Allocation (LDA) and BERTopic. The START metric is then demonstrated in practice to model how academia and industry view the EV adoption process based on the respective texts published by these groups. Results show that academic literature places more emphasis on categories of interests such as norms/attitudes and adopter knowledge, while trade journals tend to emphasize long-term cost more than academia. The two bodies of literature agree on the importance of policy and incentives in MDHD EV adoption. Together these results illustrate the potential to use ML-based text analysis to populate the characteristics of agent-based socio-technical models.

Electric vehicle adoption, fleet electrification, ↗

Relationship Extraction: Automatic Information Extraction and Organization for Supporting Analysts in Threat Assessment

In order for analysts to be able to do their work, they sift through hundreds, thousands, or even millions of documents to make connections between entities of interest. This process is time consuming, tedious, and prone to potential error from missed connections or connections made that should not have been. There exist many tools in natural language processing, or NLP, to extract information from documents. However, when it comes to relationship extraction, there has been varied success. This project began with a goal to solve the relationship extraction problem which developed into a deeper understanding of the problem and the associated challenges for solving this problem on a general scale. In this report, we explain our research and approach to relationship extraction, identify other auxiliary problems in NLP that provide additional challenges to solving relationship extraction generally, explain our analysis of the current state of relationship extraction, and postulate future work to address these problems.

97 MATHEMATICS AND COMPUTING↗

The development of a prototype intelligent user interface subsystem for NASA's scientific database systems

The National Space Science Data Center (NSSDC) has initiated an Intelligent Data Management (IDM) research effort which has as one of its components the development of an Intelligent User Interface (IUI).The intent of the latter is to develop a friendly and intelligent user interface service that is based on expert systems and natural language processing technologies. The purpose is to support the large number of potential scientific and engineering users presently having need of space and land related research and technical data but who have little or no experience in query languages or understanding of the information content or architecture of the databases involved. This technical memorandum presents prototype Intelligent User Interface Subsystem (IUIS) using the Crustal Dynamics Project Database as a test bed for the implementation of the CRUDDES (Crustal Dynamics Expert System). The knowledge base has more than 200 rules and represents a single application view and the architectural view. Operational performance using CRUDDES has allowed nondatabase users to obtain useful information from the database previously accessible only to an expert database user or the database designer.

Campbell, William J.↗

Authoring, Analyzing, and Monitoring Requirements for a Lift-Plus-Cruise Aircraft

Requirements specification and analysis is widely applied to ensure the correctness of industrial systems in safety critical domains. Requirements are often initially written in natural language, which is highly ambiguous, and as a second step transformed into a language with rigorous semantics for formal analysis. [Question/problem] In this paper, we report on our experience in requirements creation and analysis, as well as run-time monitor generation using the Formal Requirement Elicitation Tool (FRET), on an industrial case study for a Lift-Plus-Cruise concept aircraft. [Principal ideas/results] We study the creation of requirements directly in the structured language of FRET without a prior definition of the same requirements in natural language. We focus on requirements describing state machines and discuss the challenges that we faced, in terms of creating requirements and generating monitors. We demonstrate how realizability, i.e., checking whether a requirements specification can be implemented, is crucial for understanding temporal interdependencies among requirements. [Contribution] Our study is the first complete attempt at using FRET to create industrial, realizable requirements and generate run-time monitors. Insight from lessons learned was materialized into new features in the FRET and JKind analysis frameworks.

Requirements engineering↗