Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Large Language Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

HySafe research priorities workshop report Summary of the workshop organized in cooperation with US DOE and supported by EC JRC in Washington DC November 10-11 2014.

The HySafe research priorities workshop is held on the even years between the International Conference on Hydrogen Safety (ICHS) which is held on the odd years. The research priorities workshop is intended to identify the state-of-the-art in understanding of the physical behavior of hydrogen and hydrogen systems with a focus on safety. Typical issues addressed include behavior of unintended hydrogen releases, transient combustion phenomena, effectiveness of mitigation measures, and hydrogen effects in materials. In the workshop critical knowledge gaps are identified. Areas of research and coordinated actions for the near and medium term are derived and prioritized from these knowledge gaps. The stimulated research helps pave the way for the rapid and safe deployment of hydrogen technologies on a global scale. To support the idea of delivering globally accepted research priorities for hydrogen safety the workshop is organized as an internationally open meeting. In attendance are stakeholders from the academic community (universities, national laboratories), funding agencies, and industry. The industry participation is critically important to ensure that the research priorities align with the current needs of the industry responsible for the deployment of hydrogen technologies. This report presents the results of the HySafe Research Priorities Workshop held in Washing- ton, D.C. on November 10-11, 2014. At the workshop the participants presented updates (since the previous workshop organized two years before in Berlin, Germany) of their research and development work on hydrogen safety. Following the workshop, participants were asked to provide feedback on high-priority topics for each of the research areas discussed and to rank research area categories and individual research topics within these categories. The research areas were ranked as follows (with the percentage of the vote in parenthesis): 1. Quantitative Risk Assessment (QRA) Tools (23%) 2. Reduced Model Tools (15%) 3. Indoor (13%) 4. Unintended Release-Liquid (11%) 5. Unintended Release-Gas (8%) 6. Storage (8%) 7. Integration Platforms (7%) 8. Hydrogen Safety Training (7%) 9. Materials Compatibility/Sensors (7%) 10. Applications (2%) The workshop participants ranked the need for Quantitative Risk Analysis (QRA) tools as the top priority by a large margin. QRA tools enable an informed expert to quantify the risk asso- ciated with a particular hydrogen system in a particular scenario. With appropriate verification and validation such tools will enable: * system designers to achieve a desired level of risk with suitable risk mitigation strategies, * permitting officials to determine if a particular system installation meets the desired risk level (performance based Regulations, Codes, and Standards (RCS) rather than prescrip- tive RCS), and * allow code developers to develop code language based on rigorous and validated physical models, statistics and standardized QRA methodologies. Another important research topic identified is the development of validated reduced physical models for use in the QRA tools. Improvement of the understanding and modeling of specific release phenomena, in particular liquid releases, are also highly ranked research topics. Acknowledgement The International Association HySafe, represented here by the authors, would like to thank all participants of the workshop for their valuable contributions. Particularly appreciated is the active participation of the industry representatives and the steady support by the European Com- mission's Joint Research Centre (JRC). Deep gratitude is owed for the great support by the United States Department of Energy (DOE) Office of Energy Efficiency and Renewable Energy's Fuel Cell Technologies Office (EERE/FCTO) for the organization of the 2014 version of the hydrogen safety research priorities workshop. This page intentionally left blank.

08 HYDROGEN↗
DOE OSTI2022

Recent advances and applications of deep learning methods in materials science

Deep learning (DL) is one of the fastest-growing topics in materials data science, with rapidly emerging applications spanning atomistic, image-based, spectral, and textual data modalities. DL allows analysis of unstructured data and automated identification of features. The recent development of large materials databases has fueled the application of DL methods in atomistic prediction in particular. In contrast, advances in image and spectral data have largely leveraged synthetic data enabled by high-quality forward models as well as by generative unsupervised DL methods. In this article, we present a high-level overview of deep learning methods followed by a detailed discussion of recent developments of deep learning in atomistic simulation, materials imaging, spectral analysis, and natural language processing. For each modality we discuss applications involving both theoretical and experimental data, typical modeling approaches with their strengths and limitations, and relevant publicly available software and datasets. We conclude the review with a discussion of recent cross-cutting work related to uncertainty quantification in this field and a brief perspective on limitations, challenges, and potential growth areas for DL methods in materials science.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Computationally Efficient Learning of Quality Controlled Word Embeddings for Natural Language Processing

Deep learning (DL) has been used for many natural language processing (NLP) tasks due to its superior performance as compared to traditional machine learning approaches. In DL models for NLP, words are represented using word embeddings, which capture both semantic and syntactic information in text. However, 90-95% of the DL trainable parameters are associated with the word embeddings, resulting in a large storage or memory footprint. Therefore, reducing the number of word embedding parameters is critical, especially with the increase of vocabulary size. In this work, we propose a novel approximate word embeddings approach for convolutional neural networks (CNNs) used for text classification tasks. The proposed approach significantly reduces the number of model trainable parameters without noticeably sacrificing in computing performance accuracy. Compared to other techniques, our proposed word embeddings technique does not require modifications to the DL model architecture. We evaluate the performance of the the proposed word embeddings on three classification tasks using two datasets, composed of Yelp and Amazon reviews. The results show that the proposed method can reduce the number of word embeddings parameters by 98% and 99% for the Yelp and Amazon datasets respectively, with no drop in computing accuracy.

Alawad, Mohammed↗
NASA NTRS2014

Terminal Area Simulation System User's Guide - Version 10.0

The Terminal Area Simulation System (TASS) is a three-dimensional, time-dependent, large eddy simulation model that has been developed for studies of wake vortex and weather hazards to aviation, along with other atmospheric turbulence, and cloud-scale weather phenomenology. This document describes the source code for TASS version 10.0 and provides users with needed documentation to run the model. The source code is programed in Fortran language and is formulated to take advantage of vector and efficient multi-processor scaling for execution on massively-parallel supercomputer clusters. The code contains different initialization modules allowing the study of aircraft wake vortex interaction with the atmosphere and ground, atmospheric turbulence, atmospheric boundary layers, precipitating convective clouds, hail storms, gust fronts, microburst windshear, supercell and mesoscale convective systems, tornadic storms, and ring vortices. The model is able to operate in either two- or three-dimensions with equations numerically formulated on a Cartesian grid. The primary output from the TASS is time-dependent domain fields generated by the prognostic equations and diagnosed variables. This document will enable a user to understand the general logic of TASS, and will show how to configure and initialize the model domain. Also described are the formats of the input and output files, as well as the parameters that control the input and output.

Switzer, George F.↗
NASA NTRS2017

Marve: Measurement Context Extraction from Text

We propose Marve, a system for extracting measurement values, units, and related words from natural language text. Marve uses conditional random fields (CRF) to identify measurement values and units, followed by a rule-based system to find related entities, descriptors and modifiers within a sentence. Sentence tokens are represented by an undirected graphical model, and rules are based on part-of-speech and word dependency patterns connecting values and units to contextual words. Marve is unique in its focus on measurement context and early experimentation demonstrates Marve’s ability to generate high-precision extractions with strong recall. We also discuss Marve’s role in justifying NASA JPL’s proposed HyspIRI mission, a hyper spectral infrared imaging satellite that will study the world’s ecosystems. In general, our work with HyspIRI demonstrates the value of semantic measurement extractions in characterizing quantitative discussion contained in large corpuses of natural language text. These extractions accelerate broad-cross cu ing literature surveys and expose researchers and scientists new algorithmic approaches and experimental nuances. They also facilitate identification of scientific opportunities enabled by HyspIRI leading to more informed scientific investment and research.

Mattmann, Chris A.↗
DOE OSTI2022

Accelerating Hydraulic Fracture Imaging by Deep Transfer Learning

Deep transfer learning has a great success story in computer vision (CV), natural language processing, and many other fields. In this communication, we are going to push forward the deep transfer learning to the hydraulic fracture imaging problem by proposing a two-step approach: 1) train a convolutional neural network (CNN) to reconstruct target geometries by a relatively large amount of approximated field patterns generated from a simplified model and 2) fine-tune the top layers of transferred CNN by a small amount of true field patterns generated through a full model. The advantages include the rapid generation of large amount of data through the simplified model and the high reconstruction accuracy through a careful design of the deep transfer learning. Here, the CNN trained through the deep transfer learning can accurately reconstruct the lateral extent and direction of fractures with unseen conductivity and white Gaussian noise, showing a notable acceleration/accuracy enhancement over the previous CNN trained by mixing a small number of true data into the approximated data as data augmentation.

02 PETROLEUM↗
NASA NTRS1992

Real-time processing of radar return on a parallel computer

NASA is working with the FAA to demonstrate the feasibility of pulse Doppler radar as a candidate airborne sensor to detect low altitude windshears. The need to provide the pilot with timely information about possible hazards has motivated a demand for real-time processing of a radar return. Investigated here is parallel processing as a means of accommodating the high data rates required. A PC based parallel computer, called the transputer, is used to investigate issues in real time concurrent processing of radar signals. A transputer network is made up of an array of single instruction stream processors that can be networked in a variety of ways. They are easily reconfigured and software development is largely independent of the particular network topology. The performance of the transputer is evaluated in light of the computational requirements. A number of algorithms have been implemented on the transputers in OCCAM, a language specially designed for parallel processing. These include signal processing algorithms such as the Fast Fourier Transform (FFT), pulse-pair, and autoregressive modelling, as well as routing software to support concurrency. The most computationally intensive task is estimating the spectrum. Two approaches have been taken on this problem, the first and most conventional of which is to use the FFT. By using table look-ups for the basis function and other optimizing techniques, an algorithm has been developed that is sufficient for real time. The other approach is to model the signal as an autoregressive process and estimate the spectrum based on the model coefficients. This technique is attractive because it does not suffer from the spectral leakage problem inherent in the FFT. Benchmark tests indicate that autoregressive modeling is feasible in real time.

Aalfs, David D.↗
DOE OSTI2021

QLiG: Query Like a Graph For Subgraph Matching

A graph is a natural and flexible modeling approach to represent entities and relationships between them in real-world. A Knowledge Graphs (KG) is a specialized graph with formal and structured representation of facts, relationships, annotated with semantic descriptions. Subgraph matching is one of the fundamental graph problems to identify relationships, interactions and activities of interest within a large graph. A query specification is a collection of abstract components, operations, and constraints to express a pattern. The specification can be implemented in different ways based on underlying data model. Various graph query specifications have been developed over the years and have led to the development of different open-sourced and vendor-specific query languages. Such specification are modeled as an extension of relational algebra used to develop relational query languages such as SQL. Such relational concepts do not inherently support graph queries. There is a need to represent graph queries in terms on graph-based components to expedite query construction by non-database experts. We present a graph-based query approach QLiG (pronounced cleeg), to perform subgraph matching in Labeled Property Graph. We present the query specifications, salient features, and a use case to show functional examples.

Purohit, Sumit↗
DOE OSTI2018

A library of building occupant behaviour models represented in a standardised schema

During the past four decades, a significant body of literature has reported on the impacts of occupant behaviour (OB) on building technologies, operation, and energy consumption. A large number of data-driven behavioural models have been developed based on field data. These models lack standardisation and consistency, leading to difficulties in applications and comparison. To address this concern, an ontology was developed using the drivers-needs-actions-systems (DNAS) framework. Recent work has been carried out to implement the theoretical DNAS framework into an eXtensible Markup Language (XML) schema, titled ‘occupant behaviour XML’ (obXML) which is a practical implementation of OB models that can be integrated into building performance simulation (BPS) programs. This paper presents a newly developed library of OB models represented in the standardised obXML schema format. This library provides ready-to-use examples for BPS users to employ more accurate occupant representation in their energy models. The library, which contains an initial effort of 52 OB models, was made publicly available for the BPS community. As part of the library development process, limitations of the obXML schema were identified and addressed, and future improvements were proposed. Authors hope that by compiling this library building, energy modellers from all over the world can enhance their BPS models by integrating more accurate and robust OB patterns.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2018

Introduction to NASA Goddard Workshop on Artificial Intelligence

Artificial Intelligence (AI) is a collection of advanced technologies that allows machines to think and act, both humanly and rationally, through sensing, comprehending, acting and learning. AI's foundations lie at the intersection of several traditional fields Philosophy, Mathematics, Economics, Neuroscience, Psychology and Computer Science. Although the inception of AI started in the 1950's, it has recently made a strong comeback in all aspects of society and all over the world; this is mainly due to the timely combination of increased data volumes, advanced and mature algorithms, and improvements in computing power and storage. Current AI applications include big data analytics, robotics, intelligent sensing, assisted decision making, and speech recognition just to name a few.This workshop will be investigating how AI technologies can be adapted or developed to address the following challenges: Discover events of interest and correlations in large amounts of science data; improve the outcomes of science modeling and data assimilation using improved data processing, integration, and analysis. Design advisors for mission planning and operations, including anomaly detection and spacecraft health monitoring. Develop tools for engineering support, including advanced manufacturing, orbit determination, new component design and system engineering. Customize intelligent user interfaces, including visual analytics and natural language processing.

Le Moigne, Jacqueline↗
DOE OSTI2023

EXSCLAIM!: Harnessing materials science literature for self-labeled microscopy datasets

This work introduces the EXSCLAIM! toolkit for the automatic extraction, separation, and caption-based natural language annotation of images from scientific literature. EXSCLAIM! is used to show how rule-based natural language processing and image recognition can be leveraged to construct an electron microscopy data set containing thousands of keyword-annotated nanostructure images. Moreover, it is demonstrated how a combination of statistical topic modeling and semantic word similarity comparisons can be used to increase the number and variety of keyword annotations on top of the standard annotations from EXSCLAIM! With large-scale imaging datasets constructed from scientific literature, users are well positioned to train neural networks for classification and recognition tasks specific to microscopy-tasks often otherwise inhibited by a lack of sufficient annotated training data.

36 MATERIALS SCIENCE↗
NASA NTRS2005

Software for Engineering Simulations of a Spacecraft

Spacecraft Engineering Simulation II (SES II) is a C-language computer program for simulating diverse aspects of operation of a spacecraft characterized by either three or six degrees of freedom. A functional model in SES can include a trajectory flight plan; a submodel of a flight computer running navigational and flight-control software; and submodels of the environment, the dynamics of the spacecraft, and sensor inputs and outputs. SES II features a modular, object-oriented programming style. SES II supports event-based simulations, which, in turn, create an easily adaptable simulation environment in which many different types of trajectories can be simulated by use of the same software. The simulation output consists largely of flight data. SES II can be used to perform optimization and Monte Carlo dispersion simulations. It can also be used to perform simulations for multiple spacecraft. In addition to its generic simulation capabilities, SES offers special capabilities for space-shuttle simulations: for this purpose, it incorporates submodels of the space-shuttle dynamics and a C-language version of the guidance, navigation, and control components of the space-shuttle flight software.

Shireman, Kirk↗
NASA NTRS1997

Tolerant (parallel) Programming

In order to be truly portable, a program must be tolerant of a wide range of development and execution environments, and a parallel program is just one which must be tolerant of a very wide range. This paper first defines the term "tolerant programming", then describes many layers of tools to accomplish it. The primary focus is on F-Nets, a formal model for expressing computation as a folded partial-ordering of operations, thereby providing an architecture-independent expression of tolerant parallel algorithms. For implementing F-Nets, Cooperative Data Sharing (CDS) is a subroutine package for implementing communication efficiently in a large number of environments (e.g. shared memory and message passing). Software Cabling (SC), a very-high-level graphical programming language for building large F-Nets, possesses many of the features normally expected from today's computer languages (e.g. data abstraction, array operations). Finally, L2(sup 3) is a CASE tool which facilitates the construction, compilation, execution, and debugging of SC programs.

DiNucci, David C.↗
NASA NTRS2003

Computational Aspects of Data Assimilation and the ESMF

The scientific challenge of developing advanced data assimilation applications is a daunting task. Independently developed components may have incompatible interfaces or may be written in different computer languages. The high-performance computer (HPC) platforms required by numerically intensive Earth system applications are complex, varied, rapidly evolving and multi-part systems themselves. Since the market for high-end platforms is relatively small, there is little robust middleware available to buffer the modeler from the difficulties of HPC programming. To complicate matters further, the collaborations required to develop large Earth system applications often span initiatives, institutions and agencies, involve geoscience, software engineering, and computer science communities, and cross national borders.The Earth System Modeling Framework (ESMF) project is a concerted response to these challenges. Its goal is to increase software reuse, interoperability, ease of use and performance in Earth system models through the use of a common software framework, developed in an open manner by leaders in the modeling community. The ESMF addresses the technical and to some extent the cultural - aspects of Earth system modeling, laying the groundwork for addressing the more difficult scientific aspects, such as the physical compatibility of components, in the future. In this talk we will discuss the general philosophy and architecture of the ESMF, focussing on those capabilities useful for developing advanced data assimilation applications.

daSilva, A.↗
DOE OSTI2019

Inferring Convolutional Neural Networks' Accuracies from Their Architectural Characterizations

The challenge of choosing an appropriate convolutional neural network (CNN) architecture for specific applications and different data sets is still poorly understood in the literature. This is problematic, since CNNs have shown strong promise for analyzing scientific data from many domains including particle imaging detectors. In this paper, we proposed a systematic language that is useful for comparison between different CNN's architectures before training time. This helps us predict whether a network can perform better than a certain threshold accuracy before training up to 70% accuracy using simple machine learning models. Additionally, we found a coefficient of determination of 0.966 for an Ordinary Least Squares model in a regression task to predict accuracy of a large population of networks.

Hoang, Duc↗
DOE OSTI2017

RuleBuilder

RuleBuilder is a tool for drawing graphs represented by the BioNetGen rule-based modeling language (BNGL), a domain-specific language used to formulate mathematical models of biochemical interaction networks. This framework provides an intuitive plain-text representation of such networks using a graph-based formalism. The interactions between the model's components are graph-rewriting rules that denote the minimum context needed for the interaction to occur. In some cases the context required for reactions to occur can grow quite large, and the lengthy textual description of the interaction can be difficult to interpret or debug. RuleBuilder is designed to make this process simpler through visualization of individual rule or pattern strings. The software assists in the reading of existing models by converting the BNGL strings of interest into a graph-based representation composed of nodes and edges. RuleBuilder also enables the user to build the visual representation directly from its interface, composing the BNGL string as components and operators are added to the graphical object.

Fricke, Matthew↗
DOE OSTI2023

Towards POI-based large-scale land use modeling: spatial scale, semantic granularity, and geographic context

The combination of spatial distribution, semantic characteristics, and sometimes temporal dynamics of POIs inside a geographic region can capture its unique land use characteristics. Most previous studies on POI-based land use modeling research focused on one geographic region and select one spatial scale and semantic granularity for land use characterization. There is a lack of understanding on the impact of spatial scale, semantic granularity, and geographic context on POI-based land use modeling, particularly large-scale land use modeling. In this study, we developed a scalable POI-based land use modeling framework and examined the impact of these three factors on POI-based land use characterization using data from three geographic regions. We developed a unified semantic representation framework for POI semantics that can help fuse heterogeneous POI data sources. Then, by combining POIs with a neural network language model, we developed a spatially explicit approach to learn the embedding representation of POIs and AOIs. We trained multiple supervised classifiers using AOI embeddings as input features to predict AOI land use at different semantic granularities. The classification performance of different land use classes was analyzed and compared across three geographic regions to identify the semantic representativeness of POI-based AOI embedding and the impact of geographic context.

58 GEOSCIENCES↗
DOE OSTI2025

NEAR: Neural Embeddings for Amino acid Relationships

Protein language models (PLMs) have recently demonstrated potential to supplant classical protein database search methods based on sequence alignment, but are slower than common alignment-based tools and appear to be prone to a high rate of false labeling. Here, we present NEAR, a method based on neural representation learning that is designed to improve both speed and accuracy of search for likely homologs in a large protein sequence database. NEAR’s ResNet embedding model is trained using contrastive learning guided by trusted sequence alignments. It computes per-residue embeddings for target and query protein sequences, and identifies alignment candidates with a pipeline consisting of residue-level k-NN search and a simple neighbor aggregation scheme. Tests on a benchmark consisting of trusted remote homologs and randomly shuffled decoy sequences reveal that NEAR substantially improves accuracy relative to state-of-the-art PLMs, with lower memory requirements and faster embedding and search speed. While these results suggest that the NEAR model may be useful for standalone homology detection with increased sensitivity over standard alignment-based methods, in this manuscript we focus on a more straightforward analysis of the model’s value as a high-speed pre-filter for sensitive annotation. In that context, NEAR is at least 5x faster than the pre-filter currently used in the widely-used profile hidden Markov model (pHMM) search tool HMMER3, and also outperforms the pre-filter used in our fast pHMM tool, nail.

59 BASIC BIOLOGICAL SCIENCES↗