Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “query processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Expandable Log Analyzing Framework

Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.

Clark, Dylan [Unlisted, IL]↗
DOE OSTI2026

Expandable Log Analyzing Framework

Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.

Clark, Dylan [Unlisted, IL]↗
DOE OSTI2026

Database-Agnostic Log Analysis and Monitoring Framework

Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.

Clark, Dylan [Unlisted, US, IL; Fermilab]↗
DOE OSTI2025

Text Mining for Process–Structure–Properties Relationships in Metals

With the advent of large language models (LLMs), the vast unstructured text within millions of academic papers is increasingly accessible for materials discovery—although significant challenges remain. While LLMs offer promising few- and zero-shot learning capabilities, particularly valuable in the materials domain where expert annotations are scarce, general-purpose LLMs often fail to address key materials-specific queries without further adaptation. To bridge this gap, fine-tuning LLMs on human-labeled data is essential for effective structured knowledge extraction (Liu in The Importance of Human-Labeled Data in the Era of LLMs, 2023). Here, in this study, we introduce a novel annotation schema designed to extract generic process–structure–properties relationships from scientific literature. We demonstrate the utility of this approach using a dataset of 128 abstracts, with annotations drawn from two distinct domains: high-temperature materials (Domain I) and uncertainty quantification in simulating materials microstructure (Domain II). Initially, we developed a conditional random field (CRF) model based on MatBERT—a domain-specific BERT variant—and evaluated its performance on Domain I. Subsequently, we compared this model with a fine-tuned LLM (GPT-4o from OpenAI) under identical conditions. Our results indicate that fine-tuning LLMs can significantly improve entity extraction performance over the BERT-CRF baseline on Domain I. However, when additional examples from Domain II were incorporated, the performance of the BERT-CRF model became comparable to that of the GPT-4o model. These findings underscore the potential of our schema for structured knowledge extraction and highlight the complementary strengths of both modeling approaches.

Materials science↗
DOE OSTI2025

Paired Neural Network for Matching Experimental and Predicted Infrared Spectra

Here, we present a novel machine learning (ML)-based scoring technique for determining the similarity between experimental and predicted infrared (IR) spectra for identification purposes. IR spectroscopy is a powerful technique used to identify the molecular structure and composition of a sample by measuring the unique vibrational frequency pattern of the molecule’s functional groups. Molecular identifications are often made by comparing experimental and reference spectra. However, the limited number of reference spectra available in spectral libraries can confound the identification process. Alternative identification procedures rely on in silico techniques to simulate spectra for a wide range of molecules. However, scoring spectral similarity between an experimental query and computationally predicted reference remains a significant challenge. Our proposed ML-based scoring technique overcomes these barriers by accurately and efficiently determining spectral similarity.

Neural Network↗
DOE OSTI2019

One SQL to Rule Them All - an Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables

Real-time data analysis and management are increasingly critical for today's businesses. SQL is the de facto lingua franca for these endeavors, yet support for robust streaming analysis and management with SQL remains limited. Many approaches restrict semantics to a reduced subset of features and/or require a suite of non-standard constructs. Additionally, use of event timestamps to provide native support for analyzing events according to when they actually occurred is not pervasive, and often comes with important limitations. We present a three-part proposal for integrating robust streaming into SQL, namely: (1) time-varying relations as a foundation for classical tables as well as streaming data, (2) event time semantics, (3) a limited set of optional keyword extensions to control the materialization of time-varying query results. We show how with these minimal additions it is possible to utilize the complete suite of standard SQL semantics to perform robust stream processing. We motivate and illustrate these concepts using examples and describe lessons learned from implementations in Apache Calcite, Apache Flink, and Apache Beam. We conclude with syntax and semantics of a concrete proposal for extensions of the SQL standard and note further areas of exploration.

Begoli, Edmon↗
DOE OSTI2022

Marine Energy Environmental Toolkit for Permitting and Licensing

The permitting and licensing process for marine energy in the U.S. could take up to 7 years for even small demonstrations or pilot studies despite existing resources like Tethys, MarineCadstre, and the FERC E-Library. The goal of the Marine Energy Environmental Toolkit Project was to increase regulators, developers, and marine energy innovators’ understanding of potential environmental effects of marine energy conversion devices deployments and provide information to make efficient and effective determinations during permitting and licensing; develop a one-stop shop for existing environmental, spatial, regulatory, and scientific data that allows users to access disparate sources of data through a series of tags and spatial queries. The Toolkit (marineenergy.app) builds upon prior U.S. Department of Energy (DOE) Water Power Technologies Office (WPTO) investments by pulling from WPTO-funded and outside resources to facilitate the permitting and licensing process, inform technology developers of required permits, and disseminate the latest knowledge of the environmental effects of these devices to regulators, developers, and marine energy innovators and stakeholders.

16 TIDAL AND WAVE POWER↗
DOE OSTI2022

CAMERA: A method for cost-aware, adaptive, multifidelity, efficient reliability analysis

Estimating probability of failure in aerospace systems is a critical requirement for flight certification and qualification. Failure probability estimation involves resolving tails of probability distributions, and Monte Carlo sampling methods are intractable when expensive high-fidelity simulations have to be queried. Here, we propose a method to use models of multiple fidelities that trade accuracy for computational efficiency. Specifically, we propose the use of multifidelity Gaussian process models to efficiently fuse models at multiple fidelity, thereby offering a cheap surrogate model that emulates the original model at all fidelities. Furthermore, we propose a novel sequential acquisition function based experiment design framework that can automatically select samples from appropriate fidelity models to make predictions about quantities of interest at the highest fidelity. We use our proposed approach in an importance sampling setting and demonstrate our method on the failure level set and probability estimation on synthetic test functions and two real-world applications, namely, the reliability analysis of a gas turbine engine blade using a finite element method and a transonic aerodynamic wing test case using Reynolds-averaged Navier-Stokes equations. We show that our method predicts the failure boundary and probability more accurately and at a fraction of the computational cost compared with using just a single expensive high-fidelity model. Finally, we show that our sequential approach is guaranteed to asymptotically converge to the true failure boundary with high probability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Rules of Use Report Engineering evaluations of nuclear material storage containers against the packaging requirements at TA-55

A set of obligations in the yearly storage container surveillance program, ensures that a retrospective approach is applied with loaded containers in inventory to confirm compliance with all “users” to ensure nuclear material storage container requirements are being followed. Validation that packaged containers are properly used, within a specific set of container types prescribed “bounding conditions of use,” is based on meeting requirements as identified in operating procedures associated with TA55-DOP-091, TA-55 Nuclear Material Packaging, and PA-RD-01022, Nuclear Material Packaging Requirements. The surveillance plan obligation requires the application of local area nuclear material accountability software or (LANMAS) to produce queries. The data queries are used to assess attributes of containers in storage these investigations are conducted on a bi-annual basis. This rules of use (ROU) compliance process ensures proper usage of storage containers as containment systems and therefore provides effective worker protection. It is structured as a supporting effort by implementing aspects of PA-AP-01207, Nuclear Material Container Safety Management at TA-55.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

Engineering evaluations of nuclear material storage containers against the packaging requirements at TA-55 (FY22 Rules of Use Report)

A set of obligations in the yearly storage container surveillance program, ensures that a retrospective approach is applied with loaded containers in inventory to confirm compliance with all “users” to ensure nuclear material storage container requirements are being followed. Validation that packaged containers are properly used, within a specific set of container types prescribed “bounding conditions of use,” is based on meeting requirements as identified in operating procedures associated with TA55-DOP-091, TA-55 Nuclear Material Packaging, and PA-RD-01022, Nuclear Material Packaging Requirements. The surveillance plan obligation requires the application of local area nuclear material accountability software or (LANMAS) to produce queries. The data queries are used to assess attributes of containers in storage these investigations are conducted on a bi-annual basis. This rules of use (ROU) compliance process ensures proper usage of storage containers as containment systems and therefore provides effective worker protection. It is structured as a supporting effort by implementing aspects of PA-AP-01207, Nuclear Material Container Safety Management at TA-55.

36 MATERIALS SCIENCE↗
DOE OSTI2022

A general quantum algorithm for open quantum dynamics demonstrated with the Fenna-Matthews-Olson complex

Using quantum algorithms to simulate complex physical processes and correlations in quantum matter has been a major direction of quantum computing research, towards the promise of a quantum advantage over classical approaches. In this work we develop a generalized quantum algorithm to simulate any dynamical process represented by either the operator sum representation or the Lindblad master equation. We then demonstrate the quantum algorithm by simulating the dynamics of the Fenna-Matthews-Olson (FMO) complex on the IBM QASM quantum simulator. This work represents a first demonstration of a quantum algorithm for open quantum dynamics with a moderately sophisticated dynamical process involving a realistic biological structure. We discuss the complexity of the quantum algorithm relative to the classical method for the same purpose, presenting a decisive query complexity advantage of the quantum approach based on the unique property of quantum measurement.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Multi-Agent Simulation Based Framework for Power Restoration Time Estimation at Distribution Level

The growing frequency of power outages has prompted increased interest in developing a more resilient power grid that can quickly recover from weather-related damage. At the distribution level, power restoration is a complex, multi-stage process involving multiple response entities. Providing utility stakeholders, government regulators, and the public with information about outage duration and estimated time to restoration is crucial. The research employs a multi-agent simulation approach, which allows for the simulation of decision-making behaviors among different entities and the incorporation of various uncertainties. Specifically, the study uses the open-source simulation package Mesa-Geo in conjunction with the Python language and constructs a road network using the open-source network extension pgRouting for routing queries. The research design includes several experiments focused on Florida as a case study, comparing repair crew sizes, power outage numbers, and road damage scenarios. The findings could offer valuable managerial guidance on resource allocation in the restoration process.

Chen, Yang↗
DOE OSTI2022

Predicting Search Task Difficulty through a Discrete‐Time Action Log Representation on Spectrum Kernel

ABSTRACT Predicting perceived difficulty on a web search task is an open problem in the interactive information retrieval field. A common approach to tackle it, is through features obtained from full search sessions, which are then used to train classification models. In this poster we attempt to predict perceived task difficulty at different stages of the search process. To do so, we use the spectrum kernel for support vector machine (SVM) classification. Our preliminary results suggest that by using behavioral data from the first query segment, it is possible to provide timely classifications of whether a search task is perceived as hard or easy.

Gacitúa, Daniel↗
DOE OSTI2024

DQL (Django Query Logger) [SWR-24-96]

The Django Query Logger (DQL) is a tool that allows developers of Django web applications to stream the raw database queries that are being executed in real-time from any Django application for review, profiling, filtering, formatting, and analysis. Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or Alliance for Sustainable Energy, LLC. The views and opinions of authors expressed in the available or referenced documents do not necessarily state or reflect those of the United States Government or Alliance.

Swindler, Alexander↗
DOE OSTI2024

The Integration and Mapping of an Open-Source National Well Resource to Inform Geologic Carbon Storage Site Selection and Risk Prevention: The CO2-Locate Database

Geologic carbon storage (GCS) offers a way to capture and permanently store CO₂ from fossil fuel operations in underground geologic structures, aiding in the transition to a carbon-neutral energy economy. However, CO₂ injection sites can experience gas leakage through existing wells that penetrate storage reservoirs, making knowledge of well locations and characteristics crucial for permitting, infrastructure reusability, and risk assessment in GCS. Currently, public wellbore data from state, federal, and tribal entities are inconsistent and fragmented, with gaps and redundancies. To address this, the National Energy Technology Laboratory (NETL) developed CO2-Locate, an open-source, geospatial database and online application. CO2-Locate integrates over 50 data sources from federal, state, and tribal entities, creating a standardized national well database. Funded by the Bipartisan Infrastructure Law, the database is publicly available through the Energy Data eXchange (EDX) and viewable via the CO2-Locate web mapping application. This tool allows users to query, filter, and visualize well data to support GCS planning, permitting, and risk assessments. This presentation covers the methods used to create CO2-Locate, including data acquisition, processing, attribute mapping, and integration, much of which is automated for future updates. The web mapping application and its role in GCS site selection will also be discussed.

Tetteh, Daniel A.↗
DOE OSTI2020

Building Interfaces with Echo Using R

The following report details the functionality that was developed to directly query the MongoDB and read in echo records from within R, and the ability to create a cinema database directly from echo records. Both of these efforts used the data collected under the MINOS project for development. The nuclear facility under consideration is the High-Flux Isotope Reactor (HFIR) and co-located Radiochemical Engineering Development Center (REDC) at Oak Ridge National Laboratory in Oak Ridge, TN. HFIR is an 85 MW research reactor and is used primarily for production of a medical radioisotopes, material irradiation experiments, neutron activation, and neutron scattering. Targets for the reactor are fabricated and processed and dissolved at REDC, in addition to other glove-box and hot-cell type activities. An overhead view of the facility is provided. The rest of the report is organized as follows: Section 2 describes how to query the MongoDB and load in echo records from within R, with background information on the database type, MongoDB, and the file format of echo records, hdf5, as well as lessons learned during the development phase. Section 3 details the functionality and usage of creating a cinema database directly from echo records within R. A series of appendixes display all the R code (Section 5), some example scripts where the code is used (Section 6), and useful links for further exploration (Section 7).

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

A General Framework for Progressive Data Compression and Retrieval

In scientific simulations, observations, and experiments, the transfer of data to and from disk and across networks has become a major bottleneck for data analysis and visualization. Compression techniques have been employed to tackle this challenge, but traditional lossy methods often demand conservative error tolerances to meet the numerical accuracy requirements of both anticipated and unknown data analysis tasks. Progressive data compression and retrieval has emerged as a promising solution, where each analysis task dictates its own accuracy needs. However, few analysis algorithms inherently support progressive data processing, and adapting compression techniques, file formats, client/server frameworks, and APIs to support progressivity can be challenging. Here, this paper presents a framework that enables progressive-precision data queries for any data compressor or numerical representation. Our strategy hinges on a multi-component representation that successively reduces the error between the original and compressed field, allowing each field in the progressive sequence to be expressed as a partial sum of components. We have implemented this approach with four established scientific data compressors and assessed its effectiveness using real-world data sets from the SDRBench collection. The results show that our framework competes in accuracy with the standalone compressors it is based upon. Additionally, (de)compression time is proportional to the number of components requested by the user. Finally, our framework allows for fully lossless compression using lossy compressors when a sufficient number of components are employed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Redox Biochemistry of Energy Conservation in Methanogens and their Syntrophic Partners

The primary objective of this project was to uncover previously unknown modes by which living organisms harvested energy. Specifically, it queried microbial systems, which were once thought to not be able to utilize oxygen. Through the use of cutting-edge approaches, we were able to identify novel mechanistic processes that allow organisms to thrive under varying environmental conditions. This constitutes a paradigm shift in the field. Our work has opened new vistas for expanding the horizons of energy conservation in biological systems, as well as shed light on green energy innovations.

59 BASIC BIOLOGICAL SCIENCES↗