Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “language models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

SysCaps (Language Interfaces for Simulation Surrogates of Complex Systems) [SWR-24-97]

You've found the official code repository for the paper "SysCaps: Language Interfaces for Simulation Surrogates of Complex Systems," presented at the Foundation Models for Science: Progress, Opportunities, and Challenges workshop at NeurIPS 2024. Our paper conjectures that interfaces (both text templates as well as conversational) makes interacting with simulation surrogate models for complex systems more intuitive and accessible for both non-experts and experts. "System captions", or SysCaps, are text-based descriptions of systems based on information contained in simulation metadata. Our paper's goal is to train multimodal regression models that take text inputs (SysCaps) and timeseries inputs (exogenous system conditions such as hourly weather) and regress timeseries simulation outputs (e.g. hourly building energy consumption). The experiments in our paper with building and wind farm simulators, which can be reproduced using this codebase, aim to help us understand whether a) accurate regression in this setting is possible and b) if so, how well can we do it. Paper: https://arxiv.org/abs/2405.19653

Emami, Patrick↗
DOE OSTI2025

Exploring Capability of Multimodal Foundation Model for Image-based Fault Detection of Photovoltaic Modules

Multimodal Foundation Model (MFM), like ChatGPT and Gemini, have emerged as powerful tools for their exceptional natural language processing capabilities and their emerging potential in image analysis. This paper investigates the application of MFMs for photovoltaic (PV) fault detection through image analysis, focusing on ChatGPT 4.0 and Gemini 1.5 Pro. Three types of PV images and the corresponding common PV faults are detected: bird droppings using visible images, cell cracks via electroluminescence (EL) images, and hotspots using infrared (IR) images. Among the two models, Gemini 1.5 Pro demonstrated superior performance, achieving near-perfect results with an average F1 score of 0.97, consistently outperforming ChatGPT 4.0 in accuracy and reliability. Unlike traditional machine learning (ML) models, MFMs can operate in a zero shot manner that does not require additional training by the user, and the input images are not limited by size, angle, scope, or PV technology. The strong adaptability and user-friendliness make MFM a promising tool for analyzing PV images and advancing health monitoring for PV modules.

Li, Baojie↗
DOE OSTI2023

Building heights and urban canopy parameters for urban modeling

GLObal Building heights for Urban Studies (UT-GLOBUS) is a random forest model based framework that provides a level-of-detail-1 (LoD-1) building height dataset. The primary objective of UT-GLOBUS is not to precisely predict the height and footprint of individual buildings, but rather to offer a functional framework for generating building level information using open-source datasets for modeling applications. Specifically, UT-GLOBUS is tailored to meet the requirements of deriving urban canopy parameters (UCPs) for the multi-layer model within the Weather Research and Forecasting (WRF) model and building heights for the SOLWEIG and SUEWS model. Building-level data is accessible in vector file format (GeoPackage: .gpkg), which can be converted into raster file format (geoTIFF). The vector files employ the Universal Transverse Mercator (UTM) projection. The vector files are compatible with GIS platforms like QGIS and ArcGIS, and can be imported for analysis using programming languages such as Python. We are also providing UCPs required by the multi-layer urban model in the urban WRF in binary file format. Additionally, we provide the urban fractions calculated using ESA world cover dataset (https://esa-worldcover.org/en) for WRF model in binary file format. These binary files can be directly incorporated into the WRF pre-processing system (WPS).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Informing Plant Asset Reliability and Availability Through AI-Driven Analysis of Operator Logs

The availability and reliability of nuclear power plant (NPP) structures, systems, and components (SSCs) are critical parameters for NPP safety. Tracking these parameters is necessary but costly and labor-intensive, requiring the collection and evaluation of SSC event data such as shutdowns, startups, and failures. To show how these events are needed for the parameters an example is given: one measure of reliability is based on the number of equipment failure events and the number of run hours (i.e., the time from a startup event to a shutdown event). Here, this work investigates using artificial intelligence (AI) to mine NPP operator log entry texts for SSC event data. Four AI approaches were explored for identifying these events, including natural language processing (NLP) methods, generative AI, generative AI combined with NLP, and topic modeling. A key challenge addressed with all four approaches is the brevity of operator log entries. Among these four a neural network–based NLP method was shown to be the most promising for this application, achieving F1 scores of 86.0% for shutdowns, 92.2% for startups, and 80.4% for failures on a subject-matter-expert-curated dataset from NPP operator logs, compared to a baseline of 66.6% for a random classifier. This shows that NLP methods can perform better than generative AI. Additionally, the NLP methods combined with generative AI were shown to perform better than generative AI alone. Generative AI was most successful at providing the background information for the NLP methods to use. This work demonstrates the potential to use AI to automate parameter collection from NPP operator log entries and other records.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Scheduling and Performance of Asynchronous Tasks in Fortran 2018 with FEATS

Most parallel scientific programs contain compiler directives (pragmas) such as those from OpenMP (Hermanns in Parallel programming in Fortran 95 using openMP, 2002. School of Aeronautical Engineering, Universidad Politécnica de Madrid, España, 2011), explicit calls to runtime library procedures such as those implementing the Message Passing Interface (MPI) (in A message-passing interface standard version 4.0, 2021. https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf), or compiler-specific language extensions such as those provided by CUDA (Ruetsch and Fatica in CUDA Fortran for scientists and engineers: best practices for efficient CUDA Fortran programming, Elsevier, 2013). By contrast, the recent Fortran standards empower developers to express parallel algorithms without directly referencing lower-level parallel programming models (Numrich in Parallel programming with co-arrays, CRC Press, 2018, and Curcic in Modern Fortran: building efficient parallel applications, Manning Publications, 2020). Fortran’s parallel features place the language within the Partitioned Global Address Space (PGAS) class of programming models. When writing programs that exploit data parallelism, application developers often find it straightforward to develop custom parallel algorithms. Problems involving complex, heterogeneous, staged calculations, however, pose much greater challenges. Such applications require careful coordination of tasks in a manner that respects dependencies prescribed by a directed acyclic graph. When rolling one’s own solution proves difficult, extending a customizable framework becomes attractive. Further, the paper presents the design, implementation, and use of the Framework for Extensible Asynchronous Task Scheduling (FEATS), which we believe to be the first task scheduling tool written in modern Fortran. We describe the benefits and compromises associated with choosing Fortran as the implementation language, and we propose ways in which future Fortran standards can best support the use case in this paper.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Long Reach Robotic Arm

The Accelerator Division at Fermilab began a robotics program, known as the AD Robotics Initiative. A multi-axis robotic arm was proposed to be included in the program. This summer internship, our team began the planning and design for this project, called the Long Reach Robotic Arm. The problem is that critical systems around the magnets require analysis when water leaks cause them to fail. However, the tightly compacted parts and the quadrupole magnet, weighing approximately 1,000 pounds, cause access viewing problems. The process of having to dismantle and remove the magnet is time-consuming; it is not an efficient solution. The goal for the Long Reach Robotic Arm was to extend and reach behind the component of the Main Injector while "snaking" around and avoiding obstructions. At the start of the project, my teammate, Amanda Hoeksema, and I coded a program in the C language that will determine the number of links and their respective lengths for the robot arm. After, we began the 3D CAD model of the robot arm, base, and counterweight system. The design of the Long Reach Robotic Arm provided a learning experience in learning how to code in the C programming language, using SDL libraries and understanding mechanical engineering.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

MOFSimplify, machine learning models with extracted stability data of three thousand metal–organic frameworks

Abstract We report a workflow and the output of a natural language processing (NLP)-based procedure to mine the extant metal–organic framework (MOF) literature describing structurally characterized MOFs and their solvent removal and thermal stabilities. We obtain over 2,000 solvent removal stability measures from text mining and 3,000 thermal decomposition temperatures from thermogravimetric analysis data. We assess the validity of our NLP methods and the accuracy of our extracted data by comparing to a hand-labeled subset. Machine learning (ML, i.e. artificial neural network) models trained on this data using graph- and pore-geometry-based representations enable prediction of stability on new MOFs with quantified uncertainty. Our web interface, MOFSimplify, provides users access to our curated data and enables them to harness that data for predictions on new MOFs. MOFSimplify also encourages community feedback on existing data and on ML model predictions for community-based active learning for improved MOF stability models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Pattern of global spin alignment of ϕ and K*0 mesons in heavy-ion collisions

The strong force, as one of the four fundamental forces at work in the universe, governs interactions of quarks and gluons, and binds together the atomic nucleus. Notwithstanding decades of progress since Yukawa first developed a description of the force between nucleons in terms of meson exchange 1 , a full understanding of the strong interaction remains a major challenge in modern science. One remaining difficulty arises from the non-perturbative nature of the strong force, which leads to the phenomenon of quark confinement at distance scales on the order of the size of the proton. Here we show that in relativistic heavy-ion collisions, where quarks and gluons are set free over an extended volume, two species of produced vector (spin-1) mesons, namely φ and K∗0 , emerge with a surprising pattern of global spin alignment. In particular, the global spin alignment for φ is unexpectedly large, while that for K∗0 is consistent with zero. The observed spin-alignment pattern and magnitude for the φ cannot be explained by conventional mechanisms, while a model with strong force fields2, 3 accommodates the current data. This is the first time that the strong force field is experimentally supported as a key mechanism that leads to global spin alignment. We extract a quantity proportional to the intensity of the field of the strong force. Within the framework of the Standard Model, where the strong force is typically described in the quark and gluon language of Quantum Chromodynamics, the field being considered here is an effective proxy description. This is a qualitatively new class of measurement, which opens a new avenue for studying the behaviour of strong force fields via their imprint on spin alignment.

Abdallah, M. S.↗
DOE OSTI2024

Drilling Down I/O Bottlenecks with Cross-layer I/O Profile Exploration

I/O performance monitoring tools such as Darshan and Recorder collect I/O-related metrics on production systems and help understand the applications' behavior. However, some gaps prevent end-users from seeing the whole picture when it comes to detecting and drilling down to the root causes of I/O performance slowdowns and where those problems originate. These gaps arise from limitations in the available metrics, their collection strategy, and the lack of translation to actionable items that could advise on optimizations. This paper highlights such gaps and proposes solutions to drill down to the source code level to pinpoint the root causes of I/O bottlenecks scientific applications face by relying on cross-layer analysis combining multiple performance metrics related to I/O software layers. We demonstrate with two real applications how metrics collected in high-level libraries (which are closer to the data models used by an application), enhanced by source-code insights and natural language translations, can help streamline the understanding of I/O behavior and provide guidance to end-users, developers, and supercomputing facilities on how to improve I/O performance. Using this cross-layer analysis and the heuristic recommendations, we attained up to 6.9× speedup from run-as-is executions.

Ather, Hammad↗
DOE OSTI2025

Open database for GPD analyses

This article summarizes the main ideas behind creating an open database proposed for use in the exploration of generalized parton distributions (GPDs). This lightweight database is well suited for GPD phenomenology and is designed to store both experimental and lattice-QCD data. It can also aid in benchmarking GPD-related developments, such as GPD models. The database utilizes a new data format based on the YAML serialization language, enabling the storage of essential information for modern analyses, such as replica values. It includes interfaces for both Python and C++, allowing straightforward integration with analysis codes.

Burkert, V. D. [Thomas Jefferson National Accelera↗
DOE OSTI2020

Transforming the study of organisms: Phenomic data models and knowledge bases

The rapidly decreasing cost of gene sequencing has resulted in a deluge of genomic data from across the tree of life; however, outside a few model organism databases, genomic data are limited in their scientific impact because they are not accompanied by computable phenomic data. The majority of phenomic data are contained in countless small, heterogeneous phenotypic data sets that are very difficult or impossible to integrate at scale because of variable formats, lack of digitization, and linguistic problems. One powerful solution is to represent phenotypic data using data models with precise, computable semantics, but adoption of semantic standards for representing phenotypic data has been slow, especially in biodiversity and ecology. Some phenotypic and trait data are available in a semantic language from knowledge bases, but these are often not interoperable. In this review, we will compare and contrast existing ontology and data models, focusing on nonhuman phenotypes and traits. We discuss barriers to integration of phenotypic data and make recommendations for developing an operationally useful, semantically interoperable phenotypic data ecosystem.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Adversarial Attacks on Deep Neural Network-based Power System Event Classification Models

Online event classification is essential to strengthening the reliability of the power transmission system. Recently, deep learning based methods have achieved great success in numerous domains such as computer vision and natural language processing. Researchers began to adopt deep learning based methods to solve the power system event identification problem and achieved effective results. However, these previous works do not consider that deep learning models are vulnerable to adversarial attacks, potentially influencing real-world applications' reliability. In this paper, we adopt several adversarial attack mechanisms by adding tailored noise signal to the input Phasor Measurement Units (PMU) time series and make the deep learning model misclassify the power system event. This numerical study discloses that current state-of-the-art deep learning based power system event classifiers are extremely vulnerable to adversarial attacks, which may jeopardize the reliability of the power transmission system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

GriddingMachine, a database and software for Earth system modeling at global and regional scales

Land and Earth system modeling is moving towards more explicit biophysical representations, requiring increasing variety of datasets for initialization and benchmarking. However, researchers often have difficulties in identifying and integrating non-standardized datasets from various sources. We aim towards a standardized database and one-stop distribution method of global datasets. Here, we present the GriddingMachine as (1) a database of global-scale datasets commonly used to parameterize or benchmark the models, from plant traits to vegetation indices and geophysical information and (2) a cross-platform open source software to download and request a subset of datasets with only a few lines of code. The GriddingMachine datasets can be accessed either manually through traditional HTTP, or automatically using modern programming languages including Julia, Matlab, Octave, Python, and R. The GriddingMachine collections can be used for any land and Earth modeling framework and ecological research at the regional and global scales, and the number of datasets will continue to grow to meet the increasing needs of research communities.

58 GEOSCIENCES↗
DOE OSTI2025

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.

Automated medical encoding↗
DOE OSTI2022

Reimagining Codesign for Advanced Scientific Computing: Report for the ASCR Workshop on Reimagining Codesign

In March 2021, the U.S. Department of Energy’s Advanced Scientific Computing Research program convened the Workshop on Reimagining Codesign. The workshop, also known as ReCoDe, was organized around discussions on eight topic areas: (1) codesign for traditional high-performance computing workloads; (2) codesign of memory/storage systems; (3) codesign of machine learning, neuromorphic, quantum, and other non-von Neumann accelerators; (4) codesign for edge computing and processing at experimental instruments; (5) codesign for security and privacy; (6) hardware design tools and open-source hardware for high-productivity codesign; (7) tools, software stack, and programming languages for high-productivity codesign; and (8) quantitative tools and data collection for modeling and simulation for codesign. The panels identified four Priority Research Directions from these deliberations: (1) breakthrough computing capabilities with targeted heterogeneity and rapid design; (2) software and applications that embrace radical architecture diversity; (3) engineered security and integrity, from transistors to applications; and (4) design with data-rich processes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Water narratives in local newspapers within the United States

Sustainable use of water resources continues to be a challenge across the globe. This is in part due to the complex set of physical and social behaviors that interact to influence water management from local to global scales. Analyses of water resources have been conducted using a variety of techniques, including qualitative evaluations of media narratives. This study aims to augment these methods by leveraging computational and quantitative techniques from the social sciences focused on text analyses. Specifically, we use natural language processing methods to investigate a large corpus (approx. 1.8M) of newspaper articles spanning approximately 35 years (1982–2017) for insights into human-nature interactions with water. Focusing on local and regional United States publications, our analysis demonstrates important dynamics in water-related dialogue about drinking water and pollution to other critical infrastructures, such as energy, across different parts of the country. Our assessment, which looks at water as a system, also highlights key actors and sentiments surrounding water. Extending these analytical methods could help us further improve our understanding of the complex roles of water in current society that should be considered in emerging activities to mitigate and respond to resource conflicts and climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

A structural equation modeling approach to leveraging the power of extant sentiment analysis tools

Machine-derived sentiment analysis has become a pervasive and useful tool to address a wide array of issues in natural language processing. Leading technology companies such as Google now provide sentiment analysis tools (SATs) as readily accessible online products. Academic researchers develop and make available SATs to support the research enterprise. One of the major challenges with SATs is the inconsistencies in results among the various SATs. Consequently, the selection of a SAT for a specific purpose may significantly impact the application. This study addresses the foregoing problem by utilizing structural equation modeling to merge the outputs of SATs to develop a combined sentiment metric without the need for a labeled training dataset. This method is applicable to a wide range of text-based problems, is data-driven, and replicable. It was tested using three publicly available datasets and compared against seven different SATs. The results indicate that as a continous measure, the proposed method outperformed other SATs in the movie reviews and SemEval datasets, and achieved a tie for first place with IBM Watson on the Sentiment 140 dataset. Also, compared to the published major alternatives, the arithmetic mean solution, this approach performed better across these three datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Emerging materials intelligence ecosystems propelled by machine learning

We report that the age of cognitive computing and artificial intelligence (AI) is just dawning. Inspired by its successes and promises, several AI ecosystems are blossoming, many of them within the domain of materials science and engineering. These materials intelligence ecosystems are being shaped by several independent developments. Machine learning (ML) algorithms and extant materials data are utilized to create surrogate models of materials properties and performance predictions. Materials data repositories, which fuel such surrogate model development, are mushrooming. Automated data and knowledge capture from the literature (to populate data repositories) using natural language processing approaches is being explored. The design of materials that meet target property requirements and of synthesis steps to create target materials appear to be within reach, either by closed-loop active-learning strategies or by inverting the prediction pipeline using advanced generative algorithms. AI and ML concepts are also transforming the computational and physical laboratory infrastructural landscapes used to create materials data in the first place. Surrogate models that can outstrip physics-based simulations (on which they are trained) by several orders of magnitude in speed while preserving accuracy are being actively developed. Automation, autonomy and guided high-throughput techniques are imparting enormous efficiencies and eliminating redundancies in materials synthesis and characterization. The integration of the various parts of the burgeoning ML landscape may lead to materials-savvy digital assistants and to a human-machine partnership that could enable dramatic efficiencies, accelerated discoveries and increased productivity. Here, we review these emergent materials intelligence ecosystems and discuss the imminent challenges and opportunities. The materials research landscape is being transformed by the infusion of approaches based on machine learning. This Review discusses the emerging materials intelligence ecosystems and the potential of human-machine partnerships for fast and efficient virtual materials screening, development and discovery.

36 MATERIALS SCIENCE↗