SEARCH · Engineering Papers

Results for “LLAMA”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Evaluation of LLM-Generated Kokkos Code Using Compile-Time and Run-Time Testing

Due to the growing use of large language models (LLMs) by developers and researchers, it has become essential to reliably evaluate their ability to generate code that uses specialized libraries. We explore the use of compile-time and run-time evaluation of LLM-generated Kokkos code through extending the methods used by OpenAI with the HumanEval dataset. Our evaluation framework is based on the first 40 prompts from the Kokkos138 dataset. We start by discussing two different forms of LLM prompting, using entirely plain English or providing pseudocode for added context. These two methods are used to generate Kokkos code with the Llama-3.1-8B-Instruct and CodeQwen1.5-7B-Chat models. We found that both forms of prompting led to high failure rates and difficulties with reliably parsing LLM-generated code, while prompts with pseudocode for context generally led to improved results on more complicated tests.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

Additive Manufacturing (AM) for Propulsion Component and System Applications

NASA MSFC is developing several areas of additive manufacturing (AM) for propulsion component and system applications. These developments include new AM process and material development, large-scale AM pathfinder component fabrication and hot-fire testing, AM-centric design methodology, modeling and simulation, post-processing advancements, and new design applications of AM in aerospace. Some specific areas NASA has focused is the development and commercial infusion of AM supply chain for materials such as copper-alloy GRCop (Cu-Cr-Nb), Hydrogen-resistant alloys (NASA HR-1, JBK-75), refractory-based alloys (C-103, W, Mo, new formulations), and bimetallic and multi-metallic AM processes. Additional efforts have been focused on large scale AM demonstrating complex thin-wall structures up to 60” diameter using directed energy deposition. Material characterization, properties and component hot-fire testing has been completed using many of these newly developed AM alloys/processes. NASA has also focused on improvements to post-processing including powder removal, surface enhancements, and support removal to further realize cost and schedule benefits to the overall supply chain. These developments are part of the NASA Rapid Analysis and Manufacturing Propulsion Technology (RAMPT), Low Cost Upper Stage Propulsion (LCUSP) and Long Life Additive Manufacturing (LLAMA) projects, in addition to internal IRAD, and external collaborations.

Additive Manufacturing↗

NASA NTRSDate not supplied

Harnessing Large Language Models for Scientific Endeavors

The rapid proliferation of Large Language Models (LLMs) such as GPT, Bard, and Llama has revolutionized various sectors, including the scientific community. These models, with their potential to automate and augment tasks, are increasingly being recognized as both a valuable asset and a potential challenge in the realm of scientific research and data management. However, the current LLMs, primarily trained on general corpora, exhibit a limited understanding of scientific concepts and terminologies due to the lack of scientific corpus in their training data. Recognizing this gap, several groups are now advocating for the development of LLMs specifically tailored for scientific applications. A notable initiative in this direction is the Large Language Model effort initiated by NASA's CSDO. This endeavor aims to align LLM efforts across NASA’s Science Mission Directorate, develop a science-specific corpus and validation test set for model training, and create an encoder-only model for various downstream tasks. Moreover, the initiative also plans to develop a decoder-only model to explore the potential benefits and risks associated with a generative LLM for science. Lastly, the project aims to create a science evaluation suite, encompassing various categories of downstream scientific tasks, to serve as a benchmark for assessing the value of any LLM for future use. This presentation will provide an overview and current status of this ongoing initiative, highlighting its potential to reshape the use of LLMs in the scientific domain.

Rahul Ramachandran↗

DOE OSTI2024

Radiation-induced alteration of apatite on the surface of Mars: first in situ observations with SuperCam Raman onboard Perseverance

Abstract Planetary exploration relies considerably on mineral characterization to advance our understanding of the solar system, the planets and their evolution. Thus, we must understand past and present processes that can alter materials exposed on the surface, affecting space mission data. Here, we analyze the first dataset monitoring the evolution of a known mineral target in situ on the Martian surface, brought there as a SuperCam calibration target onboard the Perseverance rover. We used Raman spectroscopy to monitor the crystalline state of a synthetic apatite sample over the first 950 Martian days (sols) of the Mars2020 mission. We note significant variations in the Raman spectra acquired on this target, specifically a decrease in the relative contribution of the Raman signal to the total signal. These observations are consistent with the results of a UV-irradiation test performed in the laboratory under conditions mimicking ambient Martian conditions. We conclude that the observed evolution reflects an alteration of the material, specifically the creation of electronic defects, due to its exposure to the Martian environment and, in particular, UV irradiation. This ongoing process of alteration of the Martian surface needs to be taken into account for mineralogical space mission data analysis.

Science & Technology - Other Topics↗

DOE OSTI2025

Impact of ionization peak location on measured opaqueness in DIII-D H-mode plasmas

This study investigates the relationship between electron pedestal density and the location of the ionization peak on neutral penetration in DIII-D H-mode plasmas, utilizing a database of Lyman-α emission measurements. The high electron density leads to neutrals being ‘screened’ and the ionization front being pushed out into the Scrape-Off Layer (SOL). This is also referred to as the neutral opaqueness, which is heuristically expected to scale with edge plasma density and machine size. However, at lower electron pedestal density, the penetration depth of the neutrals varies, and measured opaqueness deviates from the heuristic scaling. The database reveals that at low density, when the ionization peak is located in SOL region, the linear relationship between the electron density and neutral penetration holds. However, when the peak is located inside the separatrix, the penetration of the neutrals (λ n 0 ) is much wider ~3.0–3.5 cm, breaking the heuristic opaqueness approximation. These findings provide valuable insights into fueling efficiency and plasma behavior, with implications for Fusion Pilot Plants where high pedestal densities are anticipated and where the neutral opaqueness behaves like its heuristic approximation. This analysis offers a framework to refine neutral opaqueness approximations, enhancing the predictive capability for advanced tokamak operations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRSDate not supplied