Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “LLAMA”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Upgrade of the Lyman-alpha diagnostic system on DIII-D for main chamber edge neutral studies

The LLAMA (Lyman Alpha Measurement Apparatus) pinhole camera diagnostic had previously been deployed on DIII-D to measure radial profiles of the Lyman-α (Ly-α) deuterium neutral line brightness across the plasma boundary in the lower chamber to infer neutral deuterium density and ionization rate profiles. This system has recently been upgraded with a new diagnostic head, named ALPACA, that also encloses two pinhole cameras and duplicates the LLAMA views in the upper chamber. Similar to LLAMA, ALPACA provides two times 20 lines of sight, viewing the plasma edge on the inboard and outboard sides with a radial resolution of ~2.5 cm (FWHM) and an effective time resolution of ~1 ms that allows for the investigation of inter-ELM dynamics. The extended Ly-α system provides better coverage to study neutrals in experiments with various plasma shapes utilizing both the upper and lower divertors. Furthermore, post-campaign calibration of the LLAMA diagnostic has successfully been demonstrated for the first time. This was facilitated by various upgrades to the calibration set-up and detailed measurements of the emissivity distribution of the Ly-α calibration source using a pinhole collimator. It was found that the sensitivity of the inboard LLAMA pinhole camera was reduced by a factor of 2.0 ± 0.2 over the course of six months of plasma operation in 2021. In conclusion, the upgraded Ly-α system, equipped with improved absolute calibration, will provide key input for neutral fueling and pedestal particle transport studies and for 2D edge transport code validation on the DIII-D tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Achieving GPT-4o level performance in astronomy with a specialized 8B-parameter large language model

AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained on the complete collection of astronomy-related arXiv papers from 2007 to 2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates remarkable proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9% on the AstroMLab-1 benchmark, greatly outperforming all models—proprietary and open-weight—in the 8-billion parameter class, and performing on par with GPT-4o. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely available, enabling widespread access to advanced AI capabilities for astronomical education and research.

AI assistant↗
DOE OSTI2024

Assurance of Reasoning Enabled Systems (ARES)

ARES was in part motivated by the determination of President’s Council of Advisors on Science and Technology (PCAST) on May 13th, 2023 that published a set of inquiries: In an era in which convincing images, audio, and text can be generated with ease on a massive scale, how can we ensure reliable access to verifiable, trustworthy information? How can we be certain that a particular piece of media is genuinely from the claimed source? What technologies, policies, and infrastructure can be developed to detect and counter AI-generated disinformation? In an effort to automatically analyze and patch/optimize code the work in this report describes various neural Machine Learning (ML) analysis engine implementations to assist in situations where source code is deficient or completely lacking to decompile (lift) binary code to ’C’. The goal is to gradually reduce human intervention. To this end, two Large Language Model (LLM) variants (Code LLama 2, LLama 3.1 and Starcoder1, Starcoder 2) where finetuned with ’before/after’ code pairs on the OpenBLAS library. LLama trained on the lowering process, Starcoder trained on the lifting process with National Security Agency’s (NSA) open-source Ghidra decompiler assist. The inferencing test results indicate correctness for only very short sequences for Starcoder 2. Moving forward, the experiments conclude with a set of recommendations of required resources and technologies

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2025

A Preliminary Study on the Feasibility of Large Language Models for Detecting Micro-Behaviors Among Team Members in Space Missions

Large-language models (LLMs) have been recently used for spoken language understanding (SLU) to infer meaning and semantics from speech in tasks such as speaker intent and sentiment classification. Due to being trained on large amounts of data, and their ability to understand context and relationships between words, LLMs are competent, enabling them to generalize across tasks without requiring many task-specific training samples. This research examines the feasibility of few-shot learning in LLMs for detecting subtle, brief, and possibly unconscious interactions between team members, called ``micro-behaviors," and provides insights into the appropriate design of LLMs for this task. Our data came from 5 teams participating in a 45-day mission at the US National Aeronautics and Space Administration’s (NASA) Human Exploration Research Analog (HERA). More specifically we used data collected from team interaction battery (TIB) tasks teams performed five times in-mission which comprise an average 1.5 hours of conversation data per day. Micro-behaviors were coded according to an adapted version of Smith & Griffins (2022) theoretical framework in terms of Violation (i.e., presence of valenced behavior, uplifting/positive or discouraging/negative), Intensity (i.e., force of behavior in terms of how uplifting or discouraging is the behavior), and Intent (i.e., motive of the behavior in terms of whether it was deliberate or unintentional). We explore the ability of LLMs to detect the presence and intensity of micro-behaviors. We examine employing and fine-tuning readily available LLMs (i.e., RoBERTa, DistilBERT), as well as prompting state-of-the-art sequence classification models (i.e., Llama-2, Llama-3). In a total of 13,058 conversational turns (17.8% uplifting, 3.3% discouraging, 75.76% neutral, 3.14% nulls), we compute the macro F1-score of the 3-way micro-behavior classification task (i.e., classifying among uplifting, discouraging, and neutral; 33% chance). Results indicate that the RoBERTa model achieves a F1-score of 36.2% (uplift: 43.3% precision (P), 15.1% recall (R); discourage: 20% P, 0.5% R). These results significantly improve when we augment the data via paraphrasing in the RoBERTa model, reaching a 41.2% macro F1-score (uplift: 37.7% P, 86.3% R; discourage: 3.5% P, 1.8% R). Finally, the Llama-2 model with 3-shot prompting yields 38% macro F1-score (uplift: 28.7% P, 20% R; discourage: 7.2% P, 18% R), which is slightly better compared to the RoBERTa model without data augmentation, highlighting the effectiveness of sequence classification models in detecting minority classes with a small sample size. Findings indicate that LLMs hold potential to detect subtle behaviors in conversations, which could be valuable in assessing team behavior in space exploration missions. Future studies will evaluate the performance of different LLM prompting strategies or fine-tuning methods.

Ankush Raut↗
DOE OSTI2024

SCITUNE: Aligning Large Language Models with Human-Curated Scientific Multimodal Instructions

Instruction finetuning is a popular paradigm to align large language models (LLM) with human intent. Despite its popularity, this idea is less explored in improving the LLMs to align existing foundation models with scientific disciplines, concepts and goals. In this work, we present SciTune as a tuning framework to improve the ability of LLMs to follow scientific multimodal instructions. To test our methodology, we use a human-generated scientific instruction tuning dataset and train a large multimodal model LLaMA-SciTune that connects a vision encoder and LLM for science-focused visual and language understanding. LLaMA-SciTune significantly outperforms the state-of-the-art models in the generated figure types and captions in multiple scientific multimodal benchmarks. In comparison to the models that are fine-tuned with machine generated data only, LLaMA-SciTune surpasses human performance on average and in many sub-categories on the ScienceQA benchmark.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2025

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages

The rapid evolution of large language models (LLMs) has opened new possibilities for automating various tasks in software development. This paper evaluates the capabilities of the LLaMA 2-70B model in automating these tasks for scientific applications written in commonly used programming languages. Using representative test problems, we assess the model's capacity to generate code, documentation, and unit tests, as well as its ability to translate existing code between commonly used programming languages. Our comprehensive analysis evaluates the compilation, runtime behavior, and correctness of the generated and translated code. Additionally, we assess the quality of automatically generated code, documentation, and unit tests. Here, our results indicate that while LLaMA 2-70B frequently generates syntactically correct and functional code for simpler numerical tasks, it encounters substantial difficulties with more complex, parallelized, or distributed computations, requiring considerable manual corrections. We identify key limitations and suggest areas for future improvements to better leverage AI-driven automation in scientific computing workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Leveraging large language models to automate the identification of healthcare access barriers for veterans

Objective: To develop and evaluate an automated system for identifying healthcare barriers focusing on transportation issues in veterans’ clinical notes using large language models (LLMs) and to assess the impact of different prompting strategies on classification performance and explanation consistency. Methods: We developed a hybrid system combining pattern matching for templated notes with LLM analysis for free-text notes. Using 2000 manually annotated clinical notes, we compared four prompting strategies (dual-role short, dual-role long, analysis-first, analysis-only) across Mistral-7B and Llama-3.1 models. We evaluated classification performance using standard metrics and assessed explanation consistency through embedding similarity analysis. Results: The analysis-first strategy achieved superior performance, with Mistral-7B reaching an F1 score of 0.914, outperforming traditional machine learning approaches (GBM: 0.786, BERT: 0.811). LLMs demonstrated higher explanation consistency within models (mean cosine similarity 0.887–0.908) compared to cross-model similarities (0.767–0.872). Pattern matching successfully handled 6.7% of templated notes deterministically. Mistral-7B showed greater internal consistency but higher abstention rates compared to Llama-3.1. Conclusion: Requiring LLMs to analyze evidence before classification improves both accuracy and explanation consistency for identifying transportation barriers in clinical notes. This approach enables automated barrier detection at scale while providing clinically relevant explanations, supporting both population-level healthcare planning and individual patient care decisions.

Healthcare access barriers↗
DOE OSTI2025

ChatHPC: Building the Foundations for a Productive and Trustworthy AI-Assisted HPC Ecosystem

ChatHPC democratizes large language models for the high-performance computing (HPC) community by providing the infrastructure, ecosystem, and knowledge needed to apply modern generative AI technologies to rapidly create specific capabilities for critical HPC components while using relatively modest computational resources. Our divide-and-conquer approach focuses on creating a collection of reliable, highly specialized, and optimized AI assistants for HPC based on the cost-effective and fast Code Llama fine-tuning processes and expert supervision. We target major components of the HPC software stack, including programming models, runtimes, I/O, tooling, and math libraries. Thanks to AI, ChatHPC provides a more productive HPC ecosystem by boosting important tasks related to portability, parallelization, optimization, scalability, and instrumentation, among others. With relatively small datasets (on the order of KB), the AI assistants, which are created in a few minutes by using one node with two NVIDIA H100 GPUs and the ChatHPC library, can create new capabilities with Meta’s 7-billion parameter Code Llama base model to produce high-quality software with a level of trustworthiness of up to 90% higher than the 1.8-trillion parameter OpenAI ChatGPT-4o model for critical programming tasks in the HPC software stack.

Young, Aaron [ORNL] (ORCID:0000000254484667)↗
DOE OSTI2025

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗
DOE OSTI2025

Can Large Language Models Understand Intermediate Representations?

Intermediate Representations (IRs) are essential in compiler design and program analysis, yet their comprehension by Large Language Models (LLMs) remains underexplored. This paper presents a pioneering empirical study to investigate the capabilities of LLMs, including GPT-4, GPT-3, Gemma 2, LLaMA 3.1, and Code Llama, in understanding IRs. We analyze their performance across four tasks: Control Flow Graph (CFG) reconstruction, decompilation, code summarization, and execution reasoning. Our results indicate that while LLMs demonstrate competence in parsing IR syntax and recognizing high-level structures, they struggle with control flow reasoning, execution semantics, and loop handling. Specifically, they often misinterpret branching instructions, omit critical IR operations, and rely on heuristic-based reasoning, leading to errors in CFG reconstruction, IR decompilation, and execution reasoning. The study underscores the necessity for IR-specific enhancements in LLMs, recommending fine-tuning on structured IR datasets and integration of explicit control flow models to augment their comprehension and handling of IR-related tasks.

Jiang, Hailong↗
DOE OSTI2025

VHH antibody loop guides design of a synthetic macrocyclic peptide that potently blocks influenza virus membrane fusion

Abstract Miniaturizing biologically complex structural motifs to produce synthetic functional mimetics holds significant promise for development of new therapeutic modalities. Here, we demonstrate a unique approach using the key binding loop of the single variable domain of a heavy chain (V H H) llama antibody as a starting point for peptide design. V H H antibodies of camelids and sharks generally have longer, but more ligand-efficient complementarity determining region 3 (CDR3) loops and are relatively stable structures. We harnessed these attributes as templates for design of a series of synthetic macrocyclic peptides. The designed peptides exhibit nanomolar binding to influenza hemagglutinin (HA) and heterosubtypic in vitro neutralization breadth against influenza A viruses by inhibiting the low pH mediated HA conformational changes that lead to membrane fusion. X-ray structures of peptide-HA complexes reveal high structural mimicry with the parent V H H antibody. One such macrocycle peptide candidate is promising for further development of broad protection against influenza A group 1 viruses.

Kadam, Rameshwar U.↗
DOE OSTI2024

Assessment of fine-tuned large language models for real-world chemistry and material science applications

The current generation of large language models (LLMs) has limited chemical knowledge. Recently, it has been shown that these LLMs can learn and predict chemical properties through fine-tuning. Using natural language to train machine learning models opens doors to a wider chemical audience, as field-specific featurization techniques can be omitted. In this work, we explore the potential and limitations of this approach. We studied the performance of fine-tuning three open-source LLMs (GPT-J-6B, Llama-3.1-8B, and Mistral-7B) for a range of different chemical questions. We benchmark their performances against “traditional” machine learning models and find that, in most cases, the fine-tuning approach is superior for a simple classification problem. Depending on the size of the dataset and the type of questions, we also successfully address more sophisticated problems. The most important conclusions of this work are that, for all datasets considered, their conversion into an LLM fine-tuning training set is straightforward and that fine-tuning with even relatively small datasets leads to predictive models. These results suggest that the systematic use of LLMs to guide experiments and simulations will be a powerful technique in any research study, significantly reducing unnecessary experiments or computations.

Van Herck, Joren↗
DOE OSTI2025

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators

Large Language Models (LLMs) have propelled groundbreaking advancements across several domains and are commonly used for text generation applications. However, the computational demands of these complex models pose significant challenges, requiring efficient hardware acceleration. Benchmarking the performance of LLMs across diverse hardware platforms is crucial to understanding their scalability and throughput characteristics. We introduce LLM-Inference-Bench, a comprehensive benchmarking suite to evaluate the hardware inference performance of LLMs. We thoroughly analyze diverse hardware platforms, including GPUs from Nvidia and AMD and specialized AI accelerators, Intel Habana and SambaNova. Our evaluation includes several LLM inference frameworks and models from LLaMA, Mistral, and Qwen families with 7B and 70B parameters. Our benchmarking results reveal the strengths and limitations of various models, hardware platforms, and inference frameworks. We provide an interactive dashboard to help identify configurations for optimal performance for a given hardware platform.

Chitty-Venkata, Krishna Teja↗
DOE OSTI2025

Chatbots can guide experimentalists to avoid many common mistakes in measuring fluorescence spectra

The concepts of fluorescence spectroscopy are well established, yet the experimental collection of a spectrum is susceptible to a range of errors. A recent study showed that chatbots – derived from substantial advances in artificial intelligence – can provide practical assistance to the experimentalist during collection of absorption spectra and thereby achieve improved quality. Fluorescence, while the complement of absorption, is prone to a far richer array of experimental errors encompassing environmental factors, instrumental settings, and experimentalist mistakes. Here, 10 chatbots (ChatGPT 5, Gemini 2.5 Pro, Microsoft Copilot, Meta AI Llama 4, Claude Sonnet 4.5, X.ai Grok 4, Perplexity Pro, DeepSeek V3.1, Z.ai GLM-4.6, and KIMI K2) have been evaluated for the suitability in providing quality-improvement advice to the experimentalist during acquisition of fluorescence spectra. Significant opportunities for practical assistance are noted, although intervention by a domain expert is often required. Here, if combined with image recognition and/or signal processing, chatbots embedded in instruments may enable real-time guidance during fluorescence data acquisition.

Artificial intelligence↗
DOE OSTI2025

Unlocking Scholarly Insights: Leveraging Machine Learning Approaches for Citation Analysis and Intent Classification

Publicly funded organizations, notably institutions like the Los Alamos National Laboratory (LANL), are deeply vested in acquiring robust productivity metrics to gauge the entirety of their research output. Motivated by the imperative to enhance institutional productivity assessment, this study investigates the utilization of Large Language Models (LLM) such as BERT-based models, as well as local LLaMa-30b-instruct and Mixtral-8x7b-instruct architectures for classifying type of URL referenced resources in academic papers such as software, dataset, as well as authorship intent. Challenges in discerning resource types from context are highlighted, along with the potential of BERT and LLMs to address these challenges. Through comprehensive analysis, this research unveils a notable surge in documents featuring URL citations, indicative of the escalating importance of digital resources in scholarly publications. Moreover, citations to datasets and software demonstrate consistent growth over time, underscoring their increasing significance. Our findings also reveal that LANL authors contribute substantially to accessible science, comprising about 10% of dataset and software mentions in LANL

Large Language Models, BERT, citation classificati↗
DOE OSTI2026

ChatMPI: LLM-Driven MPI Code Generation for HPC Workloads

The Message Passing Interface (MPI) standard plays a crucial role in enabling scientific applications for parallel computing and is an essential component in high-performance computing (HPC). However, implementing MPI code manually—especially applying a proper domain decomposition and communication pattern—is a challenging and error-prone task. We present ChatMPI, an AI assistant for MPI parallelization of sequential C codes. In our analysis, we focus on testing six essential HPC workloads, which are based on Basic Linear Algebra Subprograms levels 1, 2, and 3 as well as sparse, stencil, and iterative operations. We analyze the process of creating ChatMPI by using the ChatHPC library. This lightweight large language model (LLM)–based infrastructure enables HPC experts to efficiently create and supervise trustworthy AI capabilities for critical HPC software tasks. We study the data required for training (fine-tuning) ChatMPI to generate parallel codes that not only use MPI syntax correctly but also apply HPC techniques to reduce memory communication and maximize performance by using proper work decomposition. With a relatively small training dataset composed of a few dozen prompts and fewer than 15 minutes of fine-tuning on one node equipped with two NVIDIA H100 GPUs, ChatMPI elevates trustworthiness for MPI code generation of current LLMs (e.g., Code Llama, ChatGPT-4o and ChatGPT 5). Additionally, we evaluate the performance of the MPI codes generated by ChatMPI in comparison with the ones generated by ChatGPT-4o and ChatGPT-5. The codes generated by ChatMPI provide up to a 4 × boost in performance by using better problem decomposition, communication patterns, and HPC techniques (e.g., communication avoiding).

Valero Lara, Pedro [ORNL] (ORCID:0000000214794310)↗
DOE OSTI2025

Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

Code for ‘Lost in OCR Translation?’: robust document retrieval under degradation. Compares OCR-based, vision-only, and hybrid pipelines; includes SambaNova LLaMA Vision OCR, Nougat, and ViDoRe baselines. Provides QA data generation, RAG evaluation, and metrics (Levenshtein, nDCG@k, Recall@k, EM/F1) with reproducible scripts. Includes dataset guides

Bhattarai, Manish [Los Alamos National Labs]↗