Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “language models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASA's Space Launch System

The engineering development of the new Space Launch System (SLS) launch vehicle requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The characteristics of these spacecraft systems must be matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large and complex system engineering challenge, which is being addressed in part by focusing on the specific subsystems involved in the handling of off-nominal mission and fault tolerance with response management. Using traditional model based system and software engineering design principles from the Unified Modeling Language (UML) and Systems Modeling Language (SysML), the Mission and Fault Management (M&FM) algorithms for the vehicle are crafted and vetted in specialized Integrated Development Teams (IDTs) composed of multiple development disciplines such as Systems Engineering (SE), Flight Software (FSW), Safety and Mission Assurance (S&MA) and the major subsystems and vehicle elements such as Main Propulsion Systems (MPS), boosters, avionics, Guidance, Navigation, and Control (GNC), Thrust Vector Control (TVC), and liquid engines. These model based algorithms and their development lifecycle from inception through Flight Software certification are an important focus of this development effort to further insure reliable detection and response to off-nominal vehicle states during all phases of vehicle operation from pre-launch through end of flight. NASA formed a dedicated M&FM team for addressing fault management early in the development lifecycle for the SLS initiative. As part of the development of the M&FM capabilities, this team has developed a dedicated testbed that integrates specific M&FM algorithms, specialized nominal and off-nominal test cases, and vendor-supplied physics-based launch vehicle subsystem models. Additionally, the team has developed processes for implementing and validating these algorithms for concept validation and risk reduction for the SLS program. The flexibility of the Vehicle Management End-to-end Testbed (VMET) enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the developed algorithms utilizing actual subsystem models such as MPS. The intent of VMET is to validate the M&FM algorithms and substantiate them with performance baselines for each of the target vehicle subsystems in an independent platform exterior to the flight software development infrastructure and its related testing entities. In any software development process there is inherent risk in the interpretation and implementation of concepts into software through requirements and test cases into flight software compounded with potential human errors throughout the development lifecycle. Risk reduction is addressed by the M&FM analysis group working with other organizations such as S&MA, Structures and Environments, GNC, Orion, the Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission and Loss of Crew probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and associated detection and responses that can be tested in VMET to ensure that failures can be detected, and confirm that responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - ARINC 653 partitioned OS, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM such as telemetry packing and processing. The baseline plan for use of VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by Flight Software. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure the effectiveness of M&FM algorithms performance in the FSW development and test processes.

Trevino, Luis↗

Exploration Medical System Trade Study Tools Overview

ExMC is creating an ecosystem of tools to enable well-informed medical system trade studies. The suite of tools address important system implementation aspects of the space medical capabilities trade space and are being built using knowledge from the medical community regarding the unique aspects of space flight. Two integrating models, a systems engineering model and a medical risk analysis model, tie the tools together to produce an integrated assessment of the medical system and its ability to achieve medical system target requirements. This presentation will provide an overview of the various tools that are a part of the tool ecosystem. Initially, the presentation's focus will address the tools that supply the foundational information to the ecosystem. Specifically, the talk will describe how information that describes how medicine will be practiced is captured and categorized for efficient utilization in the tool suite. For example, the talk will include capturing what conditions will be planned for in-mission treatment, planned medical activities (e.g., periodic physical exam), required medical capabilities (e.g., provide imaging), and options to implement the capabilities (e.g., an ultrasound device). Database storage and configuration management will also be discussed. The presentation will include an overview of how these information tools will be tied to parameters in a Systems Modeling Language (SysML) model, allowing traceability to system behavioral, structural, and requirements content. The discussion will also describe an HRP-led enhanced risk assessment model developed to provide quantitative insight into each capability's contribution to mission success. Key outputs from these various tools, to be shared with the space medical and exploration mission development communities, will be assessments of medical system implementation option satisfaction of requirements and per-capability contributions toward achieving requirements.

Mindock, J.↗

Enabling Space Exploration Medical System Development Using a Tool Ecosystem

The NASA Human Research Program’s (HRP) Exploration Medical Capability (ExMC) Element is utilizing a Model Based Systems Engineering (MBSE) approach to enhance the development of systems engineering products that will be used to advance medical system designs for exploration missions beyond Low Earth Orbit. In support of future missions, the team is capturing content such as system behaviors, functional decompositions, architecture, system requirements and interfaces, and recommendations for clinical capabilities and resources in Systems Modeling Language (SysML) models. As these products mature, SysML models provide a way for ExMC to capture relationships among the various products, which includes supporting more integrated and multi-faceted views of future medical systems. In addition to using SysML models, HRP and ExMC are developing supplementary tools to support two key functions: 1) prioritizing current and future research activities for exploration missions in an objective manner; and 2) enabling risk-informed and evidence-based trade space analysis for future space vehicles, missions, and systems. This paper will discuss the long-term HRP and ExMC vision for the larger ecosystem of tools, which include dynamic Probabilistic Risk Assessment (PRA) capabilities, additional SysML models, a database of system component options, and data visualizations. It also includes a review of an initial Pilot Project focused on enabling medical system trade studies utilizing data that is coordinated across tools for consistent outputs (e.g., mission risk metrics that are associated with medical system mass values and medical conditions addressed). This first Pilot Project demonstrated successful operating procedures and integration across tools. Finally, the paper will also cover a second Pilot Project that utilizes tool enhancements such as medical system optimization capabilities, post-processing, and visualization of generated data for subject matter expert review, and increased integration amongst the tools themselves.

Amador, Jennifer R.↗

Enabling Space Exploration Medical System Development Using a Tool Ecosystem

The NASA Human Research Program's (HRP) Exploration Medical Capability (ExMC) Element is utilizing a Model Based Systems Engineering (MBSE) approach to enhance the development of systems engineering products that will be used to advance medical system designs for exploration missions beyond Low Earth Orbit. In support of future missions, the team is capturing content such as system behaviors, functional decompositions, architecture, system requirements and interfaces, and recommendations for clinical capabilities and resources in Systems Modeling Language (SysML) models. As these products mature, SysML models provide a way for ExMC to capture relationships among the various products, which includes supporting more integrated and multi-faceted views of future medical systems. In addition to using SysML models, HRP and ExMC are developing supplementary tools to support two key functions: 1) prioritizing current and future research activities for exploration missions in an objective manner; and 2) enabling risk-informed and evidence-based trade space analysis for future space vehicles, missions, and systems. This paper will discuss the long-term HRP and ExMC vision for the larger ecosystem of tools, which include dynamic Probabilistic Risk Assessment (PRA) capabilities, additional SysML models, a database of system component options, and data visualizations. It also includes a review of an initial Pilot Project focused on enabling medical system trade studies utilizing data that is coordinated across tools for consistent outputs (e.g., mission risk metrics that are associated with medical system mass values and medical conditions addressed). This first Pilot Project demonstrated successful operating procedures and integration across tools. Finally, the paper will also cover a second Pilot Project that utilizes tool enhancements such as medical system optimization capabilities, post-processing, and visualization of generated data for subject matter expert review, and increased integration amongst the tools themselves.

Amador, Jennifer R.↗

Impact, a Tool Suite for Crew Health and Performance System Trade Analyses and Decision Support - Status of Development

Mission planners, systems engineers, and clinicians that support crew health and performance face very difficult choices on upcoming exploration missions. Given that there will be a heavily constrained mass and volume allocation for a medical system on these missions, what medical capability should be manifested to minimize both medical risk and mission risk? Given that not all promising research and technology proposals can be funded, how can proposals be prioritized so that those funded research investments produce the maximum benefit in reducing overall medical risk? The Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) project seeks to answer these kinds of questions and others to support upcoming exploration missions. IMPACT enables risk-informed and evidence-based trade space analysis for future space vehicles, missions, and systems. This presentation will discuss the long-term HRP and ExMC vision for the larger ecosystem of tools, which include an updated medical database (consisting of an Evidence Library for medical conditions and a medical item database (MedID) for medical resources), dynamic Probabilistic Risk Assessment (PRA) capabilities, System Modeling Language (SysML) models, and contextual data visualizations of output data. IMPACT is the result of a multi-center collaborative effort. The trade space analyses performed by IMPACT can directly inform mission, vehicle, and habitat development by quantifying medical risk, given a design reference mission, crew attributes and a set of medical capabilities. This presentation will update the audience on the development status of the tool suite, its constituent parts and the plans for when it will deploy as an operational product, currently scheduled for the end of FY22. Recent development successes on the IMPACT project include the ability to cluster medical resources into medical capabilities and mutually-dependent bundles, the ability to perform trade analyses on different medical sets, different DRMs, and different crew complements, and the ability to specify mission segments as periods of time assigned to one or more members of the crew during which unique mission events occur (e.g., extravehicular activities, surface operations, gravity well adaptations, etc.). IMPACT is gearing up for its verification and validation phase to compile the data package necessary for a successful Transition to Operations (TtO), per the guidance in NPR 8900.1B .

IMPACT↗

IMPACT, a Tool Suite for Crew Health and Performance System Trade Analyses and Decision Support - Status of Development

Mission planners, systems engineers, and clinicians that support crew health and performance face very difficult choices on upcoming exploration missions. Given that there will be a heavily constrained mass and volume allocation for a medical system on these missions, what medical capability should be manifested to minimize both medical risk and mission risk? Given that not all promising research and technology proposals can be funded, how can proposals be prioritized so that those funded research investments produce the maximum benefit in reducing overall medical risk? The Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) project seeks to answer these kinds of questions and others to support upcoming exploration missions. IMPACT enables risk-informed and evidence-based trade space analysis for future space vehicles, missions, and systems. This presentation will discuss the long-term HRP and ExMC vision for the larger ecosystem of tools, which include an updated medical database, consisting of an Evidence Library for medical conditions and a medical item database (MedID) for medical resources, dynamic Probabilistic Risk Assessment (PRA) capabilities, System Modeling Language (SysML) models, and contextual data visualizations of output data. IMPACT is the result of a multi-center collaborative effort. The trade space analyses performed by IMPACT can directly inform mission, vehicle, and habitat development by quantifying medical risk, given a design reference mission, crew attributes and a set of medical capabilities. This presentation will update the audience on the development status of the tool suite as it nears its System Acceptance Review (SAR). It will review IMPACT’s constituent parts, briefly discuss typical outputs and outline the plans for transitioning to operations, currently scheduled for later in FY23. Recent development successes on the IMPACT project include the integration of the Medical Extensible Dynamic Probabilistic Risk Assessment Tool (MEDPRAT) v2.0 to accommodate segmented missions with multiple carriers and medical systems, full onboarding of the IMPACT Medical Database (IMPACT-MD), clustering medical resources and skills into medical capabilities and mutually-dependent bundles, and the ability to perform trade analyses on different medical sets, different design reference missions (DRM), with different crew complements and extra-vehicular activity (EVA) schedule.

IMPACT↗

Impact, A Tool Suite for Crew Health and Performance System Trade Analyses and Decision to Support - Transition to Operations

Mission planners, systems engineers, and clinicians that support crew health and performance face very difficult choices on upcoming exploration missions. Given that there will be a heavily constrained mass and volume allocation for a medical system on these missions, what medical capability should be manifested to minimize both medical risk and mission risk? Given that not all promising research and technology proposals can be funded, how can proposals be prioritized so that those funded research investments produce the maximum benefit in reducing overall medical risk? The Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) project seeks to answer these kinds of questions and others to support upcoming exploration missions. IMPACT enables risk-informed and evidence-based trade space analysis for future space vehicles, missions, and systems. This presentation will discuss the long-term HRP and ExMC vision for the larger ecosystem of tools, which include an updated medical database, consisting of an Evidence Library for medical conditions and a medical item database (MedID) for medical resources, dynamic Probabilistic Risk Assessment (PRA) capabilities, System Modeling Language (SysML) models, and contextual data visualizations of output data. IMPACT is the result of a multi-center collaborative effort. The trade space analyses performed by IMPACT can directly inform mission, vehicle, and habitat development by quantifying medical risk, given a design reference mission, crew attributes and a set of medical capabilities. This presentation will update the audience on the development status of the IMPACT tool suite as it comes out of its System Acceptance Review (SAR) and nears Transition to Operations (TTO). It will review IMPACT’s constituent parts, briefly discuss typical outputs, and outline the plans for transitioning to operations, currently scheduled for later in FY24. Recent development successes on the IMPACT project include the integration of the Medical Extensible Dynamic Probabilistic Risk Assessment Tool (MEDPRAT) v2.0 to accommodate segmented missions with multiple carriers and medical systems, full onboarding of the IMPACT Medical Database (IMPACT-MD), clustering medical resources and skills into medical capabilities and mutually-dependent bundles, verification of IMPACT-MD, and the ability to perform trade analyses on different medical sets, different design reference missions (DRM), with different crew complements and extra-vehicular activity (EVA) schedules.

IMPACT↗

IMPACT, A Tool Suite for Crew Health and Performance System Trade Analyses and Decision Support- Transition to Operations

Mission planners, systems engineers, and clinicians that support crew health and performance face very difficult choices on upcoming exploration missions. Given that there will be a heavily constrained mass and volume allocation for a medical system on these missions, what medical capability should be manifested to minimize both medical risk and mission risk? Given that not all promising research and technology proposals can be funded, how can proposals be prioritized so that those funded research investments produce the maximum benefit in reducing overall medical risk? The Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) project seeks to answer these kinds of questions and others to support upcoming exploration missions. IMPACT enables risk-informed and evidence-based trade space analysis for future space vehicles, missions, and systems. This presentation will discuss the long-term HRP and ExMC vision for the larger ecosystem of tools, which include an updated medical database, consisting of an Evidence Library for medical conditions and a medical item database (MedID) for medical resources, dynamic Probabilistic Risk Assessment (PRA) capabilities, System Modeling Language (SysML) models, and contextual data visualizations of output data. IMPACT is the result of a multi-center collaborative effort. The trade space analyses performed by IMPACT can directly inform mission, vehicle, and habitat development by quantifying medical risk, given a design reference mission, crew attributes and a set of medical capabilities. This presentation will update the audience on the development status of the IMPACT tool suite as it comes out of its System Acceptance Review (SAR) and nears Transition to Operations (TTO). It will review IMPACT’s constituent parts, briefly discuss typical outputs, and outline the plans for transitioning to operations, currently scheduled for later in FY24. Recent development successes on the IMPACT project include the integration of the Medical Extensible Dynamic Probabilistic Risk Assessment Tool (MEDPRAT) v2.0 to accommodate segmented missions with multiple carriers and medical systems, full onboarding of the IMPACT Medical Database (IMPACT-MD), clustering medical resources and skills into medical capabilities and mutually-dependent bundles, verification of IMPACT-MD, and the ability to perform trade analyses on different medical sets, different design reference missions (DRM), with different crew complements and extra-vehicular activity (EVA) schedules.

IMPACT↗

Java PathFinder: A Translator From Java to Promela

JAVA PATHFINDER, JPF, is a prototype translator from JAVA to PROMELA, the modeling language of the SPIN model checker. JPF is a product of a major effort by the Automated Software Engineering group at NASA Ames to make model checking technology part of the software process. Experience has shown that severe bugs can be found in final code using this technique, and that automated translation from a programming language to a modeling language like PROMELA can help reducing the effort required.

Havelund, Klaus↗

Do Programmers Prefer Predictable Expressions in Code?

Source code is a form of human communication, albeit one where the information shared between the programmers reading and writing the code is constrained by the requirement that the code executes correctly. Programming languages are more syntactically constrained than natural languages, but they are also very expressive, allowing a great many different ways to express even very simple computations. Still, code written by developers is highly predictable, and many programming tools have taken advantage of this phenomenon, relying on language model surprisal as a guiding mechanism. Additionally, while surprisal has been validated as a measure of cognitive load in natural language, its relation to human cognitive processes in code is still poorly understood. In this paper, we explore the relationship between surprisal and programmer preference at a small granularity—do programmers prefer more predictable expressions in code? Using meaning-preserving transformations, we produce equivalent alternatives to developer-written code expressions and run a corpus study on Java and Python projects. In general, language models rate the code expressions developers choose to write as more predictable than these transformed alternatives. Then, we perform two human subject studies asking participants to choose between two equivalent snippets of Java code with different surprisal scores (one original and transformed). We find that programmers do prefer more predictable variants, and that stronger language models like the transformer align more often and more consistently with these preferences.

97 MATHEMATICS AND COMPUTING↗

A software package for modeling and simulating fault graphs

This report describes a novel fault graph modeling language and a simulation tool for executing models specified in the language. The modeling language has three primary features that distinguish it from similar reliability analysis tools. These are (1) a random variable modeling several distinct outcomes of a single fault; (2) chains of faults in which one fault triggers another; and (3) time to fail sampled from probability distributions including positive normal, exponential, Weibull with a minimum, or immediate. These features are motivated by their use in a historical analysis of centrifuge reliability.

97 MATHEMATICS AND COMPUTING↗

Fault Graph (fg)

This software package offers a novel fault graph modeling language and a simulation tool for executing models specified in the language. The modeling language has three primary features that distinguish it from similar reliability analysis tools. These are (1) a random variable modeling several distinct outcomes of a single fault; (2) chains of faults in which one fault triggers another; and (3) time to fail sampled from probability distributions including positive normal, exponential, Weibull with a minimum, or immediate.

Nutaro, James↗

Data-Flow Based Model Analysis

The concept of (meta) modeling combines an intuitive way of formalizing the structure of an application domain with a high expressiveness that makes it suitable for a wide variety of use cases and has therefore become an integral part of many areas in computer science. While the definition of modeling languages through the use of meta models, e.g. in Unified Modeling Language (UML), is a well-understood process, their validation and the extraction of behavioral information is still a challenge. In this paper we present a novel approach for dynamic model analysis along with several fields of application. Examining the propagation of information along the edges and nodes of the model graph allows to extend and simplify the definition of semantic constraints in comparison to the capabilities offered by e.g. the Object Constraint Language. Performing a flow-based analysis also enables the simulation of dynamic behavior, thus providing an "abstract interpretation"-like analysis method for the modeling domain.

Saad, Christian↗

Rapid Adaptation of Chemical Named Entity Recognition Using Few-Shot Learning and LLM Distillation

Named entity recognition (NER) has been widely used in chemical text mining for the automatic identification and extraction of chemical entities. However, existing chemical NER systems primarily focus on scenarios with abundant training data, requiring significant human effort on annotations. This poses challenges for applications in the chemical field, such as catalysis, where many advancements have traditionally relied on trial-and-error investigations and incremental adjustment of variables. This hinders catalysis science and technology progress in addressing emerging energy and environmental crises. In this work, we propose a few-shot NER model that can quickly adapt to extract new types of chemical entities by using only a limited number of annotated examples. Our model employs a metric-learning approach to transfer entity similarity knowledge from high-resource chemical domains (with abundant annotations) to enable effective entity recognition in low-resource specialized domains (limited annotation). We validate the effectiveness of our model on a few-shot chemical NER benchmark built based on six existing chemical NER data sets. Experiments show that the proposed few-shot NER model can achieve reasonable performance with only 5 examples per entity type and shows consistent improvement as the number of examples increases. Furthermore, we demonstrate how the proposed model can be trained with large language model (LLM) annotated data, opening a new pathway for rapid adaptation of NER systems. Furthermore, our approach leverages the knowledge broadness of large language models for chemistry while distilling this knowledge into a lightweight model suitable for efficient and in-house use.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Safety Analysis of FMS/CTAS Interactions During Aircraft Arrivals

This grant funded research on human-computer interaction design and analysis techniques, using future ATC environments as a testbed. The basic approach was to model the nominal behavior of both the automated and human procedures and then to apply safety analysis techniques to these models. Our previous modeling language, RSML, had been used to specify the system requirements for TCAS II for the FAA. Using the lessons learned from this experience, we designed a new modeling language that (among other things) incorporates features to assist in designing less error-prone human-computer interactions and interfaces and in detecting potential HCI problems, such as mode confusion. The new language, SpecTRM-RL, uses "intent" abstractions, based on Rasmussen's abstraction hierarchy, and includes both informal (English and graphical) specifications and formal, executable models for specifying various aspects of the system. One of the goals for our language was to highlight the system modes and mode changes to assist in identifying the potential for mode confusion. Three published papers resulted from this research. The first builds on the work of Degani on mode confusion to identify aspects of the system design that could lead to potential hazards. We defined and modeled modes differently than Degani and also defined design criteria for SpecTRM-RL models. Our design criteria include the Degani criteria but extend them to include more potential problems. In a second paper, Leveson and Palmer showed how the criteria for indirect mode transitions could be applied to a mode confusion problem found in several ASRS reports for the MD-88. In addition, we defined a visual task modeling language that can be used by system designers to model human-computer interaction. The visual models can be translated into SpecTRM-RL models, and then the SpecTRM-RL suite of analysis tools can be used to perform formal and informal safety analyses on the task model in isolation or integrated with the rest of the modeled system. We had hoped to be able to apply these modeling languages and analysis tools to a TAP air/ground trajectory negotiation scenario, but the development of the tools took more time than we anticipated.

Nancy G. Leveson↗

DECIDER

This software offers methods and functions for building failure detectors for deep image classification models with the aid of vision-language models and LLMs. It includes functionalities for training baseline image classifiers, debiasing classifiers using vision-language models and LLMs, evaluating failure between models along with baselines. Developed using PyTorch, this software is compatible with standard neural network architectures used for imaging data. Additionally, it provides capabilities to compute evaluation metrics for assessing the performance and quality of the detectors.

Narayanaswamy, Vivek Sivaraman↗

MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models

Mixture of Experts (MoE) models have enabled the scaling of Large Language Models (LLMs) and Vision Language Models (VLMs) by achieving massive parameter counts while maintaining computational efficiency. However, MoEs introduce several inference-time challenges, including load imbalance across experts and the additional routing computational overhead. To address these challenges and fully harness the benefits of MoE, a systematic evaluation of hardware acceleration techniques is essential. We present MoE-Inference-Bench, a comprehensive study to evaluate MoE performance across diverse scenarios. We analyze the impact of batch size, sequence length, and critical MoE hyperparameters such as FFN dimensions and number of experts on throughput. We evaluate several optimization techniques on Nvidia H100 GPUs, including pruning, Fused MoE operations, speculative decoding, quantization, and various parallelization strategies. Our evaluation includes MoEs from the Mixtral, DeepSeek, OLMoE and Qwen families. The results reveal performance differences across configurations and provide insights for the efficient deployment of MoEs.

Chitty-Venkata, Krishna Teja↗

Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

Abstract Motivation Millions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. Results We developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementation The source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.

59 BASIC BIOLOGICAL SCIENCES↗