SEARCH · Engineering Papers

Results for “language models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

All sources NASA NTRS DOE OSTI

NASA NTRS2020

Enabling Assurance in the MBSE Environment

A number of specific benefits that fit within the hallmarks of effective development are realized with implementation of model-based approaches to systems and assurance. Model Based Systems Engineering (MBSE) enabled by standardized modeling languages (e.g., SysML®) is at the core. These benefits in the context of spaceflight system challenges can include [1]: • Improved management of complex development • Reduced risk in the development process • Improved cost management • Improved design decisions With appropriate modeling techniques the assurance community also can improve early oversight and insight into project development. NASA has shown the basic constructs of SysML in an MBSE environment offer several key advantages, within a Model Based Mission Assurance (MBMA) initiative [2, 3]. These include the following: • Model viewpoints that promote rapid and systematic assessment of requirements coverage, hazard tagging and risk management • Embedded safety assessments for launch vehicles • Deployment of model assisted development of reliability products - Failure Modes and Effects Analyses (FMEAs) and Fault Trees • Test Planning • Validation and Verification of complex functions • Support of Assurance Case development for complex systems In addition, while there are benefits to be harvested, there is a realization that these do not come without effort and cost. Enabling model-based approaches requires structure, not only in an organizational context, but in a modeling context as well. There can be a steep learning curve and costs associated to train skilled modelers. But, on the other hand, not all of the assurance community need to be modelers. Models themselves must conform to ontologies that enable assurance. This places constraints upon the models and modelers. Optimums have yet to be developed where resources and constraints on modeling must be traded off in the organization and modeling efforts for projects. A number of barriers need to be overcome, as well, which pose challenges to the developers of the software that supports MBSE/MBMA. Information and data must be made to flow seamlessly through the life cycle. Because there is a wide variety of tools used in the community, to avoid the problems of the past of silos, delays, and diverging interests, information should flow among these tools to support the “single source of truth” paradigm of MBSE. This will greatly facilitate MBMA and advancement of assurance functions.

Evans, John W.↗

DOE OSTI2026

matsim-agents v1.0

matsim-agents is a multi-agent AI framework for atomistic materials simulation and discovery. It orchestrates large language models (LLMs), machine-learned interatomic potentials (MLIPs), and DFT codes into a single agentic loop running on laptops and DOE leadership-class supercomputers. MULTI-AGENT ORCHESTRATION A LangGraph state machine with three nodes: a Planner that converts a natural-language research objective into structured tasks; an Executor that dispatches atomistic tools and loops until the queue is empty; and an Analyst that summarizes results into a human-readable report. State is checkpointed after every step and human-in-the-loop gates can be inserted at any edge. HYPOTHESIS-DRIVEN DISCOVERY CHAT An interactive REPL (matsim-agents chat) that couples LLM dialogue with atomistic simulation. Chemical formulas are automatically detected in conversation turns and trigger a full crystal-phase exploration: structure generation → relaxation → stability scoring → result injection back into the conversation, creating a closed hypothesis-refinement loop. CRYSTAL PHASE ENUMERATION Given a composition, the phase explorer enumerates prototypes by stoichiometry: elemental (fcc/bcc/hcp/sc/diamond), binary 1:1 (rocksalt/CsCl/zincblende/ wurtzite/fluorite/rutile), ternary 1:1:3 (cubic perovskite), ternary 1:2:4 (perovskite + spinel), quaternary 1:1:2:6 (Fm-3m double perovskite). 2-D prototypes (graphene, h-BN, MoS2 2H/1T) and multilayer stacking are also supported via --include-2d and --num-layers. SUPERCELL GENERATION AND SITE DECORATION Auto-tiling to a minimum atom count (--min-atoms), explicit NxNxN tiling (--supercell), symmetry-distinct site decorations (--n-orderings), and isotropic lattice-scale sweeps (--lattice-scales) for volume bracketing. MLFF RELAXATION AND STABILITY SCORING HydraGNN (multi-headed GNN) drives structure relaxation via ASE with FIRE, BFGS, or BFGSLineSearch. Stability output: delta-E/atom ranking across phases and a max-residual-force dynamical-stability proxy. Other MLIPs (MACE, NequIP, Orb) can be plugged in through the same interface. DFT BACKENDS Quantum ESPRESSO pw.x and VASP 6.6 are first-class labellers. Both have validated GPU builds and SLURM/PBS launchers for three DOE platforms: Frontier (AMD MI250X, ROCm), Aurora (Intel PVC, oneAPI), Perlmutter (NVIDIA A100, CUDA). QE produces ~100 binaries (pw.x, ph.x, epw.x, ...). VASP supports scf, relax, vc-relax, and vc-relax-shape run types. ACTIVE-LEARNING LOOP matsim-agents al run CONFIG.yaml drives an iterative HydraGNN-DFT loop: MD generates candidates → ensemble/MC-dropout uncertainty selects the most informative → DFT labels them in parallel inside one allocation → dataset grows → HydraGNN retrains → repeat. DFT backend is a single YAML toggle (dft.backend: vasp | qe). LLM-generated seed structures are supported (no curated POSCAR library needed). Config uses ${VAR}, ${VAR:-default}, ${VAR:?msg} shell-style substitution for cross-user/cross-site portability. LLM BACKENDS Ollama (local, default), vLLM (HPC multi-GPU serving), OpenAI, Anthropic, HuggingFace Transformers+Accelerate. Selected at runtime via flag or env var with no code changes. HPC PORTABILITY Same Python entry points run on Frontier (ROCm 7.2), Aurora (oneAPI), and Perlmutter (CUDA 12). DFT and ML stacks are never co-loaded in the same shell; they couple through the scheduler and filesystem. Advanced multi-node launchers (serve, discovery-chat, single-relaxation, active-learning, QE warm-start) are provided for all three platforms. CODABENCH COMPETITION BUNDLE A self-contained benchmark: 159 atomistic test structures across 11 material classes, 5 tasks (formation energy, forces, ML relaxation, AI-DFT relaxation, phase stability ranking), public/private leaderboard split (30/70), and four ready-to-run baselines: MACE-MP-0, HydraGNN, UMA, AllScAIP.

Lupo Pasini, Massimiliano [Oak Ridge National Labo↗

DOE OSTI2024

GOLEM: GOld standard for Learning and Evaluation of Motifs

Motifs are distinctive, recurring, widely used idiom-like words or phrases, often originating from folklore, whose meaning is anchored in a narrative and have a significance as communicative devices across a wide range of media, including news, literature, and propaganda. Many motifs concisely imply a large constellation of culturally relevant information, and their broad usage suggests their cognitive importance as touchstones of cultural knowledge. As such, their detection is a step towards culturally aware natural language processing. We present GOLEM (GOld standard for Learning and Evaluation of Motifs) a dataset of English news articles, opinion pieces, and broadcast transcripts annotated for motific information. The dataset identifies 25,737 motif candidates across 34 motif types drawn from three cultural or national groups: Jewish, Irish, and Puerto Rican. The dataset contains 2,024,141 words split into 25,737 text snippets drawn from 8,073 articles. Each motif candidate is labeled according to a scheme which identifies the type of usage (motific, referential, eponymic, or unrelated), resulting in 1,743 actual motific instances in the data. Annotation was performed by individuals identifying as members of each group and achieved a Fleiss’ kappa (?) of > 0.55. In addition to the data, we demonstrate that classification of the candidate type is a challenging task for Large Language Models (LLMs) using a few-shot approach; recent models such as T5, FLAN-T5, GPT-2, and Llama 2 (7B) achieved a performance of 41% accuracy at best, where the majority class accuracy is 41% and the average chance accuracy is 27%. These data will support development of new models and approaches for detecting (and reasoning about) motific information in text.

motif, culture, natural language, artificial intel↗

NASA NTRS2012

Model-Based Systems Engineering Pilot Program at NASA Langley

NASA Langley Research Center conducted a pilot program to evaluate the benefits of using a Model-Based Systems Engineering (MBSE) approach during the early phase of the Materials International Space Station Experiment-X (MISSE-X) project. The goal of the pilot was to leverage MBSE tools and methods, including the Systems Modeling Language (SysML), to understand the net gain of utilizing this approach on a moderate size flight project. The System Requirements Review (SRR) success criteria were used to guide the work products desired from the pilot. This paper discusses the pilot project implementation, provides SysML model examples, identifies lessons learned, and describes plans for further use on MBSE on MISSE-X.

Vipavetz, Kevin G.↗

NASA NTRS2010

BGen: A UML Behavior Network Generator Tool

BGen software was designed for autogeneration of code based on a graphical representation of a behavior network used for controlling automatic vehicles. A common format used for describing a behavior network, such as that used in the JPL-developed behavior-based control system, CARACaS ["Control Architecture for Robotic Agent Command and Sensing" (NPO-43635), NASA Tech Briefs, Vol. 32, No. 10 (October 2008), page 40] includes a graph with sensory inputs flowing through the behaviors in order to generate the signals for the actuators that drive and steer the vehicle. A computer program to translate Unified Modeling Language (UML) Freeform Implementation Diagrams into a legacy C implementation of Behavior Network has been developed in order to simplify the development of C-code for behavior-based control systems. UML is a popular standard developed by the Object Management Group (OMG) to model software architectures graphically. The C implementation of a Behavior Network is functioning as a decision tree.

Huntsberger, Terry↗

DOE OSTI2023

Latent Dirichlet Allocation modeling of environmental microbiomes

Interactions between stressed organisms and their microbiome environments may provide new routes for understanding and controlling biological systems. However, microbiomes are a form of high-dimensional data, with thousands of taxa present in any given sample, which makes untangling the interaction between an organism and its microbial environment a challenge. Here we apply Latent Dirichlet Allocation (LDA), a technique for language modeling, which decomposes the microbial communities into a set of topics (non-mutually-exclusive sub-communities) that compactly represent the distribution of full communities. LDA provides a lens into the microbiome at broad and fine-grained taxonomic levels, which we show on two datasets. In the first dataset, from the literature, we show how LDA topics succinctly recapitulate many results from a previous study on diseased coral species. We then apply LDA to a new dataset of maize soil microbiomes under drought, and find a large number of significant associations between the microbiome topics and plant traits as well as associations between the microbiome and the experimental factors, e.g. watering level. This yields new information on the plant-microbial interactions in maize and shows that LDA technique is useful for studying the coupling between microbiomes and stressed organisms.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS1990

A graphical language for reliability model generation

A graphical interface capability of the hybrid automated reliability predictor (HARP) is described. The graphics-oriented (GO) module provides the user with a graphical language for modeling system failure modes through the selection of various fault tree gates, including sequence dependency gates, or by a Markov chain. With this graphical input language, a fault tree becomes a convenient notation for describing a system. In accounting for any sequence dependencies, HARP converts the fault-tree notation to a complex stochastic process that is reduced to a Markov chain which it can then solve for system reliability. The graphics capability is available for use on an IBM-compatible PC, a Sun, and a VAX workstation. The GO module is written in the C programming language and uses the Graphical Kernel System (GKS) standard for graphics implementation. The PC, VAX, and Sun versions of the HARP GO module are currently in beta-testing.

Howell, Sandra V.↗

NASA NTRS2006

ADGS-2100 Adaptive Display and Guidance System Window Manager Analysis

Recent advances in modeling languages have made it feasible to formally specify and analyze the behavior of large system components. Synchronous data flow languages, such as Lustre, SCR, and RSML-e are particularly well suited to this task, and commercial versions of these tools such as SCADE and Simulink are growing in popularity among designers of safety critical systems, largely due to their ability to automatically generate code from the models. At the same time, advances in formal analysis tools have made it practical to formally verify important properties of these models to ensure that design defects are identified and corrected early in the lifecycle. This report describes how these tools have been applied to the ADGS-2100 Adaptive Display and Guidance Window Manager being developed by Rockwell Collins Inc. This work demonstrates how formal methods can be easily and cost-efficiently used to remove defects early in the design cycle.

Whalen, Mike W.↗

DOE OSTI2023

Towards POI-based large-scale land use modeling: spatial scale, semantic granularity, and geographic context

The combination of spatial distribution, semantic characteristics, and sometimes temporal dynamics of POIs inside a geographic region can capture its unique land use characteristics. Most previous studies on POI-based land use modeling research focused on one geographic region and select one spatial scale and semantic granularity for land use characterization. There is a lack of understanding on the impact of spatial scale, semantic granularity, and geographic context on POI-based land use modeling, particularly large-scale land use modeling. In this study, we developed a scalable POI-based land use modeling framework and examined the impact of these three factors on POI-based land use characterization using data from three geographic regions. We developed a unified semantic representation framework for POI semantics that can help fuse heterogeneous POI data sources. Then, by combining POIs with a neural network language model, we developed a spatially explicit approach to learn the embedding representation of POIs and AOIs. We trained multiple supervised classifiers using AOI embeddings as input features to predict AOI land use at different semantic granularities. The classification performance of different land use classes was analyzed and compared across three geographic regions to identify the semantic representativeness of POI-based AOI embedding and the impact of geographic context.

58 GEOSCIENCES↗

DOE OSTI2023

Code and Data for "Quantifying Uncertainties in Greenhouse Gas Savings and Mitigation Costs with Cellulosic Biofuels"

Use of cellulosic biofuels from non-feedstocks are modeled using the BEPAM (Biofuel and Environmental Policy Analysis Model) model to quantifying the uncertainties about induced land use change effects, net greenhouse gas saving potential, and economic costs. The code is in GAMS, general algebraic modeling language. NOTE: Column 3 is titled "BAU" in "merged_BAU.gdx", "merged_RFS.gdx", and "merged_CEM.gdx", but contains "RFS" data in "merged_RFS.gdx" and "CEM" data in "merged_CEM.gdx".

BEPAM↗

NASA NTRS1989

Integrated modeling tool for performance engineering of complex computer systems

This report summarizes Advanced System Technologies' accomplishments on the Phase 2 SBIR contract NAS7-995. The technical objectives of the report are: (1) to develop an evaluation version of a graphical, integrated modeling language according to the specification resulting from the Phase 2 research; and (2) to determine the degree to which the language meets its objectives by evaluating ease of use, utility of two sets of performance predictions, and the power of the language constructs. The technical approach followed to meet these objectives was to design, develop, and test an evaluation prototype of a graphical, performance prediction tool. The utility of the prototype was then evaluated by applying it to a variety of test cases found in the literature and in AST case histories. Numerous models were constructed and successfully tested. The major conclusion of this Phase 2 SBIR research and development effort is that complex, real-time computer systems can be specified in a non-procedural manner using combinations of icons, windows, menus, and dialogs. Such a specification technique provides an interface that system designers and architects find natural and easy to use. In addition, PEDESTAL's multiview approach provides system engineers with the capability to perform the trade-offs necessary to produce a design that meets timing performance requirements. Sample system designs analyzed during the development effort showed that models could be constructed in a fraction of the time required by non-visual system design capture tools.

Wright, Gary↗

NASA NTRS2023

Loft: An Automated Mesh Generator for Stiffened-Shell Aerospace Vehicles

Loft is an automated mesh generation code designed for aerospace vehicle structures. Based on user input, it can generate meshes for wings, noses, tanks, fuselage sections, thrust structures, etc. As the mesh is generated, each element is assigned properties that mark what part of the vehicle it is associated with. This property assignment is an extremely powerful feature making possible detailed analysis tasks such as load application and sizing. Loft can save its meshes in NASTRAN bulk data deck, EDS’ I-DEAS Universal File format, Abaqus input file format, VRML 2.0 (Virtual Reality Modeling Language) and STL (Stereo Lithography) files. The property assignment scheme was designed to make sizing in Collier Research’s HyperSizer easy. Support for other mesh storage formats can be added as needed.

Structural modeling↗

NASA NTRS2015

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASA's Space Launch System

The development of the Space Launch System (SLS) launch vehicle requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The characteristics of these systems must be matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large complex systems engineering challenge being addressed in part by focusing on the specific subsystems handling of off-nominal mission and fault tolerance. Using traditional model based system and software engineering design principles from the Unified Modeling Language (UML), the Mission and Fault Management (M&FM) algorithms are crafted and vetted in specialized Integrated Development Teams composed of multiple development disciplines. NASA also has formed an M&FM team for addressing fault management early in the development lifecycle. This team has developed a dedicated Vehicle Management End-to-End Testbed (VMET) that integrates specific M&FM algorithms, specialized nominal and off-nominal test cases, and vendor-supplied physics-based launch vehicle subsystem models. The flexibility of VMET enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the algorithms utilizing actual subsystem models. The intent is to validate the algorithms and substantiate them with performance baselines for each of the vehicle subsystems in an independent platform exterior to flight software test processes. In any software development process there is inherent risk in the interpretation and implementation of concepts into software through requirements and test processes. Risk reduction is addressed by working with other organizations such as S&MA, Structures and Environments, GNC, Orion, the Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission and Loss of Crew probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and detection and responses that can be tested in VMET and confirm that responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - ARINC 653 partitioned OS, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM. The plan for VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by Flight Software. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure the effectiveness of M&FM algorithms performance in the FSW development and test processes. This paper is outlined in a systematic fashion analogous to a lifecycle process flow for engineering development of algorithms into software and testing. Section I describes the NASA SLS M&FM context, presenting the current infrastructure, leading principles, methods, and participants. Section II defines the testing philosophy of the M&FM algorithms as related to VMET followed by section III, which presents the modeling methods of the algorithms to be tested and validated in VMET. Its details are then further presented in section IV followed by Section V presenting integration, test status, and state analysis. Finally, section VI addresses the summary and forward directions followed by the appendices presenting relevant information on terminology and documentation.

Trevino, Luis↗

DOE OSTI2025

AI Model Benchmarking for Nonproliferation Applications: Steel Thread Benchmarking Task Force Technical Report (Rev. 2)

Steel Thread is a NA-22 venture that seeks to build trustworthy, reliable AI models that can be used in a wide variety of nonproliferation tasks. A key aspect of building these models is developing appropriate benchmarks and evaluation methods, which will enable the venture to identify and adapt models to provide the most value in the nonproliferation domain. Benchmarks must be relevant to key tasks in this domain, such as question answering, information retrieval, document summarization and classification, consensus analysis, and image and data analysis. This report 1) provides an overview of benchmark design, evaluation, and challenges; 2) reviews a variety of open benchmarks, with a focus on language models and tasks; and 3) identifies benchmarks that are most relevant to Steel Thread. This report is intended to serve as a basis for further efforts to classify and evaluate benchmarks and their correlation with success on nonproliferation-specific tasks. The Steel Thread venture has defined benchmarks to be a particular combination of a dataset (or datasets) and a metric (or metrics) conceptualized as representing one or more specific tasks or sets of abilities for a specific modality. It is adopted by a research community as a shared framework for comparing methods.1 It includes 1) Data: Labeled (a designated subset not used for training, which could be all the data), 2) Metric: A way to quantify performance, 3) Task/Ability: What the benchmark is testing, 4) Protocol: A structured and repeatable evaluation process, 5) Baseline/Reference Model: For comparison; could be statistical, rule-based, SME-derived, or another model, and 6) Maintenance Plan: to update with new information over time; important for long-term utility. For further clarity, the definition includes what a benchmark, in this context, is not. It is not a corpus of training data, specific to a model (it is intended to apply to a range of models), a universal evaluation of performance, a guarantee that the ‘top’ model on the leaderboard will be the best fit for every specific use case, an all-encompassing proof of a model’s universal quality, nor is it a one-size-fits-all measure of success. It does not cover every real-world constraint (like operational, ethical, or cost considerations), a systems integration test, or a unit test. This definition was inspired by and resulted from discussions within the Steel Thread Benchmarking Task Force. This group was formed to define what we would mean as a benchmark within Steel Thread but persisted as the need to develop a thorough understanding of the large and expanding existing benchmarking space. This technical report is a result of the group’s divide and conquer approach to exploring this space. The release of benchmarks might not be progressing as quickly as model development, but it is moving very fast, as many benchmarks quickly become saturated, when state-of-the-art models score so close to the benchmark’s ceiling that their results are virtually indistinguishable. At that point, the test no longer differentiates between new systems, so researchers usually stop reporting scores as the benchmark no longer informs about improvements from the next generation of models. In the OpenAI announcement of GPT-5, they reported results on six flagship public benchmarks (AIME 2025, SWE-bench Verified, Aider Polyglot, MMMU, HealthBench Hard, GPQA) but the full system-card covers roughly thirty-five separate evaluations, comprising hundreds of test task items in total. There have been some efforts to summarize benchmarks in specific fields, like for text-to-image generation, but these surveys have had a narrow methodology scope. Therefore, a comprehensive survey of all benchmarks or even all benchmarks that could be relevant to Steel Thread is outside of the scope of this report. We chose some specific benchmarks to investigate in detail.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Third-Party Supplier Risk Re-Classification Using Multi-Model Semantic Voting and External Web Augmentation

Risk decisions in many third-party risk management (TPRM) workflows rely on static inherent risk questionnaires (IRQ). These static forms provide a snapshot of the vendor from the business users’ perspective, as these requests are processed without cross-referencing for evidence. Consequently, responses can be misinformed or embellished with inaccuracies, thereby masking the vendor’s true risk to the enterprise. This paper presents a multi-stage verification framework to augment IRQs with web evidence and a deterministic ensemble of large language model assessors to reclassify risk. In a case study of 100 submissions previously misclassified as low risk, the proposed framework correctly identified 76% of the cases as high risk, while the existing workflow identified none. McNemar’s continuity corrected statistics of 74 were obtained with a two sided p-value of 2.65 × 10-23, indicating a significantly more effective workflow compared to the legacy model.

99 - GENERAL AND MISCELLANEOUS↗

NASA NTRS2005

Doclet To Synthesize UML

The RoseDoclet computer program extends the capability of Java doclet software to automatically synthesize Unified Modeling Language (UML) content from Java language source code. [Doclets are Java-language programs that use the doclet application programming interface (API) to specify the content and format of the output of Javadoc. Javadoc is a program, originally designed to generate API documentation from Java source code, now also useful as an extensible engine for processing Java source code.] RoseDoclet takes advantage of Javadoc comments and tags already in the source code to produce a UML model of that code. RoseDoclet applies the doclet API to create a doclet passed to Javadoc. The Javadoc engine applies the doclet to the source code, emitting the output format specified by the doclet. RoseDoclet emits a Rose model file and populates it with fully documented packages, classes, methods, variables, and class diagrams identified in the source code. The way in which UML models are generated can be controlled by use of new Javadoc comment tags that RoseDoclet provides. The advantage of using RoseDoclet is that Javadoc documentation becomes leveraged for two purposes: documenting the as-built API and keeping the design documentation up to date.

Barry, Matthew R.↗

NASA NTRSDate not supplied

A Model-Based Systems Engineering Journey to Developing a Concept of Operations

Starting in 2017, NASA’s Human Research Program (HRP) Exploration Medical Capability (ExMC) element began a systems engineering transition from traditional, document-centric development to model-centric development when defining its foundation medical systems. These foundation medical systems define a Concept of Operations (ConOps) and identify the generic requirements for a medical system based on assumptions about a generic crew and mission environments and guidance from NASA standards (e.g., Medical “Levels of Care”). By making the transition, ExMC intends to improve communication among stakeholders about foundation medical system requirements and content. In addition, this transition will enable ExMC to lower both development and crew treatment risks for future, mission-specific medical systems. ExMC followed a Model Based Systems Engineering (MBSE) paradigm when developing the foundation medical systems. A model-based approach provides several advantages over a traditional, document-centric approach. First, when Systems Engineers (SE) develop diagrams in a model using a standard modeling language, they produce information dense pictures that facilitate understanding much more efficiently with less room for misinterpretation than text. Second, due to the evolving nature of projects, documentation becomes out of date the minute it is published. This can result in people making decisions based on information that is no longer current, especially if they are referencing a locally-stored copy of a document. A model, on the other hand, is always up to date with the latest approved changes and information. It serves as a single point of truth. Third, a model-centric approach centralizes all important information in one place. Rather than having to flip through separate ConOps documents, design specifications, requirements specifications, and the like to coordinate information, a model captures the content in one, integrated spot. This integration makes tracing information from end-to-end easier with greater reliability. The ExMC Systems Engineering Lifecycle follows a well-defined process. ExMC Systems Engineers perform all major steps of the process, regardless of the development methodology. One of the first steps in the process is developing the ConOps that describes the operation of the system from the point of view of the users. It includes a list of the users and their needs, the goals of the medical system, key assumptions about the system, and definitions of the medical system’s operational environments. For this development effort, ExMC chose to replace the traditional text-based ConOps document with a model. While the decision to change the development workflow was not difficult, implementing the structural and organizational workflows were. It required showing ExMC’s users, most of whom are not Systems Engineers, how the information they require would be presented in the model and to gain their acceptance of this approach. This paper documents key lessons learned during the ConOps transformation by focusing on how the model represents information, the agile workflow used by SEs when developing the model and how it integrates into a project plan, how leadership influenced key users to accept the transformation, and how the users interact with the model information.

Jeffrey Robert Cohen↗

DOE OSTI2023

PRIMED for the Future: Purposing Raw Intake for Machine Learning-Enabled Detection (Final Report)

The COVID-19 pandemic demonstrated how a novel, elusive, and diffuse biological threat can engender uncertainty and misinformation, and it underscored the need for flexible analytical modalities agnostic to the identity of biological material. Yet even before the pandemic, recognition of the limitations of the current, list-based approach, which focuses on known pathogens and biotoxins, and of the importance of agent-agnostic biodetection was growing within the biosecurity community. In a 2018 report on “Biodefense in the Age of Synthetic Biology,” for example, the National Academy of Sciences stated that “an overreliance on the Select Agent List is a systematic weakness affecting many aspects of the United States’ current biodefense mitigation capability”. More recently, a group of biodefense researchers proposed the identification and adoption of “bioagent-agnostic signatures (BASs)” as a way of detecting and characterizing not only existing agents but also novel ones, an approach they believe will “enable a more flexible and resilient biodefense posture”. Indeed, the future of biodetection requires us to begin developing novel analytics that can identify anomalies and/or characteristics that indicate a potential threat, whether known or unknown, without looking for a specific signature that has been identified previously. To assess potential threats more rapidly, it is critical to develop agnostic artificial intelligence (AI)/machine learning (ML) systems that can be employed for real-time assessment of the nature and source of a perturbation. Such systems should be multi scale and multi-dimensional, integrating sensor data from a range of biological, chemical, and physical application spaces. Emerging deep learning (DL) models demonstrate exceptional promise for identification of discriminatory features within multi-dimensional datasets. DL models have the capacity to recognize and encode highly complex patterns in a wide range of input data modalities, including images, text, and biological/chemical/physical spectra. As such, they can execute a wide range of assessments and determinations that have traditionally required a human operator. The promise of advances in DL is apparent in the realm of human health and medicine. DL models have been validated for evaluating a variety of clinical threats to human health in a range of contexts, including infection and cancer, and they demonstrated improved performance in predicting stroke relative to human neurologists in some categories of data. Continuously evolving advances in AI/ML are expected to support more efficient evaluation of raw sequence, spectroscopy, and spectrometry data. For instance, recent advances and deployment of large language models (LLM) such as Generative Pre training Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) have already motivated application of these models for biological function prediction. As frameworks such as LLMs become larger and more complex in their representations, their capacity to serve as pre-trained models that can be fine-tuned for biological/biodetection purposes will similarly be amplified. While existing and emerging AI/ML have found broad applicability and use cases in the clinical sciences, development for environmental evaluation and biodetection has been limited. Functionalizing such capabilities for this purpose requires an understanding of the existing technical landscape and how the respective tools and algorithms are currently being employed. This landscape awareness then allows an assessment of the current practical capabilities of existing models and the anticipated requirements and development efforts that will be needed to adapt available algorithms for biodetection applications relevant to DHS. Leveraging expertise in biodetection, ML, and operational biodetection, the effort described in this report is comprised of a systematic landscape assessment (Subtask 2.1), comparative evaluation (Subtask 2.2), and formulation of a value proposition (Subtask 2.3) for the prospect of ML-enabled, agnostic biodetection from raw, or minimally-processed, datasets.

59 BASIC BIOLOGICAL SCIENCES↗