Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Complex networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,657 records · Page 92
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

SPANet: Generalized permutationless set assignment for particle physics using symmetry preserving attention

The creation of unstable heavy particles at the Large Hadron Collider is the most direct way to address some of the deepest open questions in physics. Collisions typically produce variable-size sets of observed particles which have inherent ambiguities complicating the assignment of observed particles to the decay products of the heavy particles. Current strategies for tackling these challenges in the physics community ignore the physical symmetries of the decay products and consider all possible assignment permutations and do not scale to complex configurations. Attention based deep learning methods for sequence modelling have achieved state-of-the-art performance in natural language processing, but they lack built-in mechanisms to deal with the unique symmetries found in physical set-assignment problems. We introduce a novel method for constructing symmetry-preserving attention networks which reflect the problem's natural invariances to efficiently find assignments without evaluating all permutations. This general approach is applicable to arbitrarily complex configurations and significantly outperforms current methods, improving reconstruction efficiency between 19% - 35% on typical benchmark problems while decreasing inference time by two to five orders of magnitude on the most complex events, making many important and previously intractable cases tractable. A full code repository containing a general library, the specific configuration used, and a complete dataset release, are available at https://github.com/Alexanders101/SPANet

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1990

A failure diagnosis system based on a neural network classifier for the space shuttle main engine

A conceptual design of a model based failure detection and diagnosis system is developed for the space shuttle main engine. This design relies on the accurate and reliable identification of the parameters of the highly nonlinear and very complex engine. The design approach is presented in some detail and results for a failed valve are presented. These preliminary results verify that the developed parameter identification technique together with a neural network classifier can be used for this purpose.

Duyar, Ahmet↗
NASA NTRS1990

A failure diagnosis system based on a neural network classifier for the Space Shuttle Main Engine

A conceptual design of a model based failure deeteection and diagnosis system is deeveloped for the Space Shuttle Main Engine. This design relies on the accurate and reliable identification of the parameters of the highly nonlinear and very complex engine. The design approach is presented in some detail and results for a failed valve are presented. These preliminary results verify that the developed parameter identification technique, together with a neural network classifier, can be used for this purpose.

Duyar, Ahmet↗
DOE OSTI2020

Neural networks for modeling and control of particle accelerators

Charged particle accelerators support a wide variety of scientific, industrial, and medical applications. They range in scale and complexity from systems with just a few components for beam acceleration and manipulation, to large scientific user facilities that span many kilometers and have hundreds-to-thousands of individually-controllable components. Specific operational requirements must be met by adjusting the many controllable variables of the accelerator. Meeting these requirements can be challenging, both in terms of the ability to achieve specific beam quality metrics in a reliable fashion and in terms of the time needed to set up and maintain the optimal operating conditions. One avenue toward addressing this challenge is to incorporate techniques from the fields of machine learning (ML) and artificial intelligence (AI) into the way particle accelerators are modeled and controlled. While many promising approaches within AI/ML could be used for particle accelerators, this dissertation focuses on approaches based on neural networks. Neural networks are particularly well-suited to modeling, control, and diagnostic analysis of nonlinear systems, as well as systems with large parameter spaces. They are also very appealing for their ability to process high-dimensional data types, such as images and time series (both of which are ubiquitous in particle accelerators). In this work, key studies that demonstrated the potential utility of modern neural network-based approaches to modeling and control of particle accelerators are presented. The context for this work is important: at the start of this work in 2012, there was little interest in AI/ML in the particle accelerator community, and many of the advances in neural networks and deep learning that enabled its present success had not yet been made at that time. As such, this work was both an exploration of possible application areas and a generator of proof-of-concept demonstrations in these areas.

43 PARTICLE ACCELERATORS↗
NASA NTRS1995

DREAMS and IMAGE: A Model and Computer Implementation for Concurrent, Life-Cycle Design of Complex Systems

Computing architectures are being assembled that extend concurrent engineering practices by providing more efficient execution and collaboration on distributed, heterogeneous computing networks. Built on the successes of initial architectures, requirements for a next-generation design computing infrastructure can be developed. These requirements concentrate on those needed by a designer in decision-making processes from product conception to recycling and can be categorized in two areas: design process and design information management. A designer both designs and executes design processes throughout design time to achieve better product and process capabilities while expanding fewer resources. In order to accomplish this, information, or more appropriately design knowledge, needs to be adequately managed during product and process decomposition as well as recomposition. A foundation has been laid that captures these requirements in a design architecture called DREAMS (Developing Robust Engineering Analysis Models and Specifications). In addition, a computing infrastructure, called IMAGE (Intelligent Multidisciplinary Aircraft Generation Environment), is being developed that satisfies design requirements defined in DREAMS and incorporates enabling computational technologies.

Hale, Mark A.↗
DOE OSTI2025

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗
DOE OSTI2021

Physics-based, neural network force fields for reactive molecular dynamics: Investigation of carbene formation from [EMIM + ][OAc - ]

Reactive molecular dynamics simulations enable detailed understanding of solvent effects on chemical reaction mechanisms and reaction rates. While classical molecular dynamics using reactive force fields allows significantly longer simulation time scales and larger system sizes compared with ab initio molecular dynamics, constructing reactive force fields is a difficult and complex task. In this work, we describe a general approach following the Empirical Valence Bond (EVB) framework for constructing ab initio reactive force fields for condensed phase simulations by combining physics-based methods with neural networks (PB/NN). The physics-based terms ensure correct asymptotic behavior of electrostatic, polarization, and dispersion interactions, and are compatible with existing solvent force fields. Neural networks are utilized for versatile description of short-range orbital interactions within the transition state region, and accurate rendering of vibrational motion of the reacting complex. Herein, we demonstrate our methodology for a simple deprotonation reaction of the 1-ethyl-3-methylimidazolium (EMIM+) cation with acetate to form 1-ethyl- 3-methylimidazol-2-ylidene and acetic acid. Our PB/NN force field exhibits ~ 1 kJ/mol MAE accuracy within the transition state region for the gas-phase complex. To characterize solvent modulation of the reaction profile, we compute potentials of mean force (PMFs) for the gas-phase reaction as well as the reaction within a four ion cluster, and benchmark against ab initio molecular dynamics simulations. We find that the surrounding ionic environment significantly destabilizes formation of the carbene product, and we show that this effect is accurately captured by the reactive force field. By construction, the PB/NN potential may be directly employed for simulations of other solvents/chemical environments without additional parameterization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Neural network based analysis of multimodal bond distributions using extended x-ray absorption fine structure spectra

Knowledge of the local coordination environment around atomic species in functional materials is critical for understanding their mechanisms of operation. Heterogeneous mixtures of metal complexes are ubiquitous in catalysts, ionic liquids, molten salts, biological enzymes, and geochemical systems, among many others. Extracting information from ensemble-average measurements about the structural and compositional descriptors of each type of coordination complex comprising the mixture is not generally possible, especially when they possess multimodal bond-length distributions. Here, we developed a method that enables the mapping of an x-ray absorption spectrum on the radial distribution function describing the average environment of the metal ions. The supervised neural network based method utilizes an objective training set, for which the choice of the local structural motifs is completely agnostic to the theoretically expected structure and dynamics of the modeled system. The method was validated using first-principles modeling of structural dynamics of nickel complexation in molten salts, and it applies to a large class of heterogeneous systems, including those studied under in situ and operando conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Development of message passing-based graph convolutional networks for classifying cancer pathology reports

Abstract Background Applying graph convolutional networks (GCN) to the classification of free-form natural language texts leveraged by graph-of-words features (TextGCN) was studied and confirmed to be an effective means of describing complex natural language texts. However, the text classification models based on the TextGCN possess weaknesses in terms of memory consumption and model dissemination and distribution. In this paper, we present a fast message passing network (FastMPN), implementing a GCN with message passing architecture that provides versatility and flexibility by allowing trainable node embedding and edge weights, helping the GCN model find the better solution. We applied the FastMPN model to the task of clinical information extraction from cancer pathology reports, extracting the following six properties: main site, subsite, laterality, histology, behavior, and grade. Results We evaluated the clinical task performance of the FastMPN models in terms of micro- and macro-averaged F1 scores. A comparison was performed with the multi-task convolutional neural network (MT-CNN) model. Results show that the FastMPN model is equivalent to or better than the MT-CNN. Conclusions Our implementation revealed that our FastMPN model, which is based on the PyTorch platform, can train a large corpus (667,290 training samples) with 202,373 unique words in less than 3 minutes per epoch using one NVIDIA V100 hardware accelerator. Our experiments demonstrated that using this implementation, the clinical task performance scores of information extraction related to tumors from cancer pathology reports were highly competitive.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. 1. Due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. 2. Training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNNs called Skip-Connected Self-Recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced in a stereotyped manner by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method called backpropagated intrinsic plasticity (BIP) to further boost the performance of ScSr-SNNs by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Here, based on challenging speech, neuromorphic speech, and neuromorphic image datasets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗
DOE OSTI2024

HumoNet: A Framework for Realistic Modeling and Simulation of Human Mobility Network

Understanding, analyzing, and predicting human mobility and dynamics are valuable to solving pressing problems, developing effective plans, and prescribing timely remedies. As a computational approach, realistic human mobility simulations allow us to understand, analyze, and predict complex systems, including human societies. Accurate simulations rely on (1) the model that captures interactions and behaviors of myriad entities in our society and (2) the mapping of model instances to real-world entities. Taking this into account, this paper introduces the Human Mobility Network simulation framework (HumoNet), an integrated patterns of life (POL) simulation framework that leverages real-world data layers including transportation networks, points of interest, populations, popularity, and human trajectories. HumoNet is a data informed model in which agents are equipped with activities, locomotion, and planning capabilities. To simulate realistic kinematic maneuvers of individuals in transportation networks, HumoNet harnesses a microscopic traffic simulator that provides interaction among vehicles and traffic objects. In this paper, we describe the framework, outline our methodologies, and discuss the data processing and challenges of each data layer. Through experiments, we demonstrate that our simulations capture key features of human mobility by comparing them to the literature and real data using standard measures of human mobility (i.e., the radius of gyration, number of locations visited, level of exploration) and metrics scoring (i.e., Jensen-Shannon divergence). We envision that the synthetic data produced by HumoNet will serve as a benchmark for analyzing epidemics, deploying EV charging networks, and validating AI/ML tasks such as location prediction.

Kim, Joon-Seok↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2025

Real-World Cyber Security Demonstration for Networked Electric Drives

In this article, we present the design and implementation of a cyber-physical security testbed for networked electric drive systems, aimed at conducting real-world security demonstrations. To our knowledge, this is one of the first security testbeds for networked electric drives, seamlessly integrating the domains of power electronics and computer science, and cybersecurity. By doing so, the testbed offers a comprehensive platform to explore and understand the intricate and often complex interactions between cyber and physical systems. The core of our testbed consists of four electric machine drives, meticulously configured to emulate small-scale but realistic information technology (IT) and operational technology (OT) networks. This setup both provides a controlled environment for simulating a wide array of cyber-attacks, and mirrors potential real-world attack scenarios with a high degree of fidelity. The testbed serves as an invaluable resource for the study of cyber-physical security, offering a practical and dynamic platform for testing and validating cybersecurity measures in the context of networked electric drive systems. As a concrete example of the testbed's capabilities, we have developed and implemented a Python-based script designed to execute step-stone attacks over a wireless local area network (WLAN). This script leverages a sequence of target IP addresses, simulating a real-world attack vector that could be exploited by adversaries. To counteract such threats, we demonstrate the efficacy of our developed cyber-attack detection algorithms, which are integral to our testbed's security framework. Furthermore, the testbed incorporates a real-time visualization system using InfluxDB and Grafana, providing a dynamic and interactive representation of networked electric drives and their associated security monitoring mechanisms. This visualization component not only enhances the testbed's usability but also offers insightful, real-time data for researchers and practitioners, thereby facilitating a deeper understanding of cyber-physical security dynamics in networked electric drive systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Multilayered Network Models for Security: Enhancing System Security Engineering with Orchestration

Security engineering approaches can often focus on a particular domain—physical security, cyber security, or personnel security, for example. Yet, security systems engineering consistently faces challenges requiring socio-technical solutions to address evolving and dynamic complexity. While some drivers of this complexity stem from complex risk environments, innovative adversaries, and disruptive technologies, other drivers are endogenous and emerge from the interactions across security engineering approaches. In response, INCOSE's Systems Security Working Group identified the need to better coordinate “disparate security solutions [that] operate independently” as one of eleven key concepts in their IS21 FuSE Security Roadmap. From this perspective, this need for “security orchestration” aligns with the perspective that security is a property that emerges from interactions within complex systems. Current efforts at Sandia National Laboratories are developing a systems security engineering approach that describes high consequence facility (HCF) security as a multidomain set of interacting layers. The result is a multilayered network (MLN)-based approach that captures the interactions between infrastructure, physical components, digital components, and humans in nuclear security systems. This article will summarize the MLN-based approach to HCF security and describe two preliminary results demonstrating potential benefits from incorporating interactions across disparate security solutions. Here, leveraging the logical structure of networks, this MLN model-based approach provides an example of how security orchestration provides enhanced systems security engineering solutions.

42 ENGINEERING↗
DOE OSTI2023

Mobiliti v1.0

Mobiliti is a software platform designed to emulate the dynamics of a regional transportation road network. It is built on open-source software that provides parallel discrete-event simulation. The software is transformative in the area of transportation network simulation because of the geospatial scale and fidelity of the network model and the computational time it takes to model a full day of travel demand. For example, it runs a simulation of the entire San Francisco Bay Area, with a network model of ~1M links and a population that completes ~19M trips in ~5 minutes. This scale of simulation has not been attempted with existing simulation models due to the complexity of the model and the computational time it would take to complete. The intent of the software is to create a digital twin capability for cities to evaluate consequences of infrastructure or policy changes on road network dynamics.

Macfarlane, Jane↗
DOE OSTI2025

A generalizable machine learning-assisted fast Fourier transform algorithm to simulate the large strain phenomena in polycrystalline materials

Machine learning methods have shown initial promise in constitutive modeling for single crystals or homogenized polycrystals, delivering notable computational efficiency. However, existing machine learning-based constitutive models often lack generalizability, limiting their application across diverse boundary value problems. This study introduces a thermodynamics-informed artificial neural network model to accelerate rate-tangent crystal plasticity fast Fourier transform simulations for cross-scale deformation behaviors of polycrystals under complex loading. Our model integrates microstructural variability and local interactions effectively. To address local effects in each grain, we employ K-means clustering to group Gauss points within the microstructure into clusters assumed to be in similar mechanical states. This approach, based on self-clustering analysis, extends model scope from macroscopic stress response to the granular level, capturing mechanical responses and orientation evolution across grains. This reduces the number of nonlinear problems to solve, with cluster responses propagated throughout each group. The thermodynamics-based artificial neural network-extracted features are further processed using local material state clusters to account for history-dependent deformation and evolving microstructures. Additionally, representative volume element simulations with rate-tangent crystal plasticity fast Fourier transform provide reliable datasets for model training. The proposed model demonstrates high efficiency, accuracy, self-consistency, and enhanced generalizability in predicting strain–stress responses and orientation evolution at both individual grain and aggregate scales under complex loading conditions, such as biaxial tension and arbitrary loading scenarios.

36 MATERIALS SCIENCE↗