Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Complex networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Cognition at the Point of Sensing

Over the last 15 years, compressive sensing techniques have been developed which have the potential to greatly reduce the amount of data collected by systems while preserving the amount of information obtained. A cost of this efficiency is that a computationally-intensive optimization routine must be used to put the sensed data into a form that a person can interpret. At the same time, machine learning techniques have experienced tremendous growth as well. Machines have demonstrated the ability learn how to effectively perform tasks such as detection and classification at speeds much faster than humanly possible. Our goal in this project was to study the feasibility of using compressive sensing systems "at the edge." That is, how can compressive sensing sensors be deployed such that information is created at the remote sensor rather than sending raw data to a central processing location? Studies were performed to analyze whether machine learning could be done on the compressively sensed data in its raw form. If a machine is performing the task, is it possible to do so without putting the data into a human interpretable form? We show that this is possible for some systems, in particular a compressive sensing snapshot imaging spectrometer. Machine learning tasks were demonstrated to be more effective and more robust to noise when the machine learning algorithm worked on data in its raw form. This system is shown to outperform a traditional spectrometer. Techniques for reducing the complexity of the reconstruction routine were also analyzed. Techniques for such as data regularization, deep neural networks, and matrix completion were studied and shown to have benefits over traditional reconstruction techniques. In this project we showed that compressive sensing sensors are indeed feasible at the edge. As always, sensors and algorithms must be carefully tuned to work in the constrained environment. In this project we developed tools and techniques to enable those analyses.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Optimizing Carbon Capture, Transport, and Storage: Overcoming Challenges with Machine Learning and Cost-Benefit Analysis

Carbon Capture, Utilization, and Storage (CCUS) is a critical strategy for reducing CO₂ emissions and mitigating climate change. However, its widespread deployment faces numerous challenges across the capture, transport, and storage phases. These challenges include the technical complexity of predicting subsurface behaviors during CO₂ injection, ensuring long-term storage integrity, optimizing transportation networks, and balancing the economic and environmental trade-offs. Addressing these issues requires an integrated approach combining advanced subsurface modeling with system-level analyses to assess costs, risks, and benefits. This presentation provides an overview of studies conducted by the National Energy Technology Laboratory (NETL) to tackle these challenges. NETL’s efforts encompass cutting-edge research in subsurface fluid behavior machine learning predictions, alongside the development of innovative tools for system optimization and economic evaluation. By bridging technical expertise and strategic analysis, NETL aims to advance the deployment of CCUS technologies to support global decarbonization efforts. Presented at the Carnegie Mellon University CEE IESS Student Seminar October 4, 2024.

Shih, Chung Yan↗
DOE OSTI2023

High-Reliability Systems and the Control of National Security Data and Information

High-reliability systems are characterized by catastrophic implications in the event of failure. These implications can include substantive damage to the environment, social order, and loss of life. Examples of high-reliability systems include nuclear submarines, nuclear reactors, the electric grid, and nuclear weapons. Due to the catastrophic implications of failure, there are heightened awareness and control mechanisms surrounding related data and information. However, defining the difference between data and information is often ambiguous across scholarly disciplines and in United States policy and legislation. For high-reliability systems, the implications of ambiguity between data and information may affect the security of United States interests and even cost lives. For security, data are raw facts or figures without context, while information is the compilation or articulation of data that forms context. Security depends on clarity in the differences between data and information and how to control them. Control is necessary to ensure that data and information are not unintentionally released to foreign governments, the public, or those without need-to-know. A primary concern in the practice of security is the control of data to avoid the unintended conversion to information. Intra-institutionally, this control is highly complex given the amalgam of legacy data systems and the numerous and constantly evolving nature of modern data systems that were not necessarily designed to be integrated. The complexity of this concern is augmented when institutions are part of inter-institutional collaborations or networks of public-private partnerships that share data and information. Additionally, institutions that share data as a function of policy and legislative action—particularly formally integrated data and information system infrastructures—may be at higher security risk. This paper will present an intra-institutional paradigm that utilizes and integrates concepts from numerous disciplines to frame a critical and underspecified practical issue in security—controlling for the unintended conversion of data to information.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

ℤ2 × ℤ2 Equivariant Quantum Neural Networks: Benchmarking against Classical Neural Networks

This paper presents a comparative analysis of the performance of Equivariant Quantum Neural Networks (EQNNs) and Quantum Neural Networks (QNNs), juxtaposed against their classical counterparts: Equivariant Neural Networks (ENNs) and Deep Neural Networks (DNNs). We evaluate the performance of each network with three two-dimensional toy examples for a binary classification task, focusing on model complexity (measured by the number of parameters) and the size of the training dataset. Our results show that the Z2×Z2 EQNN and the QNN provide superior performance for smaller parameter sets and modest training data samples.

Dong, Zhongtian (ORCID:0000000210003454)↗
DOE OSTI2023

Less Is More: Oligomer Extraction and Hydrothermal Annealing Increase PDMS Adhesion Forces for Materials Studies and for Biology-Focused Microfluidic Applications

Cues in the micro-environment are key determinants in the emergence of complex cellular morphologies and functions. Primary among these is the presence of neighboring cells that form networks. For high-resolution analysis, it is crucial to develop micro-environments that permit exquisite control of network formation. This is especially true in cell science, tissue engineering, and clinical biology. We introduce a new approach for assembling polydimethylsiloxane (PDMS)-based microfluidic environments that enhances cell network formation and analyses. We report that the combined processes of PDMS solvent-extraction and hydrothermal annealing create unique conditions that produce high-strength bonds between solvent-extracted PDMS (E-PDMS) and glass—properties not associated with conventional PDMS. Extraction followed by hydrothermal annealing removes unbound oligomers, promotes polymer cross-linking, facilitates covalent bond formation with glass, and retains the highest biocompatibility. Herein, our extraction protocol accelerates oligomer removal from 5 to 2 days. Resulting microfluidic platforms are uniquely suited for cell-network studies owing to high adhesion forces, effectively corralling cellular extensions and eliminating harmful oligomers. We demonstrate the simple, simultaneous actuation of multiple microfluidic domains for invoking ATP- and glutamate-induced Ca 2+ signaling in glial-cell networks. These E-PDMS modifications and flow manipulations further enable microfluidic technologies for cell-signaling and network studies as well as novel applications.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Revisiting Current Paradigms: Subject Matter Expert Views on High Consequence Facility Security Assessments

Security assessments support decision-makers' ability to evaluate current capabilities of high consequence facilities (HCF) to respond to possible attacks. However, increasing complexity of today's operational environment requires a critical review of traditional approaches to ensure that implemented assessments are providing relevant and timely insights into security of HCFs. Using interviews and focus groups with diverse subject matter experts (SMEs), this study evaluated the current state of security assessments and identified opportunities to achieve a more "ideal" state. The SME-based data underscored the value of a systems approach for understanding the impacts of changing operational designs and contexts (as well as cultural influences) on security to address methodological shortcomings of traditional assessment processes. These findings can be used to inform the development of new approaches to HCF security assessments that are able to more accurately reflect changing operational environments and effectively mitigate concerns arising from new adversary capabilities.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Neural Networks-Based Inverter Control: Modeling and Adaptive Optimization for Smart Distribution Networks

The optimal voltage control of inverter-based resources, especially under the high penetration of solar photovoltaics, is critical to the stability of the distribution power system. However, the computational complexity as well as the coordinated operation performance of the voltage control optimization in the distribution power system limits the real-time applications. To mitigate this issue, a model-free based adaptive optimal control scheme for the smart inverter is proposed to maximize the active power generation, minimize the power loss, and maintain the bus voltages in smart distribution networks. An inverter-based optimization model for coordinated operation is first established, considering the uncertainties of renewable power generation. Subsequently, by collecting the data and control strategies, the neural networks (NNs) based algorithm is proposed to efficiently predict the best possible control strategy. The main objective of this scheme is to accurately predict candidate optimal solutions with near-negligible feasibility and optimization gaps, with the advantage of avoiding complicated iteration-based numerical algorithms. Thereafter, the co-simulation among OpenDSS, MATLAB, and Python is set up to fully take advantage of the three individual software. Experiments are conducted based on different control parameter characteristics and structures of NNs. Finally, the results reveal that an average mean squared error of 0.013 and 1 ms response time are achieved, which is lower than some state-of-the-art methods.

42 ENGINEERING↗
DOE OSTI2022

SPANet: Generalized permutationless set assignment for particle physics using symmetry preserving attention

The creation of unstable heavy particles at the Large Hadron Collider is the most direct way to address some of the deepest open questions in physics. Collisions typically produce variable-size sets of observed particles which have inherent ambiguities complicating the assignment of observed particles to the decay products of the heavy particles. Current strategies for tackling these challenges in the physics community ignore the physical symmetries of the decay products and consider all possible assignment permutations and do not scale to complex configurations. Attention based deep learning methods for sequence modelling have achieved state-of-the-art performance in natural language processing, but they lack built-in mechanisms to deal with the unique symmetries found in physical set-assignment problems. We introduce a novel method for constructing symmetry-preserving attention networks which reflect the problem's natural invariances to efficiently find assignments without evaluating all permutations. This general approach is applicable to arbitrarily complex configurations and significantly outperforms current methods, improving reconstruction efficiency between 19% - 35% on typical benchmark problems while decreasing inference time by two to five orders of magnitude on the most complex events, making many important and previously intractable cases tractable. A full code repository containing a general library, the specific configuration used, and a complete dataset release, are available at https://github.com/Alexanders101/SPANet

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Neural networks for modeling and control of particle accelerators

Charged particle accelerators support a wide variety of scientific, industrial, and medical applications. They range in scale and complexity from systems with just a few components for beam acceleration and manipulation, to large scientific user facilities that span many kilometers and have hundreds-to-thousands of individually-controllable components. Specific operational requirements must be met by adjusting the many controllable variables of the accelerator. Meeting these requirements can be challenging, both in terms of the ability to achieve specific beam quality metrics in a reliable fashion and in terms of the time needed to set up and maintain the optimal operating conditions. One avenue toward addressing this challenge is to incorporate techniques from the fields of machine learning (ML) and artificial intelligence (AI) into the way particle accelerators are modeled and controlled. While many promising approaches within AI/ML could be used for particle accelerators, this dissertation focuses on approaches based on neural networks. Neural networks are particularly well-suited to modeling, control, and diagnostic analysis of nonlinear systems, as well as systems with large parameter spaces. They are also very appealing for their ability to process high-dimensional data types, such as images and time series (both of which are ubiquitous in particle accelerators). In this work, key studies that demonstrated the potential utility of modern neural network-based approaches to modeling and control of particle accelerators are presented. The context for this work is important: at the start of this work in 2012, there was little interest in AI/ML in the particle accelerator community, and many of the advances in neural networks and deep learning that enabled its present success had not yet been made at that time. As such, this work was both an exploration of possible application areas and a generator of proof-of-concept demonstrations in these areas.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗
DOE OSTI2021

Physics-based, neural network force fields for reactive molecular dynamics: Investigation of carbene formation from [EMIM + ][OAc - ]

Reactive molecular dynamics simulations enable detailed understanding of solvent effects on chemical reaction mechanisms and reaction rates. While classical molecular dynamics using reactive force fields allows significantly longer simulation time scales and larger system sizes compared with ab initio molecular dynamics, constructing reactive force fields is a difficult and complex task. In this work, we describe a general approach following the Empirical Valence Bond (EVB) framework for constructing ab initio reactive force fields for condensed phase simulations by combining physics-based methods with neural networks (PB/NN). The physics-based terms ensure correct asymptotic behavior of electrostatic, polarization, and dispersion interactions, and are compatible with existing solvent force fields. Neural networks are utilized for versatile description of short-range orbital interactions within the transition state region, and accurate rendering of vibrational motion of the reacting complex. Herein, we demonstrate our methodology for a simple deprotonation reaction of the 1-ethyl-3-methylimidazolium (EMIM+) cation with acetate to form 1-ethyl- 3-methylimidazol-2-ylidene and acetic acid. Our PB/NN force field exhibits ~ 1 kJ/mol MAE accuracy within the transition state region for the gas-phase complex. To characterize solvent modulation of the reaction profile, we compute potentials of mean force (PMFs) for the gas-phase reaction as well as the reaction within a four ion cluster, and benchmark against ab initio molecular dynamics simulations. We find that the surrounding ionic environment significantly destabilizes formation of the carbene product, and we show that this effect is accurately captured by the reactive force field. By construction, the PB/NN potential may be directly employed for simulations of other solvents/chemical environments without additional parameterization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Neural network based analysis of multimodal bond distributions using extended x-ray absorption fine structure spectra

Knowledge of the local coordination environment around atomic species in functional materials is critical for understanding their mechanisms of operation. Heterogeneous mixtures of metal complexes are ubiquitous in catalysts, ionic liquids, molten salts, biological enzymes, and geochemical systems, among many others. Extracting information from ensemble-average measurements about the structural and compositional descriptors of each type of coordination complex comprising the mixture is not generally possible, especially when they possess multimodal bond-length distributions. Here, we developed a method that enables the mapping of an x-ray absorption spectrum on the radial distribution function describing the average environment of the metal ions. The supervised neural network based method utilizes an objective training set, for which the choice of the local structural motifs is completely agnostic to the theoretically expected structure and dynamics of the modeled system. The method was validated using first-principles modeling of structural dynamics of nickel complexation in molten salts, and it applies to a large class of heterogeneous systems, including those studied under in situ and operando conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Development of message passing-based graph convolutional networks for classifying cancer pathology reports

Abstract Background Applying graph convolutional networks (GCN) to the classification of free-form natural language texts leveraged by graph-of-words features (TextGCN) was studied and confirmed to be an effective means of describing complex natural language texts. However, the text classification models based on the TextGCN possess weaknesses in terms of memory consumption and model dissemination and distribution. In this paper, we present a fast message passing network (FastMPN), implementing a GCN with message passing architecture that provides versatility and flexibility by allowing trainable node embedding and edge weights, helping the GCN model find the better solution. We applied the FastMPN model to the task of clinical information extraction from cancer pathology reports, extracting the following six properties: main site, subsite, laterality, histology, behavior, and grade. Results We evaluated the clinical task performance of the FastMPN models in terms of micro- and macro-averaged F1 scores. A comparison was performed with the multi-task convolutional neural network (MT-CNN) model. Results show that the FastMPN model is equivalent to or better than the MT-CNN. Conclusions Our implementation revealed that our FastMPN model, which is based on the PyTorch platform, can train a large corpus (667,290 training samples) with 202,373 unique words in less than 3 minutes per epoch using one NVIDIA V100 hardware accelerator. Our experiments demonstrated that using this implementation, the clinical task performance scores of information extraction related to tumors from cancer pathology reports were highly competitive.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. 1. Due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. 2. Training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNNs called Skip-Connected Self-Recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced in a stereotyped manner by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method called backpropagated intrinsic plasticity (BIP) to further boost the performance of ScSr-SNNs by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Here, based on challenging speech, neuromorphic speech, and neuromorphic image datasets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗
DOE OSTI2024

HumoNet: A Framework for Realistic Modeling and Simulation of Human Mobility Network

Understanding, analyzing, and predicting human mobility and dynamics are valuable to solving pressing problems, developing effective plans, and prescribing timely remedies. As a computational approach, realistic human mobility simulations allow us to understand, analyze, and predict complex systems, including human societies. Accurate simulations rely on (1) the model that captures interactions and behaviors of myriad entities in our society and (2) the mapping of model instances to real-world entities. Taking this into account, this paper introduces the Human Mobility Network simulation framework (HumoNet), an integrated patterns of life (POL) simulation framework that leverages real-world data layers including transportation networks, points of interest, populations, popularity, and human trajectories. HumoNet is a data informed model in which agents are equipped with activities, locomotion, and planning capabilities. To simulate realistic kinematic maneuvers of individuals in transportation networks, HumoNet harnesses a microscopic traffic simulator that provides interaction among vehicles and traffic objects. In this paper, we describe the framework, outline our methodologies, and discuss the data processing and challenges of each data layer. Through experiments, we demonstrate that our simulations capture key features of human mobility by comparing them to the literature and real data using standard measures of human mobility (i.e., the radius of gyration, number of locations visited, level of exploration) and metrics scoring (i.e., Jensen-Shannon divergence). We envision that the synthetic data produced by HumoNet will serve as a benchmark for analyzing epidemics, deploying EV charging networks, and validating AI/ML tasks such as location prediction.

Kim, Joon-Seok↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗