Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “network embeddings”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Power System Event Identification Based on Deep Neural Network With Information Loading

Online power system event identification and classification are crucial to enhancing the reliability of transmission systems. In this study, we develop a deep neural network (DNN) based approach to identify and classify power system events by leveraging real-world measurements from hundreds of phasor measurement units (PMUs) and labels from thousands of events. Two innovative designs are embedded into the baseline model built on convolutional neural networks (CNNs) to improve the event classification accuracy. First, we propose a graph signal processing based PMU sorting algorithm to improve the learning efficiency of CNNs. Second, we deploy information loading based regularization to strike the right balance between memorization and generalization for the DNN. Numerical results based on real-world dataset from the Eastern Interconnection of the U.S power transmission grid show that the combination of PMU based sorting and the information loading based regularization techniques help the proposed DNN approach achieve highly accurate event identification and classification results.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Learning Distributed Geometric Koopman Operator for Sparse Networked Dynamical Systems

Koopman operator theory provides an alternative to study nonlinear networked dynamical systems by mapping the state space to an abstract higher dimensional space where the system evolution is linear. Recent works show the application of graph neural networks (GNNs) to learn state to object-centric embeddings and achieve centralized block-wise computation of Koopman operator (KO) under additional assumptions on the underlying agents properties and constraints on the KO structure. However, the computational complexity of learning the Koopman increases exponentially for networked systems where the number of possible system states grows in a combinatorial fashion with the number of nodes. The learning challenge is further amplified for sparse networks by two factors: 1) sample sparsity for learning the Koopman operator in the non-linear space, and 2) the divergence in the dynamics of individual nodes or from one subgraph to another. Our work aims to address these challenge by formulating the representation learning of networked dynamical systems into a multi-agent paradigm and learning the Koopman operator in a distributive manner. The computational as well as performance advantages of distributed Koopman is predominant for sparse networks whereas for fully connected networks, it is shown to coincide with the centralized one. The empirical study on rope system, network of oscillators and a synthetic power system show comparable and superior performance along with computational benefits with the state-of-the-art methods.

Mukherjee, Sayak↗
DOE OSTI2024

Privacy Preserving Federated Learning for Advanced Scientific Ecosystems

We present a framework to provide privacy preserving (PP) federating learning (FL) across multiple computational and experimental facilities. This work joins the compute capabilities of National Energy Research Scientific Computing Center (NERSC) and Oak Ridge National Laboratory Research Cloud (ORC) with simulated experimental data, such as those produced at the SLAC National Accelerator Laboratory and Spallation Neutron Source (SNS). We describe the software infrastructure developed to provide privacy for computational and experimental networks. We developed algorithmic privacy across the federated system by embedding database security, computation, and communication into the federation architecture, utilizing scientific tools developed by the experimental community.

Archibald, Rick [ORNL] (ORCID:0000000245389780)↗
DOE OSTI2023

Revealing the aging process of solid electrolyte interphase on SiOx anode

Abstract As one of the most promising alternatives to graphite negative electrodes, silicon oxide (SiO x ) has been hindered by its fast capacity fading. Solid electrolyte interphase (SEI) aging on silicon SiO x has been recognized as the most critical yet least understood facet. Herein, leveraging 3D focused ion beam-scanning electron microscopy (FIB-SEM) tomographic imaging, we reveal an exceptionally characteristic SEI microstructure with an incompact inner region and a dense outer region, which overturns the prevailing belief that SEIs are homogeneous structure and reveals the SEI evolution process. Through combining nanoprobe and electron energy loss spectroscopy (EELS), it is also discovered that the electronic conductivity of thick SEI relies on the percolation network within composed of conductive agents (e.g., carbon black particles), which are embedded into the SEI upon its growth. Therefore, the free growth of SEI will gradually attenuate this electron percolation network, thereby causing capacity decay of SiO x . Based on these findings, a proof-of-concept strategy is adopted to mechanically restrict the SEI growth via applying a confining layer on top of the electrode. Through shedding light on the fundamental understanding of SEI aging for SiO x anodes, this work could potentially inspire viable improving strategies in the future.

25 ENERGY STORAGE↗
DOE OSTI2024

Complexes of tubulin oligomers and tau form a viscoelastic intervening network cross-bridging microtubules into bundles

Abstract The axon-initial-segment (AIS) of mature neurons contains microtubule (MT) fascicles (linear bundles) implicated as retrograde diffusion barriers in the retention of MT-associated protein (MAP) tau inside axons. Tau dysfunction and leakage outside of the axon is associated with neurodegeneration. We report on the structure of steady-state MT bundles in varying concentrations of Mg 2+ or Ca 2+ divalent cations in mixtures containing αβ-tubulin, full-length tau, and GTP at 37 °C in a physiological buffer. A concentration-time kinetic phase diagram generated by synchrotron SAXS reveals a wide-spacing MT bundle phase (B ws ), a transient intermediate MT bundle phase (B int ), and a tubulin ring phase. SAXS with TEM of plastic-embedded samples provides evidence of a viscoelastic intervening network (IN) of complexes of tubulin oligomers and tau stabilizing MT bundles. In this model, αβ-tubulin oligomers in the IN are crosslinked by tau’s MT binding repeats, which also link αβ-tubulin oligomers to αβ-tubulin within the MT lattice. The model challenges whether the cross-bridging of MTs is attributed entirely to MAPs. Tubulin-tau complexes in the IN or bound to isolated MTs are potential sites for enzymatic modification of tau, promoting nucleation and growth of tau fibrils in tauopathies.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Density of states prediction for materials discovery via contrastive learning from probabilistic embeddings

Abstract Machine learning for materials discovery has largely focused on predicting an individual scalar rather than multiple related properties, where spectral properties are an important example. Fundamental spectral properties include the phonon density of states (phDOS) and the electronic density of states (eDOS), which individually or collectively are the origins of a breadth of materials observables and functions. Building upon the success of graph attention networks for encoding crystalline materials, we introduce a probabilistic embedding generator specifically tailored to the prediction of spectral properties. Coupled with supervised contrastive learning, our materials-to-spectrum (Mat2Spec) model outperforms state-of-the-art methods for predicting ab initio phDOS and eDOS for crystalline materials. We demonstrate Mat2Spec’s ability to identify eDOS gaps below the Fermi energy, validating predictions with ab initio calculations and thereby discovering candidate thermoelectrics and transparent conductors. Mat2Spec is an exemplar framework for predicting spectral properties of materials via strategically incorporated machine learning techniques.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Physics–Informed Neural Networks of the Saint–Venant Equations for Downscaling a Large–Scale River Model

Large-scale river models are being refined over coastal regions to improve the scientific understanding of coastal processes, hazards and responses to climate change. However, coarse mesh resolutions and approximations in physical representations of tidal rivers limit the performance of such models at resolving the complex flow dynamics near the river-ocean interface, resulting in inaccurate simulations of flood inundation. In this research, we propose a machine learning (ML) framework based on the state-of-the-art physics-informed neural network (PINN) to simulate the downscaled flow at the subgrid scale. First, we demonstrate that PINN is able to assimilate observations of various types and solve the one-dimensional (1-D) Saint-Venant equations (SVE) directly. We perform the flow simulations over a floodplain and along an open channel in several synthetic case studies. The PINN performance is evaluated against analytical solutions and numerical models. Our results indicate that the PINN solutions of water depth have satisfactory accuracy with limited observations assimilated. In the case of flood wave propagation induced by storm surge and tide, a new neural network architecture is proposed based on Fourier feature embeddings that seamlessly encodes the periodic tidal boundary condition in the PINN's formulation. Furthermore, we show that the PINN-based downscaling can produce more reasonable subgrid solutions of the along-channel water depth by assimilating observational data. The PINN solution outperforms the simple linear interpolation in resolving the topography and dynamic flow regimes at the subgrid scale. This study provides a promising path towards improving emulation capabilities in large-scale models to characterize fine-scale coastal processes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Cybersecurity Platform and Certification Framework Development for Extreme Fast Charging (XFC)-Integrated Charging Ecosystem (Final Project Report)

This report summarizes a pioneering effort in Electric Vehicle charging infrastructure ecosystem cybersecurity requirements, assessment methodologies, functional verification, as well as embodiment of the key technologies in the form of hardware and software tools being made available to the public. EPRI led a team of experts, as well as a stakeholder coalition encompassing all key actors in the EV charging infrastructure ecosystem that includes eXtreme Fast Charging (XFC) equipment (defined as 200kW or above). EV charging infrastructure in the United States is a patchwork of networks that have continued to grow organically and have been designed to serve the charging needs of the EV owners, who are their customers. In doing so, each network provider, as well as their connected entities such as the cloud Electric Vehicle Service Providers or EVSPs, utility back office, utility AMI networks, payment networks, as well as Original Equipment Manufacturer (EV manufacturer) telematics networks, have designed systems that may work well individually, but no single entity is responsible for the entire ecosystem to be secure in terms of data exchange. Furthermore, there is no uniformity in how each actor has implemented the cybersecurity requirements since no system-wide cybersecurity requirements existed prior to this project. The final project report describes the technical approach guided by the EV charging infrastructure cybersecurity working group, convened specifically for this project. The technical approach included definition of requirements at the ecosystem level, treated as a ‘system of systems’, and then passed down to individual systems (EVSE, EV, cloud EVSP, utility, and the payment networks), followed by developing the cybersecurity risk and vulnerability assessment methods, that were later applied to real-world cyber-physical systems at EPRI, ANL, and NREL laboratories, to validate both the process and the results. Finally, in a spotlight over the most vulnerable equipment, which is the EV charge station (AC or DC), the team developed a multi-layer cybersecurity implementation in the embedded domain embodied by the open-source Secure Network Interface Card (SNIC) demonstrating the various ways in which the infrastructure can be secured protecting against the identified attack surfaces. Finally, the entire process of EV infrastructure cybersecurity assessment was encapsulated in the Electric Vehicle Charging Cybersecurity Management (EVC2M) online GUI-based tool, that is expected to be released to the public. The report presents the objectives, the technical approach, the key results, and recommendations for future work.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Jet rotational metrics

Abstract Embedding symmetries in the architectures of deep neural networks can improve classification and network convergence in the context of jet substructure. These results hint at the existence of symmetries in jet energy depositions, such as rotational symmetry, arising from the physical features of the underlying processes. We introduce new jet observables, Jet Rotational Metrics (JRMs), which provide insights into the substructure of jets by comparing them to jets with perfect discrete rotational symmetry. We show that JRMs are formidable jet features, achieving good classification scores when used as inputs to deep neural networks. We also show that when used in combination with other jet observables, like N-subjettiness and EFPs, our features increase classification performance. The results suggest that JRMs may capture information not efficiently captured by the other observables, motivating the design of future jet observables for learning the underlying symmetries in the physical processes.

Physics↗
DOE OSTI2021

Quantifying microbial control of soil organic matter dynamics at macrosystem scales

Soil organic matter (SOM) stocks, decomposition and persistence are largely the product of controls that act locally. Yet the controls are shaped and interact at multiple spatiotemporal scales, from which macrosystem patterns in SOM emerge. Theory on SOM turnover recognizes the resulting spatial and temporal conditionality in the effect sizes of controls that play out across macrosystems, and couples them through evolutionary and community assembly processes. For example, climate history shapes plant functional traits, which in turn interact with contemporary climate to influence SOM dynamics. Selection and assembly also shape the functional traits of soil decomposer communities, but it is less clear how in turn these traits influence temporal macrosystem patterns in SOM turnover. Here, we review evidence that establishes the expectation that selection and assembly should generate decomposer communities across macrosystems that have distinct functional effects on SOM dynamics. Representation of this knowledge in soil biogeochemical models affects the magnitude and direction of projected SOM responses under global change. Yet there is high uncertainty and low confidence in these projections. To address these issues, we make the case that a coordinated set of empirical practices are required which necessitate (1) greater use of statistical approaches in biogeochemistry that are suited to causative inference; (2) long-term, macrosystem-scale, observational and experimental networks to reveal conditionality in effect sizes, and embedded correlation, in controls on SOM turnover; and (3) use of multiple measurement grains to capture local- and macroscale variation in controls and outcomes, to avoid obscuring causative understanding through data aggregation. Here, when employed together, along with process-based models to synthesize knowledge and guide further empirical work, we believe these practices will rapidly advance understanding of microbial controls on SOM and improve carbon cycle projections that guide policies on climate adaptation and mitigation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Linking microstructure to creep behavior in vertically and horizontally built LPBF Haynes 282 compared with wrought material via θ -projection

Laser Powder Bed Fusion (LPBF) has emerged as a promising route for fabricating intricate geometries in high-performance alloys. Haynes 282 (H282) is a strong candidate for applications such as heat exchangers or engines due to its excellent creep strength and thermal stability; however, the long-term creep behavior of LPBF-processed H282 remains poorly understood. In this study, the θ -projection method is used to analyze and extrapolate the creep behavior of vertically built LPBF, horizontally built LPBF, compared to wrought H282 tested at 816 °C. Vertically built LPBF H282 exhibits the lowest minimum creep rate (MCR), while the horizontally built condition shows a higher MCR comparable to that of wrought H282. Despite these differences, both LPBF conditions exhibit significantly shorter rupture life and reduced rupture strain than the wrought material, with the most severe degradation observed in the horizontal builds, consistent with an earlier onset of tertiary creep and accelerated strain-rate evolution. Microstructural characterization reveals that both LPBF and wrought H282 exhibit abundant twin-related boundary character; however, their grain boundary topologies differ markedly. The wrought alloy contains a higher fraction of low-angle grain boundaries and continuous twin lamellae, whereas the LPBF microstructure is characterized by a suppressed low-angle boundary population and fragmented twin-related boundaries embedded within irregular high-angle grain boundary networks. Fractographic analysis further reveals predominantly intergranular cracking in LPBF H282, accompanied by grain-boundary-decorated carbides, Al 2 O 3 inclusions, and high-aspect-ratio pores. These results demonstrate that grain boundary topology, rather than minimum creep rate alone, plays a critical role in governing creep damage accumulation and rupture behavior in LPBF and wrought H282.

Creep↗
DOE OSTI2023

Hybrid interatomic potential for Sn

To design materials for extreme applications, it is important to understand and predict phase transitions and their influence on material properties under high pressures and temperatures. Atomistic modeling can be a useful tool to assess these behaviors. However, this can be difficult due to the lack of fidelity of the interatomic potentials in reproducing this high pressure and temperature extreme behavior. Here, in this work, a hybrid EAM-R—which is the combination of embedded atom method (EAM) and rapid artificial neural network potential—for Tin (Sn) is described which is capable of accurately modeling the complex sequence of phase transitions between different metallic polymorphs as a function of pressure. This hybrid approach ensures that a basic empirical potential like EAM is used as a lower energy bound. By using the final activation function, the neural network contribution to energy must be positive, assuring stability over the whole configuration space. This implementation has the capacity to reproduce density functional theory results at 6 orders of magnitude slower than a pair potential for molecular dynamics simulation, including elastic and plastic characteristics and relative energies of each phase. Using calculations of the Gibbs free energy, it is demonstrated that the potential precisely predicts the experimentally observed phase changes at temperatures and pressures across the whole phase diagram. At 10.2 GPa, the present potential predicts a first-order phase transition between body-centered tetragonal (BCT) β-Sn and another polymorph of BCT-Sn. This structure transforms into body-centered cubic near the experimentally reported value at 33 GPa. Thus, the Sn potential developed in this paper can be used to study complex deformation mechanisms under extreme conditions of high pressure and strain rates unlike existing potentials. Moreover, the framework developed in this paper can be extended for different material systems with complex phase diagrams.

36 MATERIALS SCIENCE↗
DOE OSTI2025

ScaWL: Scaling k-WL (Weisfeiler-Lehman) Algorithms in Memory and Performance on Shared and Distributed-Memory Systems

The k-dimensional Weisfeiler-Lehman (k-WL) algorithm—developed as an efficient heuristic for testing if two graphs are isomorphic—is a fundamental kernel for node embedding in the emerging field of graph neural networks. Unfortunately, the k-WL algorithm has exponential storage requirements, limiting the size of graphs that can be handled. This work presents a novel k-WL scheme with a storage requirement orders of magnitude lower while maintaining the same accuracy as the original k-WL algorithm. Due to the reduced storage requirement, our scheme allows for processing much bigger graphs than previously possible on a single compute node. For even bigger graphs, we provide the first distributed-memory implementation. Our k-WL scheme also has significantly reduced communication volume and offers high scalability. Our experimental results demonstrate that our approach is significantly faster and has superior scalability compared to five other implementations employing state-of-the-art techniques.

algorithims↗
DOE OSTI2025

Detecting Living-off-the-land Attacks Using K-means And Graph Convolutional Networks

The code ingests Zeek logs derived from network packet captures and goes through data preprocessing before it gets passed into a K-Means model that labels each device as either a client or server. Graph Convolutional Network (GCN) model is used to obtain the embeddings to represent the features in lower dimension. Last, K-means cluster analysis is used to cluster the embeddings for each class.

Quach, Anna [Idaho National Laboratory (INL), Idah↗
DOE OSTI2022

Internal erosion, particle transport, and channelization driven by fluid flow

The investigations summarized in this final technical report were accomplished under Department of Energy, Basic Energy Sciences research grant number DE-SC0010274 titled Internal erosion, particle transport, and channelization driven by fluid flow, $196,683 from 07/15/13 to 12/31/17, and $149,999 from 01/01/18 to 12/31/21. The work focused on developing physical models of fluid driven evolution of sediment beds as a result of dissolution and erosion of embedded particles that lead to channelization and fracture networks in porous rock. The grant work has resulted in seven peer reviewed publications, with another one currently under review, all of which are listed at the end of the report. These publications can be freely accessed through a technical library or directly from the publisher for a nominal fee. The main results obtained during the last grant cycle are described in brief in the following.

58 GEOSCIENCES↗
DOE OSTI2025

Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification

In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.

Bayesian inference↗
DOE OSTI2020

User Access to Scientific Facilities via 5G: A Cyber Security Thought Experiment

5G is more than an over-the-air radio technology upgrade. It is a strategy to extend Mobile Network Operator service offerings beyond traditional voice, instant messaging and Internet access. 5G Mobile Network Operators will offer new telecommunication services that include enhanced guarantees of confidentiality, integrity and availability. How could such services change the way Science collaborations connect scientists to supercomputers and other scientific facilities? Current scientific collaborations implicitly trust cloud service providers to securely store and process data. The perceived risks of outsourcing Science data security are counterbalanced by assurances that cloud providers operate at a scale that allows them to implement security measures impractical for Science collaborations (e.g. continuous system administrator behavioral monitoring and strict individual separation of duties). If that is true for a cloud service provider like Amazon Web Services (2018 revenue: $25.7 billion), could it also be true for Mobile Network Operators like Verizon Wireless (2018 revenue: $91.7 billion) or AT&T Mobility (2018 revenue: $71.3 billion)? DOE Leadership Class supercomputer facility users currently access them from the public Internet via Secure Shell. The sponsors and operators of the supercomputer facilities have determined that the public Internet path between the Scientist’s Device and the Login Node does not natively provide enough confidentiality or integrity to protect those communications. Therefore, the facilities achieve additional confidentiality and integrity by requiring Secure Shell encryption across those untrusted network paths. Using 5G Network Slice technology, a Mobile Network Operator may offer communication services between supercomputer users and facilities that natively provide confidentiality and integrity guarantees. Sponsors and operators of supercomputer facilities may determine that these guarantees provide enough confidentiality and integrity to protect those communications. If so, a 5G Network Slice could replace an SSH session running over the public Internet. Finally, this use case could be extended to other Office of Science user facility access requirements. Consider microscopy instruments at (e.g.) the Center for Nanoscale Materials or the Environmental Molecular Sciences Laboratory. The embedded systems controlling such instruments may not always support encrypted network access technologies like SSH. 5G Network Slices may offer an alternative to current VPN or SSH tunneling techniques, with additional benefits like guaranteed minimum bandwidth.

5G↗
DOE OSTI2024

Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes

In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.

Gallup, Ethan R.↗