Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distillation network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A generalized distillation network synthesis model

In this report we propose a generalized superstructure-based distillation network synthesis model with improved modeling capabilities resulting in an extended solution space. The model can assign multiple mixtures to be separated (e.g., reactor network effluents) to different columns of the network while considering the interactions among different separation steps. In terms of outlets, products with general specifications, including pure components and multi-component mixtures, as well as streams without strict specifications (e.g., reactor recycle streams) can be readily handled. Furthermore, stream bypass is considered to avoid unnecessary separations, and thermal coupling is considered to yield novel solutions. The proposed model enables seamless integration with reactor network synthesis models thereby leading to solutions that are superior to the ones obtained by conventional sequential approaches.

42 ENGINEERING↗
DOE OSTI2021

Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery

We developed Distilled Graph Attention Policy Networks (DGAPNs), a curiosity-driven reinforcement learning model to generate novel graph-structured chemical representations that optimize user-defined objectives by efficiently navigating a physically constrained domain. Here, the framework is examined on the task of generating molecules that are designed to bind, noncovalently, to functional sites of SARS-CoV-2 proteins. We present a spatial Graph Attention Network (sGAT) that leverages self-attention over both node and edge attributes as well as encoding spatial structure -- this capability is of considerable interest in areas such as molecular and synthetic biology and drug discovery. An attentional policy network is then introduced to learn decision rules for a dynamic, fragment-based chemical environment, and state-of-the-art policy gradient techniques are employed to train the network with enhanced stability. Exploration is efficiently encouraged by incorporating innovation reward bonuses learned and proposed by random network distillation. In experiments, our framework achieved outstanding results compared to state-of-the-art algorithms, while increasing the diversity of proposed molecules and reducing the complexity of paths to chemical synthesis.

Wu, Yulun↗
DOE OSTI2022

Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery

We developed Distilled Graph Attention Policy Network (DGAPN), a reinforcement learning model to generate novel graph-structured chemical representations that optimize user-defined objectives by efficiently navigating a physically constrained domain. The framework is examined on the task of generating molecules that are designed to bind, noncovalently, to functional sites of SARS-CoV-2 proteins. We present a spatial Graph Attention (sGAT) mechanism that leverages self-attention over both node and edge attributes as well as encoding the spatial structure --- this capability is of considerable interest in synthetic biology and drug discovery. An attentional policy network is introduced to learn the decision rules for a dynamic, fragment-based chemical environment, and state-of-the-art policy gradient techniques are employed to train the network with stability. Exploration is driven by the stochasticity of the action space design and the innovation reward bonuses learned and proposed by random network distillation. In experiments, our framework achieved outstanding results compared to state-of-the-art algorithms, while reducing the complexity of paths to chemical synthesis.

Wu, Yulun↗
DOE OSTI2022

Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery

We developed Distilled Graph Attention Policy Networks (DGAPNs), a curiosity-driven reinforcement learning model to generate novel graph-structured chemical representations that optimize user-defined objectives by efficiently navigating a physically constrained domain. The framework is examined on the task of generating molecules that are designed to bind, noncovalently, to functional sites of SARS-CoV-2 proteins. We present a spatial Graph Attention Network (sGAT) that leverages self-attention over both node and edge attributes as well as encoding spatial structure -- this capability is of considerable interest in areas such as molecular and synthetic biology and drug discovery. An attentional policy network is then introduced to learn decision rules for a dynamic, fragment-based chemical environment, and state-of-the-art policy gradient techniques are employed to train the network with enhanced stability. Exploration is efficiently encouraged by incorporating innovation reward bonuses learned and proposed by random network distillation. In experiments, our framework achieved outstanding results compared to state-of-the-art algorithms, while increasing the diversity of proposed molecules and reducing the complexity of paths to chemical synthesis.

Wu, Yulun↗
DOE OSTI2022

SPATIAL GRAPH ATTENTION AND CURIOSITY-DRIVEN POLICY FOR ANTIVIRAL DRUG DISCOVERY

We developed Distilled Graph Attention Policy Network (DGAPN), a reinforcement learning model to generate novel graph-structured chemical representations that optimize user-defined objectives by efficiently navigating a physically constrained domain. The framework is examined on the task of generating molecules that are designed to bind, noncovalently, to functional sites of SARS-CoV-2 proteins. We present a spatial Graph Attention (sGAT) mechanism that leverages self-attention over both node and edge attributes as well as encoding the spatial structure - this capability is of considerable interest in synthetic biology and drug discovery. An attentional policy network is introduced to learn the decision rules for a dynamic, fragment-based chemical environment, and state-of-the-art policy gradient techniques are employed to train the network with stability. Exploration is driven by the stochasticity of the action space design and the innovation reward bonuses learned and proposed by random network distillation. In experiments, our framework achieved outstanding results compared to state-of-the-art algorithms, while reducing the complexity of paths to chemical synthesis.

Wu, Y↗
DOE OSTI2024

Distilling the Essential Elements of Nuclear Binding via Neural-Network Quantum States

To distill the essential elements of nuclear binding, we seek the simplest Hamiltonian capable of modeling atomic nuclei with percent-level accuracy. A critical aspect of this endeavor consists of accurately solving the quantum many-body problem without incurring an exponential computing cost with the number of nucleons. Furthermore, we address this challenge by leveraging a variational Monte Carlo method based on a highly expressive neural-network quantum state ansatz. In addition to computing binding energies and charge radii of nuclei with up to 𝐴 = 20 nucleons, by evaluating their magnetic moments, we demonstrate that neural-network quantum states are able to correctly capture the self-emerging nuclear shell structure. To this end, we introduce a novel computational protocol based on adding an external magnetic field to the nuclear Hamiltonian, which allows the neural network to learn the preferred polarization of the nucleus within the given magnetic field.

Binding energy & masses↗
DOE OSTI2023

Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

Abstract Motivation Millions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. Results We developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementation The source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Generation and characterization of ultrabroadband polarization–frequency hyperentangled photons

We generate ultrabroadband photon pairs entangled in both polarization and frequency bins through an all-waveguided Sagnac source covering the entire optical C- and L-bands (1530–1625 nm). We perform comprehensive characterization of high-fidelity states in multiple dense wavelength-division multiplexed channels, achieving full tomography of effective four-qubit systems. Additionally, leveraging the inherent high dimensionality of frequency encoding and our electro-optic measurement approach, we demonstrate the scalability of our system to higher dimensions, reconstructing states in a 36-dimensional Hilbert space consisting of two polarization qubits and two frequency-bin qutrits. Our findings hold potential significance for quantum networking, particularly dense coding and entanglement distillation in wavelength-multiplexed quantum networks.

42 ENGINEERING↗
DOE OSTI2024

Distilling particle knowledge for fast reconstruction at high-energy physics experiments

Knowledge distillation is a form of model compression that allows artificial neural networks of different sizes to learn from one another. Its main application is the compactification of large deep neural networks to free up computational resources, in particular on edge devices. In this article, we consider proton-proton collisions at the High-Luminosity Large Hadron Collider (HL-LHC) and demonstrate a successful knowledge transfer from an event-level graph neural network (GNN) to a particle-level small deep neural network (DNN). Our algorithm, DistillNet, is a DNN that is trained to learn about the provenance of particles, as provided by the soft labels that are the GNN outputs, to predict whether or not a particle originates from the primary interaction vertex. The results indicate that for this problem, which is one of the main challenges at the HL-LHC, there is minimal loss during the transfer of knowledge to the small student network, while improving significantly the computational resource needs compared to the teacher. This is demonstrated for the distilled student network on a CPU, as well as for a quantized and pruned student network deployed on a field programmable gate array. Our study proves that knowledge transfer between networks of different complexity can be used for fast artificial intelligence (AI) in high-energy physics that improves the expressiveness of observables over non-AI-based reconstruction algorithms. Such an approach can become essential at the HL-LHC experiments, e.g. to comply with the resource budget of their trigger stages.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

A survey of techniques for optimizing transformer inference

Recent years have seen a phenomenal rise in the performance and applications of transformer neural networks. The family of transformer networks, including Bidirectional Encoder Representations from Transformer (BERT), Generative Pretrained Transformer (GPT) and Vision Transformer (ViT), have shown their effectiveness across Natural Language Processing (NLP) and Computer Vision (CV) domains. Transformer-based networks such as ChatGPT have impacted the lives of common men. However, the quest for high predictive performance has led to an exponential increase in transformers' memory and compute footprint. Researchers have proposed techniques to optimize transformer inference at all levels of abstraction. Further, this paper presents a comprehensive survey of techniques for optimizing the inference phase of transformer networks. We survey techniques such as knowledge distillation, pruning, quantization, neural architecture search and lightweight network design at the algorithmic level. We further review hardware-level optimization techniques and the design of novel hardware accelerators for transformers. We summarize the quantitative results on the number of parameters/FLOPs and the accuracy of several models/techniques to showcase the tradeoff exercised by them. We also outline future directions in this rapidly evolving field of research. We believe that this survey will educate both novice and seasoned researchers and also spark a plethora of research efforts in this field.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Optimization-Based Azeotropic Distillation System Synthesis Using Geometric Insights

The synthesis of azeotropic distillation systems is challenging due to the existence of compartments in the residue curve map coupled with the combinatorial aspects from numerous possible system configurations. In this work, an optimization-based approach is introduced to synthesize homogeneous azeotropic distillation systems. The approach employs a network-based representation generated via a matrix method. To design the distillation columns, the modified Underwood equations are adopted, in which pseudocomponent-based compositions are used. For cases where separatrices are significantly curved, corrections via piece-wise linear functions and collinearity properties are implemented. Two examples are presented to illustrate the proposed approach.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

A transfer learning approach to energy-efficient control of small and medium-sized commercial buildings

Model-free reinforcement learning (RL) provides a data-driven and adaptive approach to optimize building energy use while satisfying occupant comfort. This powerful tool does not need any prior knowledge about the environment and system it is optimizing and can adapt its policy based on the changes in captures. Like any other data-driven tool, it faces high training costs due to the extensive agent-environment interactions required to capture long-term building dynamics and user comfort. Transfer learning, particularly policy distillation, offers a promising way to accelerate training by leveraging pretrained RL agents in different building and system types. Here, this study investigates online student distillation, in which the student model updates its neural network weights using outputs from teacher models. The work introduces a student distillation strategy designed for efficient knowledge transfer, along with a teacher selection method that ensures high-quality guidance. The approach is validated using a highly calibrated whole building energy model for a small/medium commercial building test facility. Results show substantial reductions in training time and data requirements while surpassing the performance of ASHRAE Guideline 36, an advanced rule-based control strategy. The distilled RL model required 45% less data and achieved 20% higher cumulative rewards than a state-of-the-art RL model, with faster convergence and lower energy consumption. These outcomes demonstrate that effective transfer learning enables a scalable and data-efficient energy management solution for commercial buildings.

ASHRAE guideline 36↗
DOE OSTI2026

Constant Overhead Entanglement Distillation via Scrambling

High-fidelity quantum entanglement enables key quantum networking capabilities such as secure communication and distributed quantum computing, but long-distance entanglement distribution is limited by noise and loss. Entanglement distillation protocols address this problem by extracting high-fidelity Bell pairs from multiple noisy ones. The primary objective is minimizing the resource overhead: the number of noisy input pairs needed to distill each high-fidelity output pair. While protocols achieving optimal overhead are known in theory, they often require complex decoding operations that make practical implementation challenging. We circumvent this challenge by introducing protocols that use quantum scrambling—the spreading of quantum information under chaotic dynamics—through random Clifford operations. Based on this scrambling mechanism, our protocol maintains asymptotically constant overhead, independent of the desired output error rate $\bar{𝜖}$ , and can be implemented with shallow quantum circuits of depth 𝑂⁡(poly log log⁡ $\bar{𝜖}$ −1 ) and memory 𝑂⁡(poly log⁡ $\bar{𝜖}$ −1 ). Our protocol remains effective even with noisy quantum gates. By incorporating error correction, our protocol achieves state-of-the-art performance: starting with pairs of 10% initial infidelity, we require only seven noisy inputs per output pair to distill a single Bell pair with infidelity $\bar{𝜖}$ =10 −12 , substantially outperforming existing schemes. We demonstrate the utility of our protocols for quantum repeater networks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Distilling physical origins of hardness in multi-principal element alloys directly from ensemble neural network models

Abstract Despite a plethora of data being generated on the mechanical behavior of multi-principal element alloys, a systematic assessment remains inaccessible via Edisonian approaches. We approach this challenge by considering the specific case of alloy hardness, and present a machine-learning framework that captures the essential physical features contributing to hardness and allows high-throughput exploration of multi-dimensional compositional space. The model, tested on diverse datasets, was used to explore and successfully predict hardness in Al x Ti y (CrFeNi) 1- x - y , Hf x Co y (CrFeNi) 1- x - y and Al x (TiZrHf) 1- x systems supported by data from density-functional theory predicted phase stability and ordering behavior. The experimental validation of hardness was done on TiZrHfAl x . The selected systems pose diverse challenges due to the presence of ordering and clustering pairs, as well as vacancy-stabilized novel structures. We also present a detailed model analysis that integrates local partial-dependencies with a compositional-stimulus and model-response study to derive material-specific insights from the decision-making process.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Field Insights: Strengthening Digital Assurance Through On-Site Network Monitoring

The accelerating deployment of digital energy infrastructure, ranging from inverter-based resources (IBRs), battery energy storage systems (BESS), to advanced grid control platforms, has brought unprecedented visibility, flexibility, and efficiency to the electric grid. However, this digital transformation also introduces new cybersecurity challenges, particularly in the form of supply chain risks and operational blind spots at the grid edge. Over the past year, the Department of Energy’s Office of Cybersecurity, Energy Security, and Emergency Response (CESER), through its Rapid Risk Assessment initiative, along with the Grid Deployment Office (GDO), through its Technical Assistance for Digital Assurance (TADA) initiative, have supported a series of on-site network engagements led by Idaho National Laboratory (INL). These engagements, conducted in partnership with asset owners across the country, have focused on identifying real-world vulnerabilities and misconfigurations in operational environments, many of which are not detectable through remote assessments or traditional compliance audits. The goal of this report is to distill key findings and lessons learned during network hunt engagements from INL’s fiscal year (FY) 2024 - 2025. It is intended to help asset owners—regardless of their participation in the program—better understand the evolving threat landscape and adopt practical measures to secure their digital energy infrastructure.

24 - POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A single-photon polarization-frequency controlled-NOT gate

We develop a high-fidelity controlled-NOT gate between photonic polarization and frequency, leveraging direction-dependent electro-optic phase modulation in a fiber Sagnac loop. This design could facilitate massively parallelizable entanglement distillation with polarization-frequency resources in lightwave networks.

Lu, Hsuan-Hao [ORNL] (ORCID:0000000340090787)↗
DOE OSTI2024

Fracture Characterization Via AI‐Assisted Analysis of Temperature Logs

Abstract Fractures control fluid flow, mass transport, and heat transfer in a geothermal reservoir. This makes accurate characterization of fracture networks a prerequisite for optimal design and control of a reservoir's exploitation. We develop a deep‐learning procedure to identify fracture locations via interpretation of temporally and spatially continuous downhole temperature measurements. A long short‐term memory fully convolutional network (LSTM‐FCN) is used both to capture long‐term dependencies in sequential temperature data and to distill local features around fractures. A wellbore and fractured‐reservoir thermal model is established to generate temperature data for network training. The trained LSTM‐FCN exhibits a unique ability to detect multiple fractures intersecting a borehole. We use the LSTM‐FCN algorithm to evaluate the effectiveness of different‐stage wellbore temperature measurements on fracture detection in a complex fractured system. Our experiments reveal that the use of various‐stage temperature information as an input feature set improves the robustness of fracture detection to noise interference. This study indicates the practical feasibility of obtaining accurate fracture‐network reconstructions from temperature signals, at reasonable computational cost.

Yang, Xiaoyu↗