Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗
DOE OSTI2022

Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions

The increasing scale of neural networks and their growing application space have produced demand for more energy- and memory-efficient artificial-intelligence-specific hardware. Avenues to mitigate the main issue, the von Neumann bottleneck, include in-memory and near-memory architectures, as well as algorithmic approaches. In this report we leverage the low-power and the inherently binary operation of magnetic tunnel junctions (MTJs) to demonstrate neural network hardware inference based on passive arrays of MTJs. In general, transferring a trained network model to hardware for inference is confronted by degradation in performance due to device-to-device variations, write errors, parasitic resistance, and nonidealities in the substrate. To quantify the effect of these hardware realities, we benchmark 300 unique weight matrix solutions of a two-layer perceptron to classify the Wine dataset for both classification accuracy and write fidelity. Despite device imperfections, we achieve software-equivalent accuracy of up to 95.3% with proper tuning of network parameters in 15 x 15 MTJ arrays having a range of device sizes. The success of this tuning process shows that new metrics are needed to characterize the performance and quality of networks reproduced in mixed signal hardware.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

OpenDSS-wrapper (Distribution System Co-simulator with Distributed Energy Resource Controls)

Electric grid transformation with the proliferation of distributed energy resources (DER), such as solar photovoltaic (PV), wind, advanced energy storage technologies, and electric vehicles, and the growing use of communication technologies in both transmission and distribution systems are increasing the need to capture the interactions among these systems. Advanced modeling, control, and simulation tools that can perform co-simulation of electric power systems with other domains become indispensable to accurately model these interactions. To address this need, we have developed a codebase that integrates an electric power distribution system simulator, DER models, and DER controls. The codebase is based on OpenDSS, a distribution system simulator, and an existing open source co-simulation framework called HELICS. The contribution and uniqueness of the proposed codebase is that it tailors the generic HELICS framework specifically for distribution grid-related applications. The codebase includes an OpenDSS wrapper that controls the simulation, implements advanced DER controls, and extracts power flows, voltages, and other power system element information from the the distribution network modeled in OpenDSS. Sample HELICS federates, including a federate for OpenDSS, are provided that can communicate messages through the HELICS interface. Sample federates can be modified and additional HELICS federates can be added by the user depending on their use case requirements. SEE ALSO: https://github.com/NREL/dss-cosim

Blonsky, Michael↗
DOE OSTI2022

Classification of animal sounds in a hyperdiverse rainforest using convolutional neural networks with data augmentation

To protect tropical forest biodiversity, we need to be able to detect it reliably, cheaply, and at scale. Automated detection of sound producing animals from passively recorded soundscapes via machine-learning approaches is a promising technique towards this goal, but it is constrained by the necessity of large training data sets. Using soundscapes from a tropical forest in Borneo and a Convolutional Neural Network model (CNN), we investigate i) the minimum viable training data set size for accurate prediction of call types (‘sonotypes’), and ii) the extent to which data augmentation and transfer learning can overcome the issue of small and imbalanced training data sets. We found that even relatively high sample sizes (>80 per sonotype) lead to mediocre accuracy, which however improved significantly with data augmentation and transfer learning, including at extremely small sample sizes (3 per sonotype), regardless of taxonomic group or call characteristics. Neither transfer learning nor data augmentation alone achieved high accuracy. Our results suggest that transfer learning and data augmentation could make the use of CNNs to classify species’ vocalizations feasible even for small soundscape-based projects with many rare species. Retraining our open-source model requires only basic programming skills which makes it possible for individual conservation initiatives to match their local context, in order to enable more evidence-informed management of biodiversity.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Inference of cell type-specific gene regulatory networks on cell lineages from single cell omic datasets

Abstract Cell type-specific gene expression patterns are outputs of transcriptional gene regulatory networks (GRNs) that connect transcription factors and signaling proteins to target genes. Single-cell technologies such as single cell RNA-sequencing (scRNA-seq) and single cell Assay for Transposase-Accessible Chromatin using sequencing (scATAC-seq), can examine cell-type specific gene regulation at unprecedented detail. However, current approaches to infer cell type-specific GRNs are limited in their ability to integrate scRNA-seq and scATAC-seq measurements and to model network dynamics on a cell lineage. To address this challenge, we have developed single-cell Multi-Task Network Inference (scMTNI), a multi-task learning framework to infer the GRN for each cell type on a lineage from scRNA-seq and scATAC-seq data. Using simulated and real datasets, we show that scMTNI is a broadly applicable framework for linear and branching lineages that accurately infers GRN dynamics and identifies key regulators of fate transitions for diverse processes such as cellular reprogramming and differentiation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

A Model for Optimally Allocating Curbside Space Among Competing Uses

The emergence of various new forms of urban mobility services in recent years is leading to new pressures on curbside space. Municipalities, the entities typically responsible for managing the curbside, are in many instances handling these growing pressures by reallocating portions of the curbside away from traditional uses (such as metered and residential parking) in favor of uses such as ridehailing, scooter and bike-share corrals. As yet, however, such actions are being undertaken on an ad-hoc basis, due to the rapidly growing complexity of the curbside and the lack of standard analytical approaches. This lack of analytical capability is due to the traditional focus of transportation network modeling being focused predominantly on the interaction of supply and demand on links and nodes, with limited focus on link edges (the curbside). In this paper we address this research need by proposing a framework for modeling inter-modal competition for curbside space, inspired by the classical Bid-Rent Model of urban land use, intended to support curb managers to move towards maximizing the aspects of economic welfare that relate to curb access. In the bi-level model, choices made by the curbside manager impact travelers’ mode choices, and vice versa. We then present a simple numerical case study to demonstrate the properties of the proposed model, showing its tractability, flexibility, and intuitive sensitivity to systematic variation in inputs. The framework demonstrates the type of adaptive and evolving approach needed to maximize benefits from increasingly dynamic curb management strategies. The paper concludes with a brief discussion of future research needs to advance this line of inquiry.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

GAAF: Searching Activation Functions for Binary Neural Networks Through Genetic Algorithm

Binary neural networks (BNNs) show promising utilization in cost and power-restricted domains such as edge devices and mobile systems. This is due to its significantly less computation and storage demand, but at the cost of degraded performance. To close the accuracy gap, in this paper we propose to add a complementary activation function (AF) ahead of the sign based binarization, and rely on the genetic algorithm (GA) to automatically search for the ideal AFs. These AFs can help extract extra information from the input data in the forward pass, while allowing improved gradient approximation in the backward pass. Fifteen novel AFs are identified through our GA-based search, while most of them show improved performance (up to 2.54% on ImageNet) when testing on different datasets and network models. Interestingly, periodic functions are identified as a key component for most of the discovered AFs, which rarely exist in human designed AFs. Our method offers a novel approach for designing general and application-specific BNN architecture.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Toward Smart, Automated Junctional Tourniquets—AI Models to Interpret Vessel Occlusion at Physiological Pressure Points

Hemorrhage is the leading cause of preventable death in both civilian and military medicine. Junctional hemorrhages are especially difficult to manage since traditional tourniquet placement is often not possible. Ultrasound can be used to visualize and guide the caretaker to apply pressure at physiological pressure points to stop hemorrhage. However, this process is technically challenging, requiring the vessel to be properly positioned over rigid boney surfaces and applying sufficient pressure to maintain proper occlusion. As a first step toward automating this life-saving intervention, we demonstrate an artificial intelligence algorithm that classifies a vessel as patent or occluded, which can guide a user to apply the appropriate pressure required to stop flow. Neural network models were trained using images captured from a custom tissue-mimicking phantom and an ex vivo swine model of the inguinal region, as pressure was applied using an ultrasound probe with and without color Doppler overlays. Using these images, we developed an image classification algorithm suitable for the determination of patency or occlusion in an ultrasound image containing color Doppler overlay. Separate AI models for both test platforms were able to accurately detect occlusion status in test-image sets to more than 93% accuracy. In conclusion, this methodology can be utilized for guiding and monitoring proper vessel occlusion, which, when combined with automated actuation and other AI models, can allow for automated junctional tourniquet application.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Opportunities for wave energy in bulk power system operations

Wave energy resources have high, yet largely untapped potential as candidate generation technology. In this paper, we perform a data-driven analysis to characterize the impact of wave energy integration on bulk-scale power systems and market operations. Through data-driven sensitivity studies centered on an optimization-based production cost modeling formulation, our work characterizes the inflection point beyond which wave integration starts impacting power system operations, considering present day transmission infrastructure. Furthermore, our analysis also considers the joint effects of wave energy integration and system-wide transmission expansion. Finally, potential resilience scenarios such as wildfire-driven transmission contingencies and heat wave events are investigated, whereby the contributions of grid-integrated wave energy in alleviating the effects of the resilience events are analyzed. As our demonstration test bed, we consider a reduced-order network topology for the U.S. Western Interconnection with wave energy generation integrated at carefully selected sites across the coastal areas of Washington, Oregon, and northern California. Our results indicate that over a representative year of operations, wave energy integration systematically reduces locational marginal prices (LMPs) of energy and price volatility, especially during periods of high wave resource availability (winter months for the U.S. west coast). Average, maximum, and minimum of hourly LMPs over a typical year of operation was reduced by 2.95, 51.28, and 1.13 $\$$/MWh respectively (over a baseline scenario with no wave energy integration), when the selected network model had a total of 5000 MW wave power installed capacity during the representative year of study. The effects of wave energy integration can remain localized with existing transmission infrastructure (identified to be most pronounced in the Pacific Northwest region in the example we studied). However, with concurrent transmission expansion, the impacts of wave energy integration are likely to have a higher geographical spread. Our results also indicate that wave energy may be able to assist power system operations during resilience events such as major transmission contingencies and heat wave events, although such benefits might be dependent on factors such as proximity of affected area to wave resources, availability of adequate resource potential and adequate transmission capacity.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

Dynamic conformational switching underlies TFIIH function in transcription and DNA repair and impacts genetic diseases

Transcription factor IIH (TFIIH) is a protein assembly essential for transcription initiation and nucleotide excision repair (NER). Yet, understanding of the conformational switching underpinning these diverse TFIIH functions remains fragmentary. TFIIH mechanisms critically depend on two translocase subunits, XPB and XPD. To unravel their functions and regulation, we build cryo-EM based TFIIH models in transcription- and NER-competent states. Using simulations and graph-theoretical analysis methods, we reveal TFIIH’s global motions, define TFIIH partitioning into dynamic communities and show how TFIIH reshapes itself and self-regulates depending on functional context. Our study uncovers an internal regulatory mechanism that switches XPB and XPD activities making them mutually exclusive between NER and transcription initiation. By sequentially coordinating the XPB and XPD DNA-unwinding activities, the switch ensures precise DNA incision in NER. Mapping TFIIH disease mutations onto network models reveals clustering into distinct mechanistic classes, affecting translocase functions, protein interactions and interface dynamics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Methods in PES-Learn: Direct-Fit Machine Learning of Born–Oppenheimer Potential Energy Surfaces

The release of PES-L EARN version 1.0 as an open-source software package for the automatic construction of machine learning models of semi-global molecular potential energy surfaces (PESs) is presented. Improvements to PES-L EARN ’s interoperability are stressed with new Python API that simplifies workflows for PES construction via interaction with QCSchema input and output infrastructure. In addition, a new machine learning method is introduced to PES-L EARN : kernel ridge regression (KRR). The capabilities of KRR are emphasized with examination of select semi-global PESs. All machine learning methods available in PES-L EARN are benchmarked with benzene and ethanol datasets from the rMD17 database to illustrate PES-L EARN ’s performance ability. Fitting performance and timings are assessed for both systems. Finally, the ability to predict gradients with neural network models is presented and benchmarked with ethanol and benzene. PES-L EARN is an active project and welcomes community suggestions and contributions.

kernel ridge regression↗
DOE OSTI2024

Sub-millisecond keyhole pore detection in laser powder bed fusion using sound and light sensors and machine learning

Laser powder bed fusion is a mainstream additive manufacturing technology widely used to manufacture complex parts in prominent sectors, including aerospace, biomedical, and automotive industries. However, during the printing process, the presence of an unstable vapor depression can lead to a type of defect called keyhole porosity, which is detrimental to the part quality. In this study, we developed an effective approach to locally detect the generation of keyhole pores during the printing process by leveraging machine learning and a suite of optical and acoustic sensors. Simultaneous synchrotron x-ray imaging allows the direct visualization of pore generation events inside the sample, offering high-fidelity ground truth. A neural network model adopting SqueezeNet architecture using single-sensor data was developed to evaluate the fidelity of each sensor for capturing keyhole pore generation events. Our comparative study shows that the near infrared images gave the highest prediction accuracy, followed by 100 kHz and 20 kHz microphones, and the photodiode sensitive to processing laser wavelength had the lowest accuracy. Using a single sensor, over 90% prediction accuracy can be achieved with a temporal resolution as short as 0.1 ms. A data fusion scheme was also developed with features extracted using SqueezeNet neural network architecture and classification using different machine learning algorithms. Our work demonstrates the correlation between the characteristic optical and acoustic emissions and the keyhole oscillation behavior, and thereby provides strong physics support for the machine learning approach.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Utilizing Physics-Informed Synthetic Data to Train a Digital Twin for Predicting Reactor Operations

Understanding techniques to strengthen the nuclear nonproliferation regime is crucial in reducing the creation of nuclear weaponry on the basis of advancements in the nuclear energy industry. Prior to construction of a nuclear power plant, it is necessary to understand the proliferation potential of the plant’s reactor. Digital twins serve as a unique solution to recognizing reactor behavior indicative of nuclear proliferation. The following research conducted serves as a validation to training a digital twin on synthetic data fabricated via means of reactor physics simulations based on parameters of Idaho State University’s AGN-201 reactor. The synthetic data is utilized to train a long short-term memory (LSTM) recurrent neural network model. The accuracy of the predicted data is measured against real operational data to verify the reliability of the synthetic data creation methods and if these methods should be used in the future to inform inspectors of a reactor’s proliferation capabilities.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

Modeling of a Networked Community Microgrid with High Solar Penetration Considering Control Dynamics of Inverter-Based Resources

Microgrids help facilitate the integration of renewable energy in distribution-level grids and increase the resilience of the electric grid to extreme weather, especially in rural areas. Compared to traditional microgrids, a networked microgrid leverages multiple grid-forming sources to form a potential meshed grid and is more flexible in operation. This paper demonstrates the simulation modeling of an actual networked microgrid located in Adjuntas, Puerto Rico. The model contains representations of power inverters that connect the battery energy storage systems and photovoltaic generation systems to the networked microgrid and is capable of simulating fast grid transients as well as long-term operation of the networked microgrid. The modeling technique for power inverters allows the time-efficient simulation of the microgrid with a minimal penalty on model accuracy.

Li, Dingrui↗
DOE OSTI2021

AI to Predict Glass Compositions Satisfying Property and Cooling Rate Criteria

This project aimed to develop a predictive, artificial intelligence/machine learning-based model to identify glass compositions satisfying specified property requirements. Such a model would provide a systematic approach for narrowing down the nearly infinite range of possible compositions for glasses and minimize unnecessary experimental trial and error. A large empirical data set for training and testing the algorithm was obtained from the SciGlass database. It contains glass compositions and corresponding property data from a wide range of literature sources. However, the currently available form of this data, recently released under an open database license, is not conducive to easy querying and use. The data structure was deciphered and a customized parsing code developed to make this data more usable for the current and future work. Neural network models were developed and trained on viscosity data from the database and demonstrated potential for improving prediction accuracy over a traditional regression model.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

Accelerating the discovery of battery electrode materials through data mining and deep learning models

The availability of crystalline materials databases allows for building accurate machine learning (ML) models that can accelerate the exploration of materials chemical space for energy storage applications. In this work, we screen all inorganic materials included in the Materials Project and AFLOW databases as potential metal-ion battery electrodes. We develop an efficient protocol to mine and screen raw data in current databases and provide a new database of electrode materials by considering pairs of charged and discharged electrodes. This effort leads to a new database with over 190,000 instances, in contrast to the original battery database which contains about 5000. The expanded battery data set is then used to build regression-based deep neural network models for predicting average voltages and percentage volume changes upon charging and discharging, which present improvements of at least 28% for target properties with respect to previous models, and are now able to predict anode electrodes (low voltage region) as well as electrodes that will not work in electrochemical cells (negative voltages), overcoming the challenges identified in previous ML models for battery electrodes. Additionally, a further screening of the expanded database itself allowed us to identify 35 novel electrode candidates with excellent battery performance metrics.

25 ENERGY STORAGE↗
DOE OSTI2020

A Hybrid Framework Combining Model-Based and Data-Driven Methods for Hierarchical Decentralized Robust Dynamic State Estimation

This paper combines model-based and data-driven methods to develop a hierarchical, decentralized, robust dynamic state estimator (DSE). A two-level hierarchy is proposed, where the lower level consists of robust, model-based, decentralized DSEs. The state estimates sent from the lower level are received at the upper level, where they are filtered by a robust data-driven DSE after a principled sparse selection. This selection allows us to shrink the dimension of the problem at the upper level and hence significantly speed up the computational time. The proposed hybrid framework does not depend on the centralized infrastructure of the control centers; thus it can be completely embedded into the wide-area measurement systems. This feature will ultimately facilitate the placement of hierarchical decentralized control schemes at the phasor data concentrator locations. Also, the network model is not necessary; thus, a topology processor is not required. Finally, there is no assumption on the dynamics of the electric loads. The proposed framework is tested on the 2,000-bus synthetic Texas system, and shown to be capable of reconstructing the dynamic states of the generators with high accuracy, and of forecasting in the advent of missing data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Multipartite edge modes and tensor networks

Holographic tensor networks model AdS/CFT, but so far they have been limited by involving only systems that are very different from gravity. Unfortunately, we cannot straightforwardly discretize gravity to incorporate it, because that would break diffeomorphism invariance. In this note, we explore a resolution. In low dimensions gravity can be written as a topological gauge theory, which can be discretized without breaking gauge-invariance. However, new problems arise. Foremost, we now need a qualitatively new kind of “area operator,” which has no relation to the number of links along the cut and is instead topological. Secondly, the inclusion of matter becomes trickier. We successfully construct a tensor network both including matter and with this new type of area. Notably, while this area is still related to the entanglement in “edge mode” degrees of freedom, the edge modes are no longer bipartite entangled pairs. Instead they are highly multipartite. Along the way, we calculate the entropy of novel subalgebras in a particular topological gauge theory. We also show that the multipartite nature of the edge modes gives rise to non-commuting area operators, a property that other tensor networks do not exhibit.

Akers, Chris (ORCID:0000000227929827)↗