Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “PYTHON”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Active Learning of Microgrid Frequency Dynamics Using Neural Ordinary Differential Equations

Accurate frequency modelling of inverter‐based resource (IBR)‐dominated power systems is crucial for ensuring stable, reliable and resilient operations, particularly given their inherent low‐inertia characteristics and fast dynamics that traditional swing equation‐based models inadequately capture. This paper explores neural ordinary differential equations (Neural ODEs) as a computationally efficient, data‐driven framework for modelling power system frequency dynamics, specifically within microgrids integrating high penetrations of distributed energy resources (DERs). The developed neural ODEs framework incorporates a neural network architecture designed to capture input dynamics. By actively perturbing the system with a known signal, the Python‐based neural ODEs framework was trained using measured system states and inputs, without the need for detailed system information. The framework, tested on a model of the Cordova, AK, microgrid, achieved a goodness of fit ranging from 60% to 99% across different state variables and maintained a mean square error in the 10 -6 p.u. range under square and step excitation signals. The proposed approach demonstrated robustness to measurement noise and initial condition variations while maintaining low computational complexity suitable for real‐time power system control applications. Furthermore, transfer learning enabled the neural ODEs model to adapt to the following changes in system topology or generator dispatch, highlighting its effectiveness for dynamic microgrids with frequently evolving configurations and diverse DERs.

Aryal, Tara [South Dakota State Univ., Brookings, ↗

MIRACL Co‐Simulation platform for control and operation of distributed wind in microgrid

Abstract This paper presents a co‐simulation platform (CSP) developed as a part of the “Microgrids, Infrastructure Resilience and Advanced Controls Launchpad (MIRACL)” project, hereafter called MIRACL‐CSP, to allow simulation‐based testing, demonstration, and evaluations of distributed wind under various grid operating conditions. MIRACL‐CSP provides modular interactions among the power distribution system, distributed wind, and utility decision‐making framework. A hierarchical engine for large‐scale infrastructure co‐simulations (HELICS) is used as the core engine of MIRACL‐CSP to establish time and information coordination among the MIRACL‐CSP modules. In this study, MIRACL‐CSP performance is demonstrated using the IEEE 123‐node test distribution grid modelled in GridLAB‐D and the utility decision support application modelled in Python. The functionality of MIRACL‐CSP is demonstrated through various grid operational scenarios in a microgrid and networked microgrid environment. A co‐simulation study is described that demonstrated MIRACL‐CSP capabilities for the microgrid operational scenario in the IEEE 123‐node test system. MIRACL‐CSP is a generic platform that facilitates distributed wind research for different test systems, applications, and valuation of distributed wind.

17 WIND ENERGY↗

Control coordination in inverter‐based microgrids using AoI‐based 5G schedulers

Abstract A coordinated set point automatic adjustment with correction enabled (C‐SPAACE) framework that uses 5G communication for real‐time control coordination between inverter‐based resources (IBR) in microgrids is proposed. Utilising slicing capability, 5G offers low‐latency communication to C‐SPAACE under normal conditions. However, given the multitude of power grid use cases, a certain 5G slice for C‐SPAACE may have access only to limited radio spectrum resources, which if not managed well, greatly undermines the communication needs of C‐SPAACE framework. Thus, optimally scheduling the available spectrum resources among IBRs in a sliced 5G network‐based C‐SPAACE framework becomes a critical problem. To address this issue, the authors utilise a novel age of information (AoI) metric and designs an AoI‐based 5G scheduler to provide low‐latency communication to C‐SPAACE. Following this, a co‐simulation environment is designed using PSCAD/EMTDC and Python to simulate a microgrid supported by 5G communication. Time‐domain simulation case studies are performed using the proposed co‐simulation environment to evaluate the performance of C‐SPAACE using 5G with both AoI‐based and other baseline (non‐AoI) schedulers.

Choudhury, Biplav↗

StaNdaRT: a repository of standardised test models and outputs for supernova radiative transfer

We present the first results of a comprehensive supernova (SN) radiative-transfer (RT) code-comparison initiative (StaNdaRT), where the emission from the same set of standardised test models is simulated by currently used RT codes. We ran a total of ten codes on a set of four benchmark ejecta models of Type Ia SNe. We consider two sub-Chandrasekhar-mass (M tot = 1.0 M ⊙ ) toy models with analytic density and composition profiles and two Chandrasekhar-mass delayed-detonation models that are outcomes of hydrodynamical simulations. We adopt spherical symmetry for all four models. The results of the different codes, including the light curves, spectra, and the evolution of several physical properties as a function of radius and time are provided in electronic form in a standard format via a public repository. We also include the detailed test model profiles and several Python scripts for accessing and presenting the input and output files. We also provide the code used to generate the toy models studied here. In this paper, we describe the test models, radiative-transfer codes, and output formats in detail, and provide access to the repository. We present example results of several key diagnostic features.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

FORSE+: Simulating non-Gaussian CMB foregrounds at 3 arcmin in a stochastic way based on a generative adversarial network

We present FORSE+, a Python package that produces non-Gaussian diffuse Galactic thermal dust emission maps at arcminute angular scales and that has the capacity to generate random realizations of small scales. This represents an extension of the FORSE (Foreground Scale Extender) package, which was recently proposed to simulate non-Gaussian small scales of thermal dust emission using generative adversarial networks (GANs). With the input of the large-scale polarization maps from observations, FORSE+ has been trained to produce realistic polarized small scales at 3′ following the statistical properties, mainly the non-Gaussianity, of observed intensity small scales, which are evaluated through Minkowski functionals. Furthermore, by adding different realizations of random components to the large-scale foregrounds, we show that FORSE+ is able to generate small scales in a stochastic way. In both cases, the output small scales have a similar level of non-Gaussianity compared with real observations and correct amplitude scaling as a power law. These realistic new maps will be useful, in the future, to understand the impact of non-Gaussian foregrounds on the measurements of the cosmic microwave background (CMB) signal, particularly on the lensing reconstruction, de-lensing, and the detection of cosmological gravitational waves in CMB polarizationB-modes.

Astronomy & Astrophysics↗

JANA2 Framework for Event Based and Triggerless Data Processing

Development of the second generation JANA2 multi-threaded event processing framework is ongoing through an LDRD initiative grant at Jefferson Lab. The framework is designed to take full advantage of all cores on modern many-core compute nodes. JANA2 efficiently handles both traditional hardware triggered event data and streaming data in online triggerless environments. Development is being done in conjunction with the Electron Ion Collider development. Anticipated to be the next large scale Nuclear Physics facility constructed. The core framework is written in modern C++ but includes an integrated Python interface. The status of development and summary of the more interesting features are presented.

Lawrence, David↗

ServiceX A Distributed, Caching, Columnar Data Delivery Service

We will describe a component of the Intelligent Data Delivery Service being developed in collaboration with IRIS-HEP and the LHC experiments. ServiceX is an experiment-agnostic service to enable on-demand data delivery specifically tailored for nearly-interactive vectorized analysis. This work is motivated by the data engineering challenges posed by HL-LHC data volumes and the increasing popularity of python and Spark-based analysis workflows.ServiceX gives analyzers the ability to query events by dataset metadata. It uses containerized transformations to extract just the data required for the analysis. This operation is colocated with the data to avoid transferring unnecessary branches over the WAN. Simple filtering operations are supported to further reduce the amount of data transferred.Transformed events are cached in a columnar datastore to accelerate delivery of subsequent similar requests. ServiceX will learn commonly related columns and automatically include them in the transformation to increase the potential for cache hits by other users.Selected events are streamed to the analysis system using an efficient wire protocol that can be readily consumed by a variety of computational frameworks. This reduces time-to-insight for physics analysis by delegating to ServiceX the complexity of event selection, slimming, reformatting, and streaming.

Galewsky, B.↗

Bringing heterogeneity to the CMS software framework

The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their computations on the co-processors as well as keeping the traditional CPU busy doing other work. The CMS data processing framework, CMSSW, implements multithreading using the Intel Threading Building Blocks (TBB) library, that utilizes tasks as concurrent units of work. In this paper we will discuss a generic mechanism to interact effectively with non-CPU resources that has been implemented in CMSSW. In addition, configuring such a heterogeneous system is challenging. In CMSSW an application is configured with a configuration file written in the Python language. The algorithm types are part of the configuration. The challenge therefore is to unify the CPU and co-processor settings while allowing their implementations to be separate. We will explain how we solved these challenges while minimizing the necessary changes to the CMSSW framework. We will also discuss on a concrete example how algorithms would offload work to NVIDIA GPUs using directly the CUDA API.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

SpackDev: Multi-Package Development with Spack

High Energy Physics (HEP) software environments often involve ~ hundreds of external packages and libraries, and similar numbers of experiment-specific, science-critical packages—many under continuous development. Managing coherent releases of the external software stack is challenging enough, but managing the highly-collaborative—and distributed—development of a large body of code against such a stack adds even more complexity and room for error. Spack is a popular Python-based package management tool with a specific focus on the needs of High Performance Computing (HPC) systems and system administrators whose strength is orchestrating the discrete download, build, testing, and installation of pre-packaged or tagged third-party software against similarly stable dependencies. As such it is becoming increasingly popular within HEP as that community makes increasing use of HPC facilities, and as efforts to develop future HPC systems utilize Spack to provide scientific software on those platforms. SpackDev is a system to facilitate the simultaneous development of interconnected sets of packages. Intended to handle packages without restriction to one internal build system, SpackDev is integrated with Spack as a command extension in order to leverage features such as dependency calculations and build system configuration, and is generally applicable outside HEP. We describe SpackDev’s features and development over the last two years, initial experience using SpackDev in the context of the LArSoft liquid argon detector toolkit, and work remaining before it can be considered a fully-functional multi-package build system for HEP experiments utilizing Spack.

Green, Chris↗

Striped Data Analysis Framework

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

Gutsche, Oliver↗

HOSS!

The Hall-D Online Skim System (HOSS) was developed to simultaneously solve two issues for the high intensity GlueX experiment. One was to parallelize the writing of raw data files to disk in order to improve bandwidth. The other was to distribute the raw data across multiple compute nodes in order to produce calibration skims of the data online. The highly configurable system employs RDMA, RAM disks, and zeroMQ driven by Python to simultaneously store and process the full high intensity GlueX data stream.

Lawrence, David↗

Geant4 Monte-Carlo (GEMC) A database-driven simulation program

GEMC[1] is an application that harnesses the power of databases to execute Geant4 Monte-Carlo simulations. The databases (MYSQL, CSQL, TEXT) define the geometry, materials, digitization algorithms, readout electronics and output formats. Implemented in C++, GEMC also boasts a user-friendly Python API that facilitates detector construction and database population. GEMC can handle real-life scenarios such as geometry variations and the run number-dependent calibration constants and digitization parameters. This abstract provides an overview of GEMC, accompanied by examples that showcase its versatility. We delve into the practical application of GEMC within the the CLAS12 experimental program at Jefferson Lab.

Ungaro, Maurizio↗

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗

Sensor Placement Optimization Software Applied to Site-Scale Methane-Emissions Monitoring

Advances in sensor technology have increased our ability to monitor a wide range of environments. However, even as the cost of sensors decline, only a limited number of sensors can be installed at any given site. The physical placement of sensors, along with the sensor technology and operating conditions, can have a large impact on our ability to adequately monitor environmental change. This paper introduces a new open-source Python package, called Chama, that determines optimal sensor placement and technology to improve a sensor network’s detection capabilities. Additionally, the methods are demonstrated using site-specific methane emission scenarios that capture uncertainty in wind conditions and emission characteristics. Mixed-integer linear programming formulations are used to determine sensor locations and detection thresholds that maximize detection of the emission scenarios. The optimized sensor networks consistently increase the ability to detect leaks, as compared to sensors placed near each potential emission source or along the perimeter of the site.

47 OTHER INSTRUMENTATION↗

Geospatial Capabilities to Couple Hazard and Social Vulnerability Data in Water Distribution Criticality Analysis

A resilience analysis of a water distribution system is greatly enhanced by the integration of up-to-date geospatial data describing the water system, hazards, and surrounding community. The Water Network Tool for Resilience (WNTR), an open-source Python package designed to simulate and analyze the resilience of water distribution systems, was recently updated to incorporate geographic information system (GIS) data into the resilience analysis. This paper describes the GIS capabilities and includes a case study using the drinking water distribution system model for a large city in Pennsylvania. The case study focuses on potential pipe damage from landslides and on pipes that are particularly difficult to repair. The analysis couples data on hazards, social vulnerability, and the location of emergency services to identify and prioritize high-impact critical infrastructure for mitigation. Results demonstrate that pipes can be prioritized for mitigation based on water shortage and vulnerable populations that are affected. In conclusion, the methods can be adopted for general use and are available as part of the WNTR software.

GIS, landslide↗

Recent developments in the PySCF program package

PySCF is a Python-based general-purpose electronic structure platform that supports first-principles simulations of molecules and solids as well as accelerates the development of new methodology and complex computational workflows. Here, we explain the design and philosophy behind PySCF that enables it to meet these twin objectives. With several case studies, we show how users can easily implement their own methods using PySCF as a development environment. We then summarize the capabilities of PySCF for molecular and solid-state simulations. Finally, we describe the growing ecosystem of projects that use PySCF across the domains of quantum chemistry, materials science, machine learning, and quantum information science.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Lowest-dimensional portals to SU( N ) exotics

New matter fields charged under the strong nuclear force would have dramatic phenomenological implications. In this work, we systematically explore how these new states, which we postulate belong to some representation of SU(3) of quantum chromo-dynamics, could interact with standard model fields: We analyze all lowest-dimensional “portal” operators for any SU(3) representation, and motivated by grand unification, we extend our results to SU( N ) for N > 3. We provide a publicly available python code, “tessellation,” that automatically constructs said lowest-dimensional portal operators for any new exotic matter field charged under SU( N ) for any N .

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Machine-learning accelerated geometry optimization in molecular simulation

Geometry optimization is an important part of both computational materials and surface science because it is the path to finding ground state atomic structures and reaction pathways. These properties are used in the estimation of thermodynamic and kinetic properties of molecular and crystal structures. This process is slow at the quantum level of theory because it involves an iterative calculation of forces using quantum chemical codes such as density functional theory (DFT), which are computationally expensive and which limit the speed of the optimization algorithms. It would be highly advantageous to accelerate this process because then one could do either the same amount of work in less time or more work in the same time. Here, we provide a neural network (NN) ensemble based active learning method to accelerate the local geometry optimization for multiple configurations simultaneously. We illustrate the acceleration on several case studies including bare metal surfaces, surfaces with adsorbates, and nudged elastic band for two reactions. In all cases, the accelerated method requires fewer DFT calculations than the standard method. In addition, we provide an Atomic Simulation Environment (ASE)-optimizer Python package to make the usage of the NN ensemble active learning for geometry optimization easier.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗