Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Python toolkit”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Spatial Microsimulation and Activity Allocation in Python: An Update on the Likeness Toolkit

Understanding human security and social equity issues within human systems requires large-scale models of population dynamics that simulate high-fidelity representations of individuals and access to essential activities (work/school, social, errands, health). Likeness is a Python toolkit that provides these capabilities for Oak Ridge National Laboratory's (ORNL) UrbanPop spatial microsimulation project. In step with the initial development phase for Likeness (2021 - 2022), we built out several foundational examples of work/school and health service access. In this paper, we describe expansion and scaling of Likeness capabilities to metropolitan areas in the United States. We then provide an integrated demonstration of our methods based on a case study of Leon County, FL and perform validation exercises on 1) neighborhood demographic composition and 2) visits by demographic cohorts (gender/age) obtained from point of interest (POI) footfall data for essential services (grocery stores). Taking into account lessons learned from our case study, we scope improvements to our model as well as provide a roadmap of the anticipated Likeness development cycle into 2023 - 2024.

Tuccillo, Joe↗

Portable, heterogeneous ensemble workflows at scale using libEnsemble

libEnsemble is a Python-based toolkit for running dynamic ensembles, developed as part of the DOE Exascale Computing Project. The toolkit utilizes a unique generator–simulator–allocator paradigm, where generators produce input for simulators, simulators evaluate those inputs, and allocators decide whether and when a simulator or generator should be called. The generator steers the ensemble based on simulation results. Generators may, for example, apply methods for numerical optimization, machine learning, or statistical calibration. libEnsemble communicates between a manager and workers. Flexibility is provided through multiple manager–worker communication substrates each of which has different benefits. These include Python’s multiprocessing, mpi4py, and TCP. Multisite ensembles are supported using Balsam or Globus Compute. We overview the unique characteristics of libEnsemble as well as current and potential interoperability with other packages in the workflow ecosystem. We highlight libEnsemble’s dynamic resource features: libEnsemble can detect system resources, such as available nodes, cores, and GPUs, and assign these in a portable way. These features allow users to specify the number of processors and GPUs required for each simulation; and resources will be automatically assigned on a wide range of systems, including Frontier, Aurora, and Perlmutter. Such ensembles can include multiple simulation types, some using GPUs and others using only CPUs, sharing nodes for maximum efficiency. We also describe the benefits of libEnsemble’s generator–simulator coupling, which easily exposes to the user the ability to cancel, and portably kill, running simulations based on models that are updated with intermediate simulation output. We demonstrate libEnsemble’s capabilities, scalability, and scientific impact via a Gaussian process surrogate training problem for the longitudinal density profile at the exit of a plasma accelerator stage. In conclusion, the study uses gpCAM for the surrogate model and employs either Wake-T or WarpX simulations, highlighting efficient use of resources that can easily extend to exascale.

Dynamic ensembles↗

PyCBC Inference: A Python-based Parameter Estimation Toolkit for Compact Binary Coalescence Signals

We introduce new modules in the open-source PyCBC gravitational-wave astronomy toolkit that implement Bayesian inference for compact-object binary mergers. We review the Bayesian inference methods implemented and describe the structure of the modules. Here, we demonstrate that the PyCBC Inference modules produce unbiased estimates of the parameters of a simulated population of binary black hole mergers. We show that the parameters' posterior distributions obtained using our new code agree well with the published estimates for binary black holes in the first Advanced LIGO–Virgo observing run.

79 ASTRONOMY AND ASTROPHYSICS↗

Snakes on a Spaceship—An Overview of Python in Heliophysics

Computational analysis has become ubiquitous within the heliophysics community. However, community standards for peer review of codes and analysis have lagged behind these developments. This absence has contributed to the reproducibility crisis, where inadequate analysis descriptions and loss of scientific data have made scientific studies difficult or impossible to replicate. The heliophysics community has responded to this challenge by expressing a desire for a more open, collaborative set of analysis tools. Here, this article summarizes the current state of these efforts and presents an overview of many of the existing Python heliophysics tools. It also outlines the challenges facing community members who are working toward the goal of an open, collaborative, Python heliophysics toolkit and presents guidelines that can ease the transition from individualistic data analysis practices to an accountable, communalistic environment.

79 ASTRONOMY AND ASTROPHYSICS↗

Wind Integration National Dataset (WIND) Toolkit

Wind resource data for North America was produced using the Weather Research and Forecasting Model (WRF). The WRF model was initialized with the European Centre for Medium Range Weather Forecasts Interim Reanalysis (ERA-Interm) data set with an initial grid spacing of 54 km. Three internal nested domains were used to refine the spatial resolution to 18, 6, and finally 2 km. The WRF model was run for years 2007 to 2014. While outputs were extracted from WRF at 5 minute time-steps, due to storage limitations instantaneous hourly time-step are provided for all variables while full 5 min resolution data is provided for wind speed and wind direction only. The following variables were extracted from the WRF model data: - Wind Speed at 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Wind Direction at 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Temperature at 2, 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Pressure at 0, 100, 200 m - Surface Precipitation Rate - Surface Relative Humidity - Inverse Monin Obukhov Length

API↗

LLNL/thicket

Thicket is a python-based toolkit for Exploratory Data Analysis (EDA) of parallel performance data that enables performance optimization and understanding of applications’ performance on supercomputers. It bridges the performance tool gap between being able to consider only a single instance of a simulation run (e.g., single platform, single measurement tool, or single scale) and finding actionable insights in multi-dimensional, multi-scale, multi-architecture, and multi-tool performance datasets.

Brink, Stephanie Labasan↗

LLM Information Extraction Toolkit

A modular Python framework for information extraction using large language models with support for multiple backends and optional verification workflows.

Yoon, Hong-Jun [Oak Ridge National Laboratory (ORN↗

Automatic Parallelization of Numerical Python Applications using the Global Arrays Toolkit

Global Arrays is a software system from Pacific Northwest National Laboratory that enables an efficient, portable, and parallel shared-memory programming interface to manipulate distributed dense arrays. The NumPy module is the de facto standard for numerical calculation in the Python programming language, a language whose use is growing rapidly in the scientific and engineering communities. NumPy provides a powerful N-dimensional array class as well as other scientific computing capabilities. However, like the majority of the core Python modules, NumPy is inherently serial. Using a combination of Global Arrays and NumPy, we have reimplemented NumPy as a distributed drop-in replacement called Global Arrays in NumPy (GAiN). Serial NumPy applications can become parallel, scalable GAiN applications with only minor source code changes. Scalability studies of several different GAiN applications will be presented showing the utility of developing serial NumPy codes which can later run on more capable clusters or supercomputers.

99 GENERAL AND MISCELLANEOUS↗

Scanning ARM Precipitation Radar (SAPR) Velocity Azimuth Display (VAD)

The Velocity Azimuth Display (VAD, historical term) product contains retrievals of the zonal (north/south) and meridional (east/west) wind speed components, in precipitating systems, as a function of time and height. This product, which uses azimuthal variations in radial air velocity, allows quick insight into wind speed and wind direction in the vicinity of the radar when rainfall or snowfall is present. VAD data are derived from Corrected Moments in Antenna Coordinates Version 2 (CMAC2) using code within the Python ARM Radar Toolkit (Py-ART), an open-source architecture for interacting with radar data in the Python programming language.

54 ENVIRONMENTAL SCIENCES↗

Scanning ARM Precipitation Radar (SAPR) Quasi-Vertical Profile (QVP)

The Quasi-Vertical Profiles (QVP) VAP averages quality-controlled radar measurements along the axis of the azimuth. QVP allows quick insight into domain mean hydrometeor (e.g., snow, rain, melting layer) location and polarimetric data. QVP data are derived from Corrected Moments in Antenna Coordinates Version 2 (CMAC2) using code within the Python ARM Radar Toolkit (Py-ART), an open-source architecture for interacting with radar data in the Python programming language.

54 ENVIRONMENTAL SCIENCES↗

Mapped Moments to a Cartesian Grid (MMCG) Value-Added Product Report

Objective analysis (OA) is a method of mapping unstructured data to a structured grid. In the context of scanning radar data, OA is used to interpolate data in antenna coordinates (range, azimuth, and elevation) onto a regularly spaced Cartesian grid (Trapp and Doswell 2000). The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s Mapped Moments to a Cartesian Grid (MMCG) Value-Added Product (VAP) uses the Python ARM Radar Toolkit (Py-ART), a data model-driven interactive architecture for working with weather radar data, to map the data to a Cartesian grid (Helmus and Collis 2016). MMCG, with Py-ART built in, has the ability to take radar data in antenna coordinates and map the gates to a Cartesian grid using inverse distance weight functions such as Cressman (square) and Barnes (exponential), but also can filter the data during the interpolation. MMCG also allows arbitrary formulations for the radius of influence, which are matched to particular radar scanning strategies. This creates a complex parameter space for optimizing the retention of storm structure detail while minimizing artifacts. MMCG takes data processed with ARM’s Corrected Precipitation Radar Moments in Antenna Coordinates (CMAC) VAP and maps it to a Cartesian grid as the output product. A variety of fields that have been mapped to the Cartesian grid are then saved to plots to complement each grid file.

54 ENVIRONMENTAL SCIENCES↗

InfraPy: Python-Based Signal Analysis Tools for Infrasound

InfraPy is a Python-based analysis toolkit being development at LANL. The algorithms are intended for ground-based nuclear detonation detection applications to detect, locate, and characterize explosive sources using infrasonic observations. The implementation is usable as a stand-alone Python library or as a command line driven tool operating directly on a database. With multiple scientists working on the project, we've begun using a LANL git repository for collaborative development and version control. Current and planned work on InfraPy focuses on the development of new algorithms and propagation models. Collaboration with Southern Methodist University (SMU) has helped identify bugs and limitations of the algorithms. The current focus of usage development is focused on library imports and CLI.

97 MATHEMATICS AND COMPUTING↗

RadTraQ (Radar Tracking and Quality Toolkit

RadTraQ is a python library housing routines related to assessing and monitoring the quality of a radar's quality and calibration. This library is a work in progress, currently housing scripts to plot radar CFADs, plot and analyze corner reflector scans, calculate noise floors, create cloud masks, and plot up average profiles for comparisons.

THEISEN, ADAMK↗

Fat Crayon Toolkit

The Fat Crayon Toolkit (FCT) provides python modules for assisting in the design and visualization of boreholes and tunnels and their relationship with natural features such as fractures. Distances of closest approach among the boreholes can be calculated, along with other key design metrics. Additionally, the shear propensity of natural fractures (either deterministic or stochastic) can be visualized in either 3D (as VTK files) or in 2D (stereonets). The example provided demonstrates a test bed developed for the Enhanced Geothermal Systems (EGS) Collab project. This material is based upon work supported by the Geothermal Office of the Department of Energy and the Sanford Underground Research Facility and was prepared by LLNL under Contract DE-AC52-07NA27344.

Morris, JosephP.↗

The ARM Data-oriented Metrics and Diagnostics Package for Climate Models - A New Tool for Evaluating Climate Models with Field Data

The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) program User Facility produces ground-based long-term continuous unique measurements for atmospheric state, precipitation, turbulent fluxes, radiation, aerosol, cloud and the land surface, which are collected at multiple sites. These comprehensive datasets have been widely used to calibrate climate models and are proven to be invaluable for climate model development and improvement. This article introduces an evaluation package to facilitate the use of ground-based ARM measurements in climate model evaluation. The ARM data-oriented metrics and diagnostics package (ARM-DIAGS) includes both ARM observational datasets and a Python-based analysis toolkit for computation and visualization. The observational datasets are compiled from multiple ARM data products and specifically tailored for use in climate model evaluation. In addition, ARM-DIAGS also includes simulation data from models participating the Coupled Model Inter-comparison Project (CMIP), which will allow climate-modeling groups to compare a new, candidate version of their model to existing CMIP models. The analysis toolkit is designed to make the metrics and diagnostics quickly available to the model developers.

54 ENVIRONMENTAL SCIENCES↗

Determining the significance of associations between two series of discrete events: bootstrap methods

We review and develop techniques to determine associations between series of discrete events. The bootstrap, a nonparametric statistical method, allows the determination of the significance of associations with minimal assumptions about the underlying processes. We find the key requirement for this method: one of the series must be widely spaced in time to guarantee the theoretical applicability of the bootstrap. If this condition is met, the calculated significance passes a reasonableness test. We conclude with some potential future extensions and caveats on the applicability of these methods. The techniques presented have been implemented in a Python-based software toolkit.

97 MATHEMATICS AND COMPUTING↗

The Satellite Image Simulation Toolkit

The Satellite Image Simulation Toolkit (SatIST) is a python software package designed to generate diverse and realistic satellite imaging scenarios. It serves as a toolkit for simulating data that supports the development and testing of algorithms used in satellite detection, calibration, and characterization. SIST provides a suite of simulation tools that allow users to replicate various satellite observation conditions, including sidereal and target tracking. By enabling the creation of scenarios that mimic real-world satellite operations, SIST facilitates advancements in satellite image data processing and the study of satellite behavior under different observational parameters.

Perloff, AlexxS [Lawrence Livermore National Labor↗