Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Python toolkit”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Flare Statistics for Young Stars from a Convolutional Neural Network Analysis of TESS Data

All-sky photometric time-series missions have allowed for the monitoring of thousands of young (t(age) < 800 Myr) stars in order to understand the evolution of stellar activity. Here, we developed a convolutional neural network (CNN), stella, specifically trained to find flares in Transiting Exoplanet Survey Satellite (TESS) short-cadence data. We applied the network to 3200 young stars in order to evaluate flare rates as a function of age and spectral type. The CNN takes a few seconds to identify flares on a single light curve. We also measured rotation periods for 1500 of our targets and find that flares of all amplitudes are present across all spot phases, suggesting high spot coverage across the entire surface. Additionally, flare rates and amplitudes decrease for stars t(age) > 50 Myr across all temperatures T(eff) ≥ 4000 K, while stars from 2300 ≤ T(eff) < 4000 K show no evolution across 800 Myr. Stars of T(eff) ≤ 4000 K also show higher flare rates and amplitudes across all ages. We investigate the effects of high flare rates on photoevaporative atmospheric mass loss for young planets. In the presence of flares, planets lose 4%–7% more atmosphere over the first 1 Gyr. stella is an open-source Python toolkit hosted on GitHub and PyPI.

Adina D. Feinstein↗

Python ARM Radar Toolkit (Py-ART) Roadmap

Radar software is key for producing scientific insight from radar data. There are several platforms for interacting with radar data; the open-source variants are well documented in Heistermann et al. 2014. The Python ARM Radar Toolkit (Py-ART; Helmus and Collis, 2016) is one of these. This document is a guiding roadmap for development for Py-ART. It sets priorities for U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Climate Research Facility-funded work and it advises the community which contributions are desired and will receive the most assistance from ARM-funded software engineers.

Python ARM Radar Toolkit, linear programming, valu↗

Interfacing the X-Ray Toolkit and the PYTHON Radiography Analysis Tool Final Report for FY12

This document is the final report for the X-Ray Toolkit (XTK) and the PYTHON Radiography Analysis Tool (PyRAT) merger for project LANL12-RS-107J in FY12. This project involves merging the radiography analysis capabilities in PyRAT with the image processing capabilities of XTK. The merger is the focus of project LANL12-RS 107J at Los Alamos National Laboratory (LANL) and SNL12-RS-107J at Sandia National Laboratory.

Radiology & Nuclear Medicine(62)↗

PYSAT: Python Satellite Data Analysis Toolkit

A common problem in space science data analysis is combining complementary data sources that are provided and analyzed in different formats and programming languages. The Python Satellite Data Analysis Toolkit (pysat) addresses this issue by providing an open source toolkit that implements the general process of space science data analysis, from beginning to end, in an instrumentindependent manner. This toolkit uses an Instrument object that enables systematic analysis of science data from a variety of platforms within a single interface. Basic functions such as downloading, loading, and cleaning are included for all supported instruments. Common analysis routines are also included, which are instrument and data source independent. A nanokernel is used to provide instrument independence, it is attached to the Instrument object and mediates the systematic and arbitrary modification of loaded data. Pysat uses the nanokernel to improve the rigor of time series analysis, support onthefly orbit determination, and cleanly span file breaks. Pysat's functions and higherlevel scientific analysis features are validated through the use of unit testing. Further adoption by the community provides a set of scientific results produced by a common core, constituting a distributed heritage that supports the validity of the underlying processing and scientific output. These features are used to demonstrate consistency between derived electron density profiles and measured ion drifts, particularly downward ion drifts in the afternoon hours during extreme solar minimum. Pysat builds upon open source Python software that is freely available and encourages communitydriven development.

Stoneback, R.A.↗

Spatial Microsimulation and Activity Allocation in Python: An Update on the Likeness Toolkit

Understanding human security and social equity issues within human systems requires large-scale models of population dynamics that simulate high-fidelity representations of individuals and access to essential activities (work/school, social, errands, health). Likeness is a Python toolkit that provides these capabilities for Oak Ridge National Laboratory's (ORNL) UrbanPop spatial microsimulation project. In step with the initial development phase for Likeness (2021 - 2022), we built out several foundational examples of work/school and health service access. In this paper, we describe expansion and scaling of Likeness capabilities to metropolitan areas in the United States. We then provide an integrated demonstration of our methods based on a case study of Leon County, FL and perform validation exercises on 1) neighborhood demographic composition and 2) visits by demographic cohorts (gender/age) obtained from point of interest (POI) footfall data for essential services (grocery stores). Taking into account lessons learned from our case study, we scope improvements to our model as well as provide a roadmap of the anticipated Likeness development cycle into 2023 - 2024.

Tuccillo, Joe↗

Portable, heterogeneous ensemble workflows at scale using libEnsemble

libEnsemble is a Python-based toolkit for running dynamic ensembles, developed as part of the DOE Exascale Computing Project. The toolkit utilizes a unique generator–simulator–allocator paradigm, where generators produce input for simulators, simulators evaluate those inputs, and allocators decide whether and when a simulator or generator should be called. The generator steers the ensemble based on simulation results. Generators may, for example, apply methods for numerical optimization, machine learning, or statistical calibration. libEnsemble communicates between a manager and workers. Flexibility is provided through multiple manager–worker communication substrates each of which has different benefits. These include Python’s multiprocessing, mpi4py, and TCP. Multisite ensembles are supported using Balsam or Globus Compute. We overview the unique characteristics of libEnsemble as well as current and potential interoperability with other packages in the workflow ecosystem. We highlight libEnsemble’s dynamic resource features: libEnsemble can detect system resources, such as available nodes, cores, and GPUs, and assign these in a portable way. These features allow users to specify the number of processors and GPUs required for each simulation; and resources will be automatically assigned on a wide range of systems, including Frontier, Aurora, and Perlmutter. Such ensembles can include multiple simulation types, some using GPUs and others using only CPUs, sharing nodes for maximum efficiency. We also describe the benefits of libEnsemble’s generator–simulator coupling, which easily exposes to the user the ability to cancel, and portably kill, running simulations based on models that are updated with intermediate simulation output. We demonstrate libEnsemble’s capabilities, scalability, and scientific impact via a Gaussian process surrogate training problem for the longitudinal density profile at the exit of a plasma accelerator stage. In conclusion, the study uses gpCAM for the surrogate model and employs either Wake-T or WarpX simulations, highlighting efficient use of resources that can easily extend to exascale.

Dynamic ensembles↗

PyCBC Inference: A Python-based Parameter Estimation Toolkit for Compact Binary Coalescence Signals

We introduce new modules in the open-source PyCBC gravitational-wave astronomy toolkit that implement Bayesian inference for compact-object binary mergers. We review the Bayesian inference methods implemented and describe the structure of the modules. Here, we demonstrate that the PyCBC Inference modules produce unbiased estimates of the parameters of a simulated population of binary black hole mergers. We show that the parameters' posterior distributions obtained using our new code agree well with the published estimates for binary black holes in the first Advanced LIGO–Virgo observing run.

79 ASTRONOMY AND ASTROPHYSICS↗

Snakes on a Spaceship—An Overview of Python in Heliophysics

Computational analysis has become ubiquitous within the heliophysics community. However, community standards for peer review of codes and analysis have lagged behind these developments. This absence has contributed to the reproducibility crisis, where inadequate analysis descriptions and loss of scientific data have made scientific studies difficult or impossible to replicate. The heliophysics community has responded to this challenge by expressing a desire for a more open, collaborative set of analysis tools. Here, this article summarizes the current state of these efforts and presents an overview of many of the existing Python heliophysics tools. It also outlines the challenges facing community members who are working toward the goal of an open, collaborative, Python heliophysics toolkit and presents guidelines that can ease the transition from individualistic data analysis practices to an accountable, communalistic environment.

79 ASTRONOMY AND ASTROPHYSICS↗

Wind Integration National Dataset (WIND) Toolkit

Wind resource data for North America was produced using the Weather Research and Forecasting Model (WRF). The WRF model was initialized with the European Centre for Medium Range Weather Forecasts Interim Reanalysis (ERA-Interm) data set with an initial grid spacing of 54 km. Three internal nested domains were used to refine the spatial resolution to 18, 6, and finally 2 km. The WRF model was run for years 2007 to 2014. While outputs were extracted from WRF at 5 minute time-steps, due to storage limitations instantaneous hourly time-step are provided for all variables while full 5 min resolution data is provided for wind speed and wind direction only. The following variables were extracted from the WRF model data: - Wind Speed at 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Wind Direction at 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Temperature at 2, 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Pressure at 0, 100, 200 m - Surface Precipitation Rate - Surface Relative Humidity - Inverse Monin Obukhov Length

API↗

LLNL/thicket

Thicket is a python-based toolkit for Exploratory Data Analysis (EDA) of parallel performance data that enables performance optimization and understanding of applications’ performance on supercomputers. It bridges the performance tool gap between being able to consider only a single instance of a simulation run (e.g., single platform, single measurement tool, or single scale) and finding actionable insights in multi-dimensional, multi-scale, multi-architecture, and multi-tool performance datasets.

Brink, Stephanie Labasan↗

Snakes on a Spaceship - An Overview of Python in Heliophysics

Computational analysis has become ubiquitous within the heliophysics community. However, community standards for peer review of codes and analysis have lagged behind these developments. This absence has contributed to the reproducibility crisis, where inadequate analysis descriptions and loss of scientific data have made scientific studies difficult or impossible to replicate. The heliophysics community has responded to this challenge by expressing a desire for a more open, collaborative set of analysis tools. This article summarizes the current state of these efforts and presents an overview of many of the existing Python heliophysics tools. It also outlines the challenges facing community members who are working toward the goal of an open, collaborative, Python heliophysics toolkit and presents guidelines that can ease the transition from individualistic data analysis practices to an accountable, communalistic environment.

Burrell, A.G.↗

LLM Information Extraction Toolkit

A modular Python framework for information extraction using large language models with support for multiple backends and optional verification workflows.

Yoon, Hong-Jun [Oak Ridge National Laboratory (ORN↗

Automatic Parallelization of Numerical Python Applications using the Global Arrays Toolkit

Global Arrays is a software system from Pacific Northwest National Laboratory that enables an efficient, portable, and parallel shared-memory programming interface to manipulate distributed dense arrays. The NumPy module is the de facto standard for numerical calculation in the Python programming language, a language whose use is growing rapidly in the scientific and engineering communities. NumPy provides a powerful N-dimensional array class as well as other scientific computing capabilities. However, like the majority of the core Python modules, NumPy is inherently serial. Using a combination of Global Arrays and NumPy, we have reimplemented NumPy as a distributed drop-in replacement called Global Arrays in NumPy (GAiN). Serial NumPy applications can become parallel, scalable GAiN applications with only minor source code changes. Scalability studies of several different GAiN applications will be presented showing the utility of developing serial NumPy codes which can later run on more capable clusters or supercomputers.

99 GENERAL AND MISCELLANEOUS↗

Python-Based Scientific Analysis and Visualization of Precipitation Systems at NASA Marshall Space Flight Center

At NASA Marshall Space Flight Center (MSFC), Python is used several different ways to analyze and visualize precipitating weather systems. A number of different Python‐based software packages have been developed, which are available to the larger scientific community. The approach in all these packages is to utilize pre‐existing Python modules as well as to be object‐oriented and scalable. The first package that will be described and demonstrated is the Python Advanced Microwave Precipitation Radiometer (AMPR) Data Toolkit, or PyAMPR for short. PyAMPR reads geolocated brightness temperature data from any flight of the AMPR airborne instrument over its 25‐year history into a common data structure suitable for user‐defined analyses. It features rapid, simplified (i.e., one line of code) production of quick‐look imagery, including Google Earth overlays, swath plots of individual channels, and strip charts showing multiple channels at once. These plotting routines are also capable of significant customization for detailed, publication‐ready figures. Deconvolution of the polarization‐varying channels to static horizontally and vertically polarized scenes is also available. Examples will be given of PyAMPR's contribution toward real‐time AMPR data display during the Integrated Precipitation and Hydrology Experiment (IPHEx), which took place in the Carolinas during May‐June 2014. The second software package is the Marshall Multi‐Radar/Multi‐Sensor (MRMS) Mosaic Python Toolkit, or MMM‐Py for short. MMM‐Py was designed to read, analyze, and display three‐dimensional national mosaicked reflectivity data produced by the NOAA National Severe Storms Laboratory (NSSL). MMM‐Py can read MRMS mosaics from either their unique binary format or their converted NetCDF format. It can also read and properly interpret the current mosaic design (4 regional tiles) as well as mosaics produced prior to late July 2013 (8 tiles). MMM‐Py can easily stitch multiple tiles together to provide a larger regional or national picture of precipitating weather systems. Composites, horizontal and vertical crosssections, and combinations thereof are easily displayed using as little as one line of code. MMM‐Py can also write to the native MRMS binary format, and sub‐sectioning of tiles (or multiple stitched tiles) is anticipated to be in place by the time of this meeting. Thus, MMM‐Py also can be used to power the creation of custom mosaics for targeted regional studies. Overlays of other data (e.g., lightning observations) are easily accomplished. Demonstrations of MMM‐Py, including the creation of animations, will be shown. Finally, Marshall has done significant work to interface Python‐based analysis routines with the U.S. Department of Energy's Py‐ART software package for radar data ingest, processing, and analysis. One example of this is the Python Turbulence Detection Algorithm (PyTDA), an MSFC‐based implementation of the National Center for Atmospheric Research (NCAR) Turbulence Detection Algorithm (NTDA) for the purposes of convective‐scale analysis, situational awareness, and forensic meteorology. PyTDA exploits Py‐ART's radar data ingest routines and data model to rapidly produce aviation‐relevant turbulence estimates from Doppler radar data. Work toward processing speed optimization and better integration within the Py‐ART framework will be highlighted. Python‐based analysis within the Py‐ART framework is also being done for new research related to intercomparison of ground‐based radar data with satellite estimates of ocean winds, as well as research on the electrification of pyrocumulus clouds.

Lang, Timothy J.↗

Scanning ARM Precipitation Radar (SAPR) Velocity Azimuth Display (VAD)

The Velocity Azimuth Display (VAD, historical term) product contains retrievals of the zonal (north/south) and meridional (east/west) wind speed components, in precipitating systems, as a function of time and height. This product, which uses azimuthal variations in radial air velocity, allows quick insight into wind speed and wind direction in the vicinity of the radar when rainfall or snowfall is present. VAD data are derived from Corrected Moments in Antenna Coordinates Version 2 (CMAC2) using code within the Python ARM Radar Toolkit (Py-ART), an open-source architecture for interacting with radar data in the Python programming language.

54 ENVIRONMENTAL SCIENCES↗

Scanning ARM Precipitation Radar (SAPR) Quasi-Vertical Profile (QVP)

The Quasi-Vertical Profiles (QVP) VAP averages quality-controlled radar measurements along the axis of the azimuth. QVP allows quick insight into domain mean hydrometeor (e.g., snow, rain, melting layer) location and polarimetric data. QVP data are derived from Corrected Moments in Antenna Coordinates Version 2 (CMAC2) using code within the Python ARM Radar Toolkit (Py-ART), an open-source architecture for interacting with radar data in the Python programming language.

54 ENVIRONMENTAL SCIENCES↗

Mapped Moments to a Cartesian Grid (MMCG) Value-Added Product Report

Objective analysis (OA) is a method of mapping unstructured data to a structured grid. In the context of scanning radar data, OA is used to interpolate data in antenna coordinates (range, azimuth, and elevation) onto a regularly spaced Cartesian grid (Trapp and Doswell 2000). The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s Mapped Moments to a Cartesian Grid (MMCG) Value-Added Product (VAP) uses the Python ARM Radar Toolkit (Py-ART), a data model-driven interactive architecture for working with weather radar data, to map the data to a Cartesian grid (Helmus and Collis 2016). MMCG, with Py-ART built in, has the ability to take radar data in antenna coordinates and map the gates to a Cartesian grid using inverse distance weight functions such as Cressman (square) and Barnes (exponential), but also can filter the data during the interpolation. MMCG also allows arbitrary formulations for the radius of influence, which are matched to particular radar scanning strategies. This creates a complex parameter space for optimizing the retention of storm structure detail while minimizing artifacts. MMCG takes data processed with ARM’s Corrected Precipitation Radar Moments in Antenna Coordinates (CMAC) VAP and maps it to a Cartesian grid as the output product. A variety of fields that have been mapped to the Cartesian grid are then saved to plots to complement each grid file.

54 ENVIRONMENTAL SCIENCES↗

InfraPy: Python-Based Signal Analysis Tools for Infrasound

InfraPy is a Python-based analysis toolkit being development at LANL. The algorithms are intended for ground-based nuclear detonation detection applications to detect, locate, and characterize explosive sources using infrasonic observations. The implementation is usable as a stand-alone Python library or as a command line driven tool operating directly on a database. With multiple scientists working on the project, we've begun using a LANL git repository for collaborative development and version control. Current and planned work on InfraPy focuses on the development of new algorithms and propagation models. Collaboration with Southern Methodist University (SMU) has helped identify bugs and limitations of the algorithms. The current focus of usage development is focused on library imports and CLI.

97 MATHEMATICS AND COMPUTING↗