Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Jupyter notebook”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Data and scripts associated with the manuscript evaluating the hydrologic responses of the Pacific Northwest watersheds to wildfires (v2)

This data package is associated with the publication “Evaluating Post-fire Watershed Response to Varying Burn Severity and Precipitation Regimes Using Fully-distributed and Integrated Hydrologic Models” submitted to Journal of Hydrology (Li et al. 2025). In this study, we employed the Advanced Terrestrial Simulator (ATS), an integrated watershed model that couples surface flow, subsurface flow, and canopy biophysical processes, to investigate post-fire hydrologic responses in a few selected watersheds with varying burn severity.The data package contains the required input data (meteorological forcing, Leaf Area Index, wildfire burn severities, etc.) to run the model, configuration files, the Jupyter notebooks in Python to pre-process and post-process data, the figures in the manuscript, and the modeling output files. The variables include watershed-averaged evapotranspiration, watershed-averaged surface/subsurface/canopy water content, and river discharge at watershed outlet.The data package contains a file-level metadata that lists and describes all the files contained in the data package (ATS_flmd.csv), a data dictionary file that defines columns headers across all csv files contained in the data package (ATS_dd.csv), a data package level readme file (the current file), and four zipped folders.The ‘data’ folder provides data needed to run the model in .h5, .i2s, .xyz, .shp, and .exo formats. The sub-folders are for each data types. The ‘model’ folder provides input files (.xml format) and essential model outputs. Each sub-folder provides the files from each simulated watershed. The ‘notebooks’ folder provides the Jupyter notebooks (.ipynb format) for pre- and post- processing model files, and for producing the figures in the manuscript. The ‘figures’ folder provides the figures associated with manuscript in .pdf and .png formats.The ‘model’ folder and the ‘data’ folder have been split into 5GB-large pieces using the Linux command ‘split -b 5120m model.zip model.zip.’ and ‘split -b 5120m data.zip data.zip.’, respectively. They can be merged back using the Linux command ‘cat model.zip.* > model.zip’ and ‘cat data.zip.* > data.zip’, respectively.

54 ENVIRONMENTAL SCIENCES↗

EDD Basic Stats and Graphs Notebook analysis (EDD BSG Notebook) v1.0

This jupyter notebook calculates basic statistics (e.g., mean, standard deviation, coefficient of variation) and simple graphs (e.g., bar graphs, line plots) for data from the Experiment Data Depot (EDD) to provide rapid and reproducible assessment of data quality to aid research efforts across the JBEI and ABF projects. It rapidly and reproducibly calculates basic statistical values for data stored in the EDD which aids researchers and strengthens comparisons across different experiments and projects.

Petzold, ChristopherJ↗

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle↗

Hydrologic Model Data for the East Fork Poplar Creek Watershed Simulated with the Advanced Terrestrial Simulator (ATS): Streamflow and Network Expansion–Contraction Dynamics

This dataset supports hydrologic modeling and stream network expansion–contraction analysis for the East Fork Poplar Creek (EFPC) Watershed in Tennessee. It includes a Jupyter notebook for model setup, model configuration files, simulation outputs, and derived products used to evaluate model performance and investigate stream dynamics under varying hydrologic conditions. The dataset was generated using the Watershed Workflow Python package and the Advanced Terrestrial Simulator (ATS), enabling integrated surface–subsurface hydrologic simulations using a stream-aligned mesh. Outputs include high-resolution time series of streamflow, active network length, water table depth, and related hydrologic variables. Also included are spatially explicit stream persistency indices and classifications of reaches as perennial or non-perennial. These data facilitate reproducibility and support further research on stream intermittency and variability in network extent.The model data archive is organized in following directories:1) model_setup_inputsContains the Watershed Workflow Jupyter notebooks (accessed through any open source code editor), selected input datasets, and resulting ATS input files, including XML files (access through any open source code editor), computational mesh (.exo files can be viewed using Paraview), and meteorological forcing files (.h5 files can be accessed through h5py python package and HDFView open source software). 2) model_outputsIncludes ATS simulation outputs relevant to this study. Time series of spatially integrated or averaged variables (e.g., streamflow, water table depth) are provided as CSV files. Select spatial fields (e.g., ponded depth and water table depth) are saved as pickled Python objects to reduce file size, and can be accessed through pickle package in Python. Key geometry objects from Watershed Workflow—such as the surface mesh and river tree—are also included to support analysis of streamflow persistency and expansion–contraction dynamics. These files can also be accessed through Watershed Workflow Python package.3) model_evaluationProvides observed streamflow time series and field survey-based flow regime classifications used to evaluate model performance. Jupyter notebooks for processing ATS outputs and comparing model predictions with observations to build confidence in the model prior to scientific analysis are also included.4) Q_L_relationshipsContains workflows for generating time series of discharge, active network length, and related hydrologic variables used in the stream network expansion–contraction analysis. Includes routines for delineating baseflow-dominated periods. For each catchment, notebooks and processed data (as pickled DataFrames accessed through Pandas Python package) are provided. 5) figure_scriptsProvides the Jupyter notebooks used to generate the figures presented in the paper.

54 ENVIRONMENTAL SCIENCES↗

A Novel Architecture of JupyterHub on Amazon Elastic Kubernetes Service for Open Data Cube Sandbox

The Open Data Cube (ODC) initiative, with support from the Committee on Earth Observation Satellites (CEOS) System Engineering Office (SEO) has developed a state-of-the-art suite of software tools and products to facilitate the analysis of Earth Observation data. This paper presents a short summary of our novel architecture approach in a project related to the Open Data Cube (ODC) community that provides users with their own ODC sandbox environment. Users can have a sandbox environment all to themselves for the purpose of running Jupyter notebooks that leverage the ODC. This novel architecture layout will remove the necessity of hosting multiple users on a single Jupyter notebook server and provides better management tooling for handling resource usage. In this new layout each user will have their own credentials which will give them access to a personal Jupyter notebook server with access to a fully deployed ODC environment enabling exploration of solutions to problems that can be supported by Earth observation data.

Open Data Cube↗

Data and Scripts associated with “Lambda-PFLOTRAN: Workflow for Incorporating Organic Matter Chemistry Informed by Ultra High Resolution Mass Spectrometry into Biogeochemical Modeling.”

This data package is associated with the publication “Lambda-PFLOTRAN: Workflow for Incorporating Organic Matter Chemistry Informed by Ultra High Resolution Mass Spectrometry into Biogeochemical Modeling” submitted to Geoscientific Model Development (Muller et al., 2024). In this manuscript, organic matter chemistry and thermodynamics are directly connected to reactive transport simulators through the newly developed Lambda-PFLOTRAN (Parallel Reactive Flow and Transport model) workflow tool that succinctly incorporates organic matter chemistry data generated from Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) into reaction networks to simulate aerobic respiration of the organic matter and the resulting biogeochemistry. Lambda-PFLOTRAN is a python-based workflow, executed through a Jupyter Notebook interface, that digests raw FTICR-MS data, develops a representative reaction network based on substrate-explicit thermodynamic modeling (also termed lambda modeling due to its key thermodynamic parameter λ used therein), and completes a biogeochemical simulation with the open source, reactive flow, and transport code PFLOTRAN. This data package contains Jupyter Notebook based workflows for two test cases for running biogeochemical simulations of organic matter oxidation identified by FTICR-MS. It contains four primary folders (workflow, data, src, and analysis), a file-level metadata file (Muller_2024_Lambda_PFLOTRAN_Manuscript_Data_Package_flmd.csv) that lists all the files contained in this data package with a short description of each, and a data dictionary (Muller_2024_Lambda_PFLOTRAN_Manuscript_Data_Package_dd.csv) file that describes the tabular column headers. The ‘workflow’ folder contains the Jupyter Notebook based workflows for running the lambda analysis, PFLOTRAN simulation, sensitivity analysis and parameter estimation. The ‘data’ folder contains the FTICR-MS data, initial conditions, and incubation data for test cases 1 and 2 in folders titled ‘WHONDRS’ and ‘Colloids’, respectively. The data folder also has a ‘Database’ folder containing a reaction network for bulk organic matter (assumed to be CH2O) and a general database for PFLOTRAN (hanford_rxn_network). The CH2O reaction network defines bulk organic matter oxidation. Biogeochemical simulations are completed for both the lambda binned organic matter and bulk organic matter reaction networks. The ‘hanford_rxn_network’ database includes information required for PFLTORAN simulations including ion size, molar mass, and charge of the aqueous species, gases, and minerals phases. The ‘src’ folder contains python source codes for performing lambda analysis, PFLOTRAN simulation, sensitivity analysis and parameter estimation. The ‘analysis’ folder contains outputs from the test cases 1 and 2 including lambda analysis, PFLOTRAN runs and the calibration results.

54 ENVIRONMENTAL SCIENCES↗

Files and scripts to support manuscript Needham et al. Canopy Gradients of Respiration

This dataset includes the parameter files, relevant output files, and scripts to perform analysis with Jupyter notebooks that support the manuscript Needham et al 2025 “Canopy Gradients of Respiration Drive Plant Carbon Budgets and Leaf Area Index.” We add functionality to the Functionally Assembled Terrestrial Ecosystem Simulator (FATES) to allow flexible vertical gradients of leaf maintenance respiration (Rdark) and maximum carboxylation rate (Vcmax) through the canopy. We test the sensitivity of FATES to canopy gradients in Rdark, both in global simulations to assess broad scale impacts on leaf area index (LAI) and vegetation carbon, and in single site simulations where we assess impacts on plant functional type (PFT) competitive dynamics. Parameter files are netcdf files that can be converted to human readable .cdl files using NCO tools. Analysis scripts are Jupyter notebook files. These can be opened and run using the open source Jupyter notebook software. Model outputs are netcdf files.

54 ENVIRONMENTAL SCIENCES↗

TXM-Sandbox : an open-source software for transmission X-ray microscopy data analysis

A transmission X-ray microscope (TXM) can investigate morphological and chemical information of a tens to hundred micrometre-thick specimen on a length scale of tens to hundreds of nanometres. It has broad applications in material sciences and battery research. TXM data processing is composed of multiple steps. A workflow software has been developed that integrates all the tools required for general TXM data processing and visualization. The software is written in Python and has a graphic user interface in Jupyter Notebook . Users have access to the intermediate analysis results within Jupyter Notebook and have options to insert extra data processing steps in addition to those that are integrated in the software. The software seamlessly integrates ImageJ as its primary image viewer, providing rich image visualization and processing routines. As a guide for users, several TXM specific data analysis issues and examples are also presented.

36 MATERIALS SCIENCE↗

Transformer eXplainability and eXploration

The Transformer eXplainability and eXploration library is intended to aid in the explorability and explainability of transformer classification networks, or transformer language models with sequence classification heads. The basic function of this library is to take a trained transformer and test/train dataset and produce an ipywidget dashboard which can be displayed in a jupyter notebook or in jupyter lab.

Martindale, Nathan [Oak Ridge National Lab. (ORNL)↗

CTT: Tools for Fine Alignment of Flash X-ray Systems

The CTT code is a compilation of the python code that was prototyped in the interest of making a method for fine alignment of Flash X-ray Systems at the lab. It contains python code for interacting with LTT and running optimization loops using LTT simulations or ray tracing with RaySpace. It Includes classes for creating phantom objects and modular geometry objects in the simulations. It allows one to generate a random phantom that has been optimized to stay with a certain cubic volume and maximize the minimum distance between any two ball phantom centers in all projections. It was primarily made to work on ball phantoms which were simulated as Teflon spheres. It allows the alignment from some nominal position to some displaced position to simulate recovery of a real geometry from the nominal geometry of the system. Fine alignment can be done with the projections in LTT while a rough alignment is faster using rayspace. If rayspace works well for the use case it should be preferred since it is considerably faster and less resource intensive in general. Some Notes about using this code is that one will have to edit the imports for the files to get to the correct LTT path, LTT GUI path and to the CTT path when importing. This was made by using anaconda with python 3 on windows 10. The LTT code was stored in a anaconda environment which seemed to help the python find the LTT although it shouldn’t be necessary if you append the path in your code using sys.path.append(r'add the path you want here’). The sys.path.append method is a quick way to give your python code access to a given folder when running it (such as LTT or CTT). Note that since the code was originally made with paths for a specific machine that a new user will have to edit those paths to make it work in all the files that use the old paths. This is somewhat tedious but will be required to run the code on a new machine. The methodology used to run and make these notebooks starts with opening a command prompt as administrator in windows 10. After this one would activate the anaconda environment in the appropriate directory using “conda activate myenv” in the command prompt. After this open a jupyter-notebook using the keyword “jupyter-notebook". After that a notebook should open in a browser. Note that some libraries used may not come with anaconda so those may have to be installed. There shouldn’t be much though since the main things used are scipy, numpy, matplotlib, and LTT. LTTQuicksetup also requires some paths that will be specific to the computer so that should be changed as well.

42 ENGINEERING↗

Model data for Flood Frequency Analysis using Stochastic Storm Transposition and an Integrated Surface-Subsurface Hydrological Model

This archived provides scripts and input files used for the implementation of a novel approach to conduct process-based Flood Frequency Analysis using a Stochastic Storm Transposition (SST) and an Integrated Surface-Subsurface Hydrological Model (ISSHM). As a proof-of-concept, this study uses the ISSHM, Advanced Terrestrial Simulator (Amanzi-ATS) model, and the SST model, RainyDay, to conduct flood frequency analysis by simulating the flood response to 5,000 annual synthetic storm events in a ~2000 km2 Southeast Texas watershed.The Watershed Workflow package is implemented in Python3. The Jupyter notebooks can be executed through multiple open-source tools, for example, Anaconda Jupyter Lab, VS Studio Code, etc. Other data files include TXT, CSV, DAT, SBATCH, SHP, TIF, NetCDF, and HDF5 files, which can be read through Python scripts. The input files for the ATS model and RainyDay model have .XML and .SST extensions, respectively, and can be edited in any commonly used text editors.This archive contains:* Scripts and data files essential for generating the ATS model input. It uses the Watershed Workflow package to produce both mesh and ATS input files. * Jupyter notebooks designated for the ATS model evaluation, covering both long-term simulations and 40 rainfall-runoff events.* Input files required to simulate SST storm events using RainyDay.

54 ENVIRONMENTAL SCIENCES↗

Automating Surface Attitude Positioning and Pointing Operations for Mars 2020

The Surface Attitude Positioning and Pointing (SAPP) subsystem of the Mars Perseverance rover keeps track of the rover’s position and attitude on the surface of Mars. The SAPP Downlink Engineering Operations team members receive data from the rover on a daily basis. They must interpret the data to make sure the rover is staying safe and to support uplink planning. The SAPP team keeps track of the error growth in the rover’s attitude estimate due to noise in the Rover Inertial Measurement Unit’s (RIMU) gyroscopes used to propagate that attitude estimate whenever the rover is moving. Whenever this error grows to a particular threshold, SAPP is responsible for updating the onboard attitude knowledge using the RIMU’s accelerometers to estimate rover roll and pitch and sun imaging to estimate rover yaw, thereby reducing this attitude estimation error. Accurate attitude estimation is required so that the rover can successfully point its High Gain Antenna (HGA) to receive information from Earth and as a backup to the Mars orbiters used for sending data from the rover to Earth, point instruments on its Remote Sensing Mast (RSM), and support safe movement and placement of instruments by the rover’s ARM relative to the Martian surface. The Mars 2020 Engineering Operations team has been working to increase the operational efficiency of the mission and eventually move to a five-hour timeline for daily operations. In pursuit of this goal, the SAPP Engineering Operations team has automated their downlink process by developing a centralized Jupyter notebook to analyze the data received daily from the rover. The SAPP downlink Jupyter notebook automatically collects the data relevant to the SAPP subsystem and visualizes this information in plots and tables that can be easily read by downlink operators to aid them in assessing the status of the subsystem. Various Application Programming Interfaces (APIs) have been incorporated into the downlink daily notebook to automate the collection and posting of data, such as gathering and posting data products to the cloud. The SAPP team has also developed a SAPP downlink software library that includes functions to aid the notebook in processing data. In addition to assessing the SAPP subsystem on a daily basis, operators need to assess the long-term trending behavior of the subsystem over time. An automated trending process has been developed to collect information from the daily notebooks in order to plot and analyze that data in a centralized place. These daily and trending processes have expedited the SAPP downlink assessment and laid the groundwork to completely automate the SAPP downlink process so that SAPP operators are unnecessary unless something unexpected occurs. This paper will provide an overview of the functions that the SAPP subsystem carries out on a daily basis, and will then dive into the automations that have been developed for daily and trending downlink assessment. An assessment of the downlink efficiency will be provided, along with a summary of lessons learned and work to go. Finally, the authors will discuss how these types of automated spacecraft health assessments could be more broadly used within mission operations.

Zarifian, Anais↗

pnnl/EXPERT2

This software includes the Jupyter notebooks, model pretraining and evaluation code for the EXPERT 2.0 Human-AI Reasoning Engine V0.1. It supports pre-training a Human-AI model for reasoning over multi-layer network representations. It includes prompt-based evaluation framework in a Jupyter notebook for AI reasoning and Jupyter widgets with AI-based techniques for evidence generation and uncertainty quantification to support human-AI reasoning.

Horawalavithana, Sameera↗

Amazon windthrow disturbances are likely to increase with storm frequency under global warming: Data and Codes

This zipfile includes datasets and codes that were used to produce the results in the paper entitled Amazon windthrow disturbances are likely to increase with storm frequency under global warming. Datasets include: 1. Windthrow density across the entire Amazon - GIS shapefile format 2. Current ERA 5 mean afternoon convective available potential energy (CAPE) (1990-2019) - Remote Sensing TIFF format 3. Estimated future mean CAPE from 10 models in CMIP 6 (2070-2099) - Remote Sensing TIFF format 4. Python codes in jupyter notebook and processed datasets used to generate Fig.2a and Table 1 in the paper. You will need to use Jupyter Notebook and Python for accessing and reading the codes. Please contact Yanlei Feng (ylfeng@berkeley.edu) for any questions. Paper associated with this dataset: Feng, Y., Negrón-Juárez, R.I., Romps, D.M. and Chambers, J.Q., 2023. Amazon windthrow disturbances are likely to increase with storm frequency under global warming. Nature communications, 14(1), p.101.

54 ENVIRONMENTAL SCIENCES↗

PARETO 0.8.0 Release

PARETO 0.8.0 Release. Highlights: Model Updates - Applied unified sets for pipeline and trucking arcs in strategic model - Apply unified sets for pipeline and trucking arcs in operational model - Added new config argument for removal efficiency calculation method - Standardized bidirectional capacity constraint - Added dependencies removed in IDAES 2.1 - Created bounding functions & utilities - Added Hydraulics module to the strategic model - Add additional arc types to strategic model Documentation and Tutorial Updates - Improved PARETO treatment document - Introduced general tutorial and treatment module Jupyter notebooks for Strategic Model - Update docs with correct support email list address - Consolidate and deduplicate Getting Started and resources for developers - Enable Black formatting for Jupyter notebooks - Add Binder configuration files and README Bug Fixes - Fix strategic model documentation typos - Removed duplicated units from output file header UI Updates - Added view for comparing different scenarios - Added functionality for manually overriding PARETO decisions

PARETO,PARETO-UI,PSE,Process Systems Engineering,P↗

PACE Water Resources: Demonstrating the Use of NASA's PACE Hyperspectral Ocean Color Instrument Data for Enhanced Coastal Management

This project developed tools to support the future use of Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) hyperspectral imagery in water resource monitoring and research by NASA DEVELOP teams and members of the PACE applications community. We sought to address a need for support in processing and visualizing hyperspectral PACE Ocean Color Instrument (OCI) data among researchers and decision-makers working in coastal water quality management and harmful algal bloom (HAB) monitoring. To supplement the day of simulated PACE imagery available, we used Aqua MODIS earth observations with Level 3 processing from March 2022 to build a Python graphical user interface (GUI) for visualizing ocean biogeochemical parameters relevant to the early detection and monitoring of HABs. We used simulated PACE OCI Level 2 data derived from the Python Top of Atmosphere Simulation Tool (PyTOAST) to build Jupyter Notebooks for band subset and selection. The Level 3 PACE Viewer components support users with quick visualizations as well as the creation of geoTIFFs and time-series. The Level 2 Jupyter Notebooks address users’ concerns over the volume and complexity of hyperspectral imagery. The PACE Viewer is useful for visual inspection and netCDF data processing but should not be used for geospatial analysis. Once PACE launches, this tool will alleviate the technical burdens of working with hyperspectral data and support the early detection and monitoring of HABs using PACE satellite imagery.

Python Top of Atmosphere Simulation Tool↗

Modeling the Effects of Artificial Drainage on Agriculture-dominated Watersheds using a Fully Distributed Integrated Hydrology Model: Datasets, scripts, model files

This model-data archive supports the research paper that demonstrates the integration of agricultural drainage features—specifically, narrow engineered ditches and tile drains—into a fully distributed, basin-scale integrated surface-subsurface hydrology model (ISSHM), Amanzi-ATS. The model employs innovative computational meshes aligned with agricultural ditches and incorporates the physically based Hooghoudt's drainage equation to simulate tile drainage, offering a novel strategy that enhances the accuracy of hydrological simulations.The archived dataset includes input parameters, model configurations, and select simulation outputs for the Amanzi-ATS model that successfully captured the streamflow patterns in the Portage River Watershed as validated by USGS gauge readings. Jupyter notebook for the preparation of model inputs and post-processing of outputs are also included. The model's predictive performance achieved a normalized Kling-Gupta Efficiency (KGE) of 0.81, surpassing SWAT without the necessity for site-specific calibration.The Amanzi-ATS model presented in this modeL-data archive allows for numerical experiments to explore the shifts in the flow structure under different drainage scenarios. As a tool for advancing the understanding of distributed hydrological responses and nutrient cycling, this archived model provides valuable insights for researchers, modelers, and decision-makers involved in watershed management and environmental modeling.The Watershed Workflow package is implemented in Python3. The Jupyter notebooks can be executed through multiple open-source tools, for example, Anaconda Jupyter Lab, VS Studio Code, etc. Other data files include CSV and HDF5 files, which can be read through Python scripts. The input files for the ATS model, open-source integrated hydrology, and transport model, are in XML format and can be edited in any commonly used text editors.

54 ENVIRONMENTAL SCIENCES↗

BLDAP Intro to Python/Data Science Curriculum v1

The Github repository contains the Jupyter notebooks for the intro to Python / Data Science course for Berkeley Lab Director's Apprenticeship Program (BLDAP). This course is designed for students with little to no experience in coding to learn skills in Python necessary for data science. Students utilize Jupyter notebooks throughout the course. The overall goal is for students to learn how to use Python to clean, analyze, and visualize large data sets in order to communicate effectively their conclusions about the data set. Students apply the skills they learned on actual data sets provided by researchers in Berkeley Lab.

Hales, Laurel [Lawrence Berkeley National Laborato↗