Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Jupyter”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Files and scripts to support manuscript Shuman et al 2023 FATES-SPITFIRE ecosystem assembly across tropics

The dataset includes the parameter and domain files, relevant output files, and scripts to generate simulations and perform analysis with Jupyter notebooks that support the manuscript Shuman, JK et al 2023 “Dynamic ecosystem assembly and escaping the “fire-trap” in the tropics: Insights from FATES_15.0.0”. We have adapted the fire-behavior and effects module, SPITFIRE, for use with the Functionally Assembled Terrestrial Ecosystem Simulator (FATES), a size-structured vegetation demographic model. We test how climate, fire regime and fire-tolerance plant traits interact to determine the biogeography of tropical forests and grasslands. We assign different fire-tolerance strategies based on crown, leaf and bark characteristics, which are key observed fire-tolerance traits across woody plants. For these simulations, three types of vegetation compete for resources: a fire-vulnerable tree with thin bark, a vulnerable deep crown and fire-intolerant foliage; a fire-tolerant tree with thick bark, a thin crown and fire-tolerant foliage; and a fire-promoting C4 grass. We explore the model sensitivity to a critical parameter governing fuel moisture, and show that drier fuels promote increased burning, an expansion of area for grass and fire-tolerant trees and a reduction of area for fire-vulnerable trees. This conversion to lower biomass or grass areas with increased fuel drying results in increased fire burned area and its effects, which could fee back to local climate variables. Simulated size-based fire mortality for trees less than 20 cm in diameter and those with fire-vulnerable traits is higher than that for larger and/or fire-tolerant trees, in agreement with observations. Fire-disturbed forests demonstrate reasonable productivity and capture observed patterns of aboveground biomass in areas dominated by natural vegetation for the recent historical period, but have a large bias in less disturbed areas. Though the model predicts a greater extent of burned fraction than observed in areas with grass dominance, the resulting biogeography of fire-tolerant, thick-bark trees and fire-vulnerable, thin-bark trees corresponds to observations across the tropics. In areas with more than 2500 mm of precipitation, simulated fire frequency and burned area are low, with fire intensities below 150 kW m-1, consistent with observed understory fire behavior across the Amazon. Areas drier than this demonstrate fire intensities consistent with those measured in savannas and grasslands, with high values up to 4000 kW m-1. The results support a positive grass-fire feedback across the region, and suggest that forests which have existed without frequent burning may be vulnerable at higher fire intensities, which is of greater concern under intensifying climate and land use pressures. The ability of FATES to capture the connection between fire disturbance and plant fire-tolerance strategies in determining biogeography provides a useful tool for assessing the vulnerability and resilience of these critical carbon storage areas under changing conditions across the tropics.

54 ENVIRONMENTAL SCIENCES↗

PARETO 0.9.0 Release

PARETO 0.9.0 Release. Highlights: New Features - Initial beneficial reuse implementation - Post-optimization timing for infrastructure buildout Bug Fixes - Address Pyomo solver bug for UI Gurobi solve - Update Toy Case Study to feasible data for hydraulics post_process - Update Jupyter notebook for fall release UI Updates - Added functionality to optimize with hydraulics options - Added new plots to KPI dashboard for water quality and hydraulics timelines

PARETO,PARETO-UI,PSE,Process Systems Engineering,P↗

IDAES-PSE 2.6.0 Release

The Institute for the Design of Advanced Energy Systems (IDAES) Integrated Platform is a versatile computational environment offering extensive process systems engineering (PSE) capabilities for optimizing the design and operation of complex, interacting technologies and systems. IDAES enables users to efficiently search vast, complex design spaces to discover the lowest cost solutions while supporting the full process modeling lifecycle, from conceptual design to dynamic optimization and control. The extensible, open platform empowers users to create models of novel processes and rapidly develop custom analyses, workflows, and end-user applications. IDAES-PSE 2.6.0 Release Highlights Upcoming Changes IDAES will be switching to the new Pyomo solver interface in the next release. Whilst this will hopefully be a smooth transition for most users, there are a few important changes to be aware of. The new solver interface uses a different version of the IPOPT writer (“ipopt_v2”) and thus any custom configuration options you might have set for IPOPT will not carry over and will need to be reset. By default, the new Pyomo linear presolver will be activated with ipopt_v2. Whilst are working to identify any bugs in the presolver, it is possible that some edge cases will remain. IDAES will begin deploying a new set of scaling tools and APIs over the next few releases that make use of the new solver writers. The old scaling tools and APIs will remain for backward compatibility but will begin to be deprecated. New Models, Tools and Features New Intersphinx extension automatically linking Jupyter notebook examples to project documentation New end-to-end diagnostics example demonstrated on a real problem New complementarity formulation for VLE with cubic equations of state, backward compatibility for old formulation New solver interface with presolve (ipopt_v2) in support of upcoming changes to the initialization and APIs methods, with default set to ipopt to maintain backwards compatibility; this will deprecate once all examples have been updated New forecaster and parameterized bidder methods within grid integration library Updated surrogates API and examples to support Keras 3, with backwards compatibility for older formats such as TensorFlow SavedModel (TFSM) Updated costing base dictionary to include the 2023 cost year index value Updated ProcessBlock to include information on the constructing block class Updated Flowsheet Visualizer to allow visualize() method to return value and functions Bug Fixes Fixed bug in the Modular Property Framework that would cause errors when trying to use phase-based material balances with phase equilibria. Fixed bug in Modular Properties Framework that caused errors when initializing models with non-vapor-liquid phase equilibria. Fixed typos flagged by June update to crate-ci/typos and removed DMF-related exceptions Minor corrections of units of measurement handling in power plant waste/transport costing expressions, control volume material holdup expressions, and BTX property package parameters Fixed throwing >7500 numpy deprecation warnings by replacing scalar value assignment with element extraction and item iteration calls Testing and Robustness Migrated slow tests (>10s) to integration, impacting test coverage but also yielding a nearly 30% decrease in local test runtime Pinned pint to avoid issues with older supported Python versions Pinned codecov versions to avoid tokenless upload behavior with latest version Bumped extensions to version 3.4.2 to allow pointing to non-standard install location Deprecations and Removals Python 3.8 is no longer supported. The supported Python versions are 3.9 through 3.12 The Data Management Framework (DMF) is no longer supported. Importing idaes.core.dmf will cause a deprecation warning to be displayed until the next release The SOFC Keras surrogates have been removed. The current version of the SOFC surrogate model in the examples repository is a PySMO Kriging model.

AS↗

python-regionalizer

A Python module and Jupyter Notebook for data regionalization (i.e., discretizing data to county or community levels).

LCA; data processing↗

CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluation

Evaluation beyond aggregate performance metrics, e.g. F1-score, is crucial to both establish an appropriate level of trust in machine learning models and identify future model improvements. In this paper we demonstrate CrossCheck, an interactive visualization tool for rapid crossmodel comparison and reproducible error analysis. We describe the tool and discuss design and implementation details. We then present three use cases (named entity recognition, reading comprehension, and clickbait detection) that show the benefits of using the tool for model evaluation. CrossCheck allows data scientists to make informed decisions to choose between multiple models, identify when the models are correct and for which examples, investigate whether the models are making the same mistakes as humans, evaluate models’ generalizability and highlight models’ limitations, strengths and weaknesses. Furthermore, CrossCheck is implemented as a Jupyter widget, which allows rapid and convenient integration into data scientists’ model development workflows.

Arendt, Dustin L.↗

OpenOA: An Open-Source Codebase For Operational Analysis of Wind Farms

OpenOA is an open source framework for operational data analysis of wind energy plants, implemented in the Python programming language. OpenOA provides a common data model, high level analysis workflows, and low-level convenience functions that engineers, analysts, and researchers in the wind energy industry can use to facilitate analytics workflows on operational data sets. OpenOA contains documentation, worked out examples in Jupyter notebooks, and a corresponding example dataset from the Engie Renewable’s La Haute Borne Dataset.

17 WIND ENERGY↗

Dimensionally Aligned Signal Projection Algorithms Library

Dimensionally aligned signal projection (DASP) algorithms are used to analyze fast Fourier transforms (FFTs) and generate visualizations that help focus on the harmonics for specific signals. At a high level, these algorithms extract the FFT segments around each harmonic frequency center, and then align them in equally sized arrays ordered by increasing distance from the base frequency. This allows for a focused view of the harmonic frequencies, which, among other use cases, can enable machine learning algorithms to more easily identify salient patterns. This work seeks to provide an effective open-source implementation of the DASP algorithms proposed by Vann et al. (2018) as well as functionality to help explore and test how these algorithms work with an interactive dashboard and signal-generation tool. The DASP library is implemented in Python and contains four types of algorithms for implementing these feature engineering techniques: fixed harmonically aligned signal projection (HASP), decimating HASP, interpolating HASP, and frequency aligned signal projection (FASP). Each algorithm returns a numerical array, which can be visualized as an image. The HASP algorithms are variations of the algorithms originally presented by Vann et al. (2018). For consistency, FASP, which is the terminology used for the short-time Fourier transform (STFT), has been implemented as part of the library to provide a similar interface to the STFT of the raw signal. Additionally, the library contains an algorithm to generate artificial signals with basic customizations such as the base frequency, sample rate, duration, number of harmonics, noise, and number of signals. Finally, the library provides multiple interactive visualizations, each of which is implemented using IPyWidgets and works in a Jupyter environment. A dashboard-style visualization is provided, which contains some common signal-processing visual components (signal, FFT, spectogram) updating in unison with the HASP functions (see Figure 1 below). Separate from the dashboard, an independent visualization is provided for each of the DASP algorithms as well as for the artifical signal generator. These visualizations are included in the library to aid in developing an intuitive understanding how the algorithms are affected by different input signals and parameter selections.

harmonics↗

ICAT: The Interactive Corpus Analysis Tool

The Interactive Corpus Analysis Tool (ICAT) is a Python library for creating dashboards to explore textual datasets and build simple binary classification models to help filter through them and focus on entries of interest. This tool uses a form of interactive machine learning (IML), a paradigm of “machine teaching” (Simard et al., 2017) that sits at the intersection of the fields of human computer interaction (HCI), visual analytics, and machine learning. The intent of ICAT is to allow subject matter experts (SME) with limited to no experience in machine learning to benefit from an iterative human-in-the-loop (HITL) approach to building their own model without needing to understand the details of the underlying algorithm. This interactivity is achieved by allowing the user to create features, label data points, and visually manipulate a representation of the features to manually cluster and investigate data, while a model is trained on the fly based on these actions. ICAT is built on top of the Panel (Holoviz, 2018) library, using a combination of Vega, a custom IPyWidget using D3, and ipyvuetify, and is intended to be used inside of a Jupyter environment.

Martindale, Nathan [Oak Ridge National Laboratory ↗

Usable Data Abstractions for Next-Generation Scientific Workflows

Data- and computationally-intensive scientific research, such as numerical simulations and inversions or the training of large neural networks in machine learning applications, that are well suited for HPC environments also often require expert insight and evaluation throughout the computation which can be greatly facilitated with the use of interactive computing tools, such as those in the Jupyter ecosystem. HPC workflows and interactive workflows are typically treated as orthogonal, however, the next generation of research will require both. The first challenge we face in this project is thus designing the right level of abstractions to allow interactive capabilities in the JupyterLab environment to allow the working scientist to flexibly explore and query their data at multiple levels, with a minimal amount of customization required of the underlying optimized codes. In addition to these questions regarding the high-level representation of data for interactive use in HPC, we tackled two additional issues that are part of the entire lifecycle of research and that become particularly acute in HPC contexts: how to improve the experience of interfacing with the HPC system's scheduling environment for a scientist focused on exploratory questions, and how can that scientist then best share the results of their work with others in a self-contained, reproducible manner.

97 MATHEMATICS AND COMPUTING↗

IBM Open Science Price - SWAP Gate Challenge

On November 30, 2020, IBM announced an open science price competition aimed at improving the fidelity of a SWAP gate on their 7 qubit Quantum system named ”Casablanca”. The details of the SWAP gate problem was outlined in a Jupyter notebook, which utilizes the IBM developed open source package Qiskit. The participants were allowed to work in teams with up to five people and restricted to only use open source software in solving the problem. Their results were due on April 16, 2021. Our approach to solving the SWAP gate challenge is based on the quantum optimal control techniques implemented in our open source packages Juqbox.jl and Quandary. The results from the optimal control approach is only as good as the accuracy in the description of the quantum system dynamics, characterized by a Hamiltonian model that was provided by IBM. Based on the calibrated control pulses IBM provide for the standard gate set, we developed a reverse engineering approach to calibrate our computational model. Techniques were developed to translate between Qiskit’s pulse representation and the B-spline formulation used in Juqbox and Quandary. The fidelity of the optimized pulse sequences were estimated using Qiskit’s randomized bench marking techniques.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

simple_RDE_model

Jupyter notebook containing a simple rotating detonation engine model to generate synthetic data for exploring data analysis techniques.

42 ENGINEERING↗

Testing Job Submission to FermiGrid via Dask on the Fermilab Elastic Analysis Facility

Over the course of a summer internship, Jobsub Lite task submission and Dask operations were tested in development and production EAF at Fermilab, first via terminal and subsequently via Jupyter notebook cell. Testing allowed for development of a script available to EAF users that expands and makes easier access to parallel computing resources at Fermilab and associated sites.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Another Set of Python Tools for Visualizing and Manipulating Small-Angle Neutron Scattering Data: Descriptions and Examples

The GP-SANS, Bio-SANS and EQ-SANS instruments at ORNL utilize drtsans for data reduction. drtsans is built on Python, and it can be run using python scripts and Jupyter notebooks. The flexibility afforded by Python makes it possible to incorporate additional actions into the scripts used for data reduction, such as analysis and visualization. Here, a new set of tools for visualizing and manipulating SANS data that can be incorporated into the data reduction scripts for the ORNL SANS instruments, or employed during post–processing, is presented that expands the capabilities of the two previously-released tool sets.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗

Quantifying Capital Cost Reduction Pathways for Advanced Nuclear Reactors

The framework developed in this study is provided both as an excel sheet (https://inl.gov/content/uploads/2023/11/Nuclear-Reactor-Cost-Reduction-Pathway-Spreadsheet-Tool.xlsx) and a Python (Jupyter) Notebook (link: https://github.com/accert-dev/ACCERT/tree/main/Cost%20Reduction). Capital cost considerations are one of the primary inhibitors to the large-scale deployment of nuclear power plants. While it is widely accepted that first units will likely be expensive and relatively uncompetitive, it is reasonable to expect that subsequent units, built in relative quick succession, will be cheaper as they benefit from the so-called “learning effects”. However, the large degree of uncertainty associated with this parameter renders it challenging for first movers to invest in the first few expensive units. To resolve this impasse, the U.S. Department of Energy’s Advanced Nuclear Liftoff study advocated for the formation of large, committed order books of plants of the same technology to spread the costs across several units and kickstart the nuclear supply chain. The study also advocated best practices for avoiding overruns and keeping reactors on budget. This report builds on these key recommendations by attempting to quantify specific pathways toward cost reduction for nuclear energy. A capital cost estimation framework was built to untangle the effect of learning into a subset of key cost drivers, referred to as “levers”. Collectively, the choice of these levers is intended to reflect the decision-making of high-level stakeholders like plant owners and the government. In addition to the size of the firm orderbook, these levers included (a) cost drivers that are most often attributed to cost overruns such as architect/engineering (A/E) proficiency, construction proficiency, procurement service proficiency, design completion prior to the start of construction, and design maturity, and (b) cost reduction strategies such as modular construction, cross-site standardization, safety classification of the reactor building, and of the balance of plant. Two advanced reactor designs were leveraged as use cases and bottom-up cost estimates made with assumptions consistent with a well-executed first-of-a-kind project (WE-FOAK, i.e., almost no overruns) were used as baselines for the models. Cost correlations were surveyed from the literature to determine the impact of important variables on projected timelines and costs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

US Department of Energy, Office of Science, High-Performance Computing Facility: 2023 Operational Assessment Oak Ridge Leadership Computing Facility

The Oak Ridge Leadership Computing Facility (OLCF) was established to accelerate scientific discovery by providing world-leading computational performance and advanced data infrastructure. As a US Department of Energy (DOE) Office of Science user facility, the OLCF has managed the successful deployment and operation of a succession of leadership-class resources dedicated to open science. In addition to these resources, the OLCF staff continually strive to develop innovative processes and technologies, improve security, and empower users through effective allocation management and comprehensive user support and training. These efforts support the advancement of science by the OLCF users and benefit high-performance computing (HPC) facilities around the world. In calendar year (CY) 2023, the OLCF supported 1,676 users and 598 projects and exceeded all targets for user satisfaction. The facility received an average satisfaction score of 4.52 out of 5 on the annual user survey, and 94% of respondents reported a high satisfaction rate with the OLCF overall. Of the 3,619 user tickets submitted in CY 2023, OLCF staff resolved 97% within 3 business days. The facility opened Frontier to full scientific operations this year. Two projects conducted on Frontier received the Association for Computing Machinery (ACM) Gordon Bell Prize and the Gordon Bell Special Prize for Climate Modeling, and a third earned a nomination as a Gordon Bell Prize finalist. The facility’s previous flagship machine, Summit, gained new life and was extended through 2024 in part to help provide resources to the Integrated Research Infrastructure (IRI) projects and the National Artificial Intelligence Research Resource (NAIRR) pilot program. The facility instantiated an Advanced Computing Ecosystem testbed in part to support IRI workflows. OLCF made interactivity easier and more accessible to users than ever through tools like Jupyter notebooks and workflows.

97 MATHEMATICS AND COMPUTING↗

Mid-circuit Measurement & Branching in QSCOUT: A Ping-Pong Teleportation Exemplar Program

This document is intended to help users program the new mid-circuit measurement (MCM) and classical branching capabilities of the Quantum Scientific Computing Open User Testbed (QSCOUT). Here, we present and explain an exemplar “ping-pong teleportation” program that makes repeated MCM and branching calls. The program is written in Jaqal, the quantum assembly language used by QSCOUT. This document is intended to accompany a companion Jupyter notebook Exemplar_one_bit_teleportation_pingpong.ipynb.

97 MATHEMATICS AND COMPUTING↗

Evidence-based Graph Adversary Mapping (EGRAM) [Poster]

Cybersecurity companies such as CrowdStrike, Dragos, Microsoft and Unit 42 categorize Advanced Persistent Threats (APTs) using their own naming schemes. As a result, these APTs are mapped to different malware sources and campaigns, all from differing sources, leading to inconsistent mapping. Inconsistent mapping causes confusion and adds further obscurity around these groups, making it difficult to track and mitigate APT cyberattacks. The Evidence-based Graph Adversary Mapping (EGRAM) tool remediates the mapping challenge by collecting, updating and converting adversary data and their sources into a valid, codified STIX v2.1 bundle which is then stored in a Neo4j graph database. It utilizes graph traversal methods and centrality analysis to generate actionable information as a Structured Threat Intelligence Graph (STIG), based on user queries. EGRAM exists as Python code and a Jupyter Notebook that acts as a searchable, evidence-based, source of intelligence for APT groups’ artifacts and cyber campaigns.

24 - POWER TRANSMISSION AND DISTRIBUTION↗