Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Python Programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

PETSc Users Manual (Rev. 3.3)

This manual describes the use of PETSc for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific Computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear, nonlinear equation solvers and time integrators that may be used in application codes written in Fortran, C, C++, Python, and MATLAB (sequential). PETSc provides many of the mechanisms needed within parallel application codes, such as parallel matrix and vector assembly routines. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background, experience programming in C, C++ or Fortran and experience using a debugger such as gdb or dbx, it may require a significant amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates may make the efficient implementation of many application codes simpler than “rolling them” yourself; For many tasks a package such as MATLAB is often the best tool; PETSc is not intended for the classes of problems for which effective MATLAB code can be written. PETSc also has a MATLAB interface, so portions of your code can be written in MATLAB to “try out” the PETSc solvers. The resulting code will not be scalable however because currently MATLAB is inherently not scalable; and PETSc should not be used to attempt to provide a “parallel linear solver” in an otherwise sequential code. Certainly all parts of a previously sequential code need not be parallelized but the matrix generation portion must be parallelized to expect any kind of reasonable performance. Do not expect to generate your matrix sequentially and then “use PETSc” to solve the linear system in parallel. Since PETSc is under continued development, small changes in usage and calling sequences of routines will occur. PETSc is supported; see the web site http://www.mcs.anl.gov/petsc for information on contacting support. A http://www.mcs.anl.gov/petsc/publications may be found a list of publications and web sites that feature work involving PETSc. We welcome any reports of corrections for this document.

97 MATHEMATICS AND COMPUTING↗

PETSc Users Manual (Rev. 3.4)

This manual describes the use of PETSc for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific Computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear, nonlinear equation solvers and time integrators that may be used in application codes written in Fortran, C, C++, Python, and MATLAB (sequential). PETSc provides many of the mechanisms needed within parallel application codes, such as parallel matrix and vector assembly routines. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background, experience programming in C, C++ or Fortran and experience using a debugger such as gdb or dbx, it may require a significant amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates may make the efficient implementation of many application codes simpler than “rolling them” yourself; For many tasks a package such as MATLAB is often the best tool; PETSc is not intended for the classes of problems for which effective MATLAB code can be written. PETSc also has a MATLAB interface, so portions of your code can be written in MATLAB to “try out” the PETSc solvers. The resulting code will not be scalable however because currently MATLAB is inherently not scalable; and PETSc should not be used to attempt to provide a “parallel linear solver” in an otherwise sequential code. Certainly all parts of a previously sequential code need not be parallelized but the matrix generation portion must be parallelized to expect any kind of reasonable performance. Do not expect to generate your matrix sequentially and then “use PETSc” to solve the linear system in parallel. Since PETSc is under continued development, small changes in usage and calling sequences of routines will occur. PETSc is supported; see the web site http://www.mcs.anl.gov/petsc for information on contacting support. A http://www.mcs.anl.gov/petsc/publications may be found a list of publications and web sites that feature work involving PETSc. We welcome any reports of corrections for this document.

97 MATHEMATICS AND COMPUTING↗

PETSc Users Manual (Rev. 3.5)

This manual describes the use of PETSc for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific Computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear, nonlinear equation solvers and time integrators that may be used in application codes written in Fortran, C, C++, Python, and MATLAB (sequential). PETSc provides many of the mechanisms needed within parallel application codes, such as parallel matrix and vector assembly routines. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background, experience programming in C, C++ or Fortran and experience using a debugger such as gdb or dbx, it may require a significant amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates may make the efficient implementation of many application codes simpler than “rolling them” yourself. ;For many tasks a package such as MATLAB is often the best tool; PETSc is not intended for the classes of problems for which effective MATLAB code can be written. PETSc also has a MATLAB interface, so portions of your code can be written in MATLAB to “try out” the PETSc solvers. The resulting code will not be scalable however because currently MATLAB is inherently not scalable; and PETSc should not be used to attempt to provide a “parallel linear solver” in an otherwise sequential code. Certainly all parts of a previously sequential code need not be parallelized but the matrix generation portion must be parallelized to expect any kind of reasonable performance. Do not expect to generate your matrix sequentially and then “use PETSc” to solve the linear system in parallel. Since PETSc is under continued development, small changes in usage and calling sequences of routines will occur. PETSc is supported; see the web site http://www.mcs.anl.gov/petsc for information on contacting support. A http://www.mcs.anl.gov/petsc/publications may be found a list of publications and web sites that feature work involving PETSc. We welcome any reports of corrections for this document.

97 MATHEMATICS AND COMPUTING↗

OpenCGRA: An Open-Source Unified Framework for Modeling,Testing, and Evaluating CGRAs

Coarse-grained reconfigurable arrays (CGRAs),loosely defined as arrays of functional units (e.g, adder, sub-tractor, multiplier, divider, or larger multi-operation units, butsmaller than a general-purpose core) interconnected through aNetwork-on-Chip, provide higher flexibility than domain-specificASIC accelerators while offering increased hardware efficiencywith respect to fine-grained reconfigurable devices, such as FieldProgrammable Gate Arrays (FPGAs). The fast evolving fieldsof machine learning and edge computing, which are seeing acontinuous flow of novel algorithms and larger models, makeCGRAs ideal target architectures to allow domain specializationwithout loosing too much generality. They also generally offerquicker and more effective reconfigurability than FPGAs, po-tentially allowing adaptation during actual algorithm execution,and implement a dataflow programming paradigm that adaptswell to these emerging workloads. Designing and generating aCGRA, however, still requires to define the type and number ofthe specific functional units, implement their interconnect andthe network topology, and perform its simulation and validation,given a variety of workloads of interest.In this paper, we propose OpenCGRA, a Python-based unifiedframework that integrates generation, modeling, testing and eval-uation for CGRAs. OpenCGRA is the first open-source integratedframework able to support the full top-to-bottom design flow forspecializing and implementing CGRAs: modeling at different ab-straction levels (functional level, cycle level, register-transfer level),generation, simulation, testing at different granularities (unit test-ing, integration testing, property-based testing), and characteriza-tion (area, power, and timing). OpenCGRAs will be made availableon GitHub.

CGRA, synthesis↗

SLIA Reference Architecture Models

The SLIA Reference Architecture Models project, sponsored by the DOE CESER Energy CyberSense Program (Oct 2024–Sep 2025), advanced LLNL’s PySCES simulation tool to better support CyTRICS Prioritization and Initial Risk Assessment (PIRA) reference architectures. Key achievements include enhancements to the PySCES transmission substation facility model, expanded asset coverage, and enhancements to the PySCES code base. Software improvements reduced code complexity, migrated PySCES to Python version 3.11, introduced an object-oriented design, and added a schema database for easier updates and validation. New features support device criticality assessments and a more precise parametric simulation mode. Remaining gaps include model validation, workflow limitations, Monte Carlo convergence issues, full device criticality metric implementation, model fidelity, and general software improvements. Continued development is recommended to address these gaps and fully align PySCES with CyTRICS PIRA requirements.

97 MATHEMATICS AND COMPUTING↗

Developing Concepts of Operations Using Multi-Step Tool Techniques With Large Language Models

The National Aeronautics and Space Administration (NASA) Air Mobility Pathfinders (AMP) project is developing and evaluating concepts of operations (ConOps) for safe, secure, and scalable Urban Air Mobility (UAM) operations. The AMP project’s Operational Concepts, Architecture, and Requirements Integration (OCARI) Team is using a Model Based System Engineering (MBSE) approach for integration, interoperability, and traceability of Advanced Air Mobility (AAM) ecosystems centered around urban air taxi services. The team’s goal is to define structures and behaviors needed for system feasibility, readiness, and interoperability, establish a UAM knowledge base, and trace and validate assumptions and requirements relevant to AAM. NASA Langley Research Center (LaRC) is spearheading an innovative digital engineering approach to integrate, communicate, and facilitate the research of multi-modal transportation systems. The Knowledge-based Digital Platform (KbDP) is a concept being developed that ties the workflows of Project Managers (PM), Principal Investigators (PI), and System Engineers together across organizational boundaries. It does so through the management of an information database defined by mathematical, data science, and system engineering principles. Machine Learning (ML) algorithms play a key role in this concept by extracting meaningful knowledge from relational and graph databases, document repositories, and system artifacts, which the human user leverages to greatly improve the efficiency and effectiveness of their research. Recent advancements in the field of Large Language Models (LLMs), specifically models trained for tool use, such as Command-R , now allow for the reliable implementation of single-step and multi-step tool-centric systems. These techniques provide the LLM with a set of tools, in our case Python functions, that can be called on to answer a much wider range of questions compared to LLMs implemented using a traditional single-source or Retrieval Augmented Generation (RAG) approach. Through this method, the LLM can pull information from multiple data sources, such as relational or graph databases, document repositories, application programming interfaces (APIs), and SysML artifacts depending on the user’s question. The LLM can also output the information in a variety of different formats, using output generation tools, such as CSV, UML, or SysML artifacts. Additionally, tools can be assigned roles and can work together to provide answers to queries in an “agent” like approach, similar to that implemented by Microsoft’s AutoGen framework where different agents can converse with each other to accomplish tasks. Previously, our team developed a chatbot system with “agent like” functionality in the form of different “modes” the user could select from a user interface (UI), this architecture can be seen on the left in figure 1. Three different modes were implemented, the first mode allowed the LLM to utilize the structures and algorithms within a graph database to trace UAM requirements. The second mode gave the LLM access to a vector search capable of providing relevant information from thousands of document pages related to UAM ConOps and requirements. The third mode served as a general assistant where users could enter open-ended questions and custom prompts to utilize the LLM for different use-cases. This system improved the process surrounding generating and analyzing information related to UAM requirements, however, the implementation provided a clunky user experience. Users were required to know what mode to select within the UI in advance before entering their question to the selected tool. Moreover, the different tools were isolated from each other, they lacked bidirectional links that would allow for tools to collaborate to generate better responses. Our team is working on a new architecture, seen on the right in the below figure, with the goal to address many of the UX shortcomings of our original system while improving the accuracy and depth of responses from the LLM. This new system will automatically select the appropriate tool to use based off the user’s question. Each tool will be capable of calling on any of the other tools available to the LLM, resulting in a collaborative pipeline where tools can pass data between other tools until enough data is received to generate an answer to the user’s question. Using a locally deployed, open-source, LLM, the NASA OCARI team, in collaboration with Collins Aerospace, will implement a prototype application that will bridge knowledge across multiple sources to assist System Engineers (SEs) with requirements discovery and tracing, research question and use case identification, and assumption validation. Such a system will also allow SEs to more easily, and intuitively, explore the AAM ecosystem, ultimately improving the efficiency and effectiveness of the SE's research and decision-making processes surrounding ConOps development and validation. In this session, our team will provide a video demonstration of our new prototype architecture in action. We will also present an overview of our prototype system architecture and talk about its advantages over traditional LLM deployments along with how those advantages can provide additional value to the field of System Engineering.

systems engineering↗

Installing python software packages : the good, the bad and the ugly.

These slides describe different strategies for installing Python software. Although I am a big fan of Python software development, robust strategies for software installation remains a challenge. This talk describes several different installation scenarios. The Good: the user has administrative privileges - Installing on Windows with an installer executable, Installing with Linux application utility, Installing a Python package from the PyPI repository, and Installing a Python package from source. The Bad: the user does not have administrative privileges - Using a virtual environment to isolate package installations, and Using an installer executable on Windows with a virtual environment. The Ugly: the user needs to install an extension package from source - Installing a Python extension package from source, and PyCoinInstall - Managing builds for Python extension packages. The last item referring to PyCoinInstall describes a utility being developed for the COIN-OR software, which is used within the operations research community. COIN-OR includes a variety of Python and C++ software packages, and this script uses a simple plug-in system to support the management of package builds and installation.

99 GENERAL AND MISCELLANEOUS↗

dfdjaxGP

A small python package to fit Gaussian processes using Jax and leveraging the automatic differentiation in Jax to predict arbitrary derivatives from the GP. This package is meant to supplement the scientific community use of Gaussian process prediction with derivatives. The package is designed to smoothly work standalone or be used with the numpyro probabilistic programming language.

Grosskopf, Micheal↗

The Hydropower Game: An Interactive Learning Tool for the Future Hydropower Workforce

This report documents a Seedling project supported by the U.S. Department of Energy (DOE) Water Power Technologies Office (WPTO), conducted by Argonne National Laboratory (ANL) and the National Laboratory of the Rockies (NLR). The Seedling and Sapling Program provides small grants over short duration to enable early-stage research at national laboratories. This funding mechanism is intended to cultivate innovative ideas and expand research road maps in hydropower and marine energy technologies. Under this Seedling award, the project team developed an educational outreach tool, or “serious game”, built in Python and powered by the Pygame library, aimed at teaching fundamental hydropower concepts through an engaging, interactive learning experience. The game features multiple lessons covering several hydropower topics while allowing for expansion and customization in the instance of future funding availability: • Hydropower Plant Types: Players explore and compare the mechanics and applications of run-of-river, dam hydropower, and pumped-storage hydropower. • Flow Dynamics and Power Generation: Interactive tasks demonstrate how release rate and hydraulic head combine to determine power output. • Grid Operations and Load Following: Simulations illustrate how hydropower facilities respond to fluctuating electricity demand to balance the grid. • Market Integration: Levels scaffold understanding of how hydropower interfaces with the broader energy market, including operational and economic considerations. The tool was showcased at several science, technology, engineering, and mathematics (STEM) outreach events, where it was demoed to students, educators, and the general public. These events provided valuable opportunities to collect feedback on the gameplay, storyline, and educational objectives from a wide audience. The game was presented at STEMapalooza, Introduce a Girl to Engineering Day, STEMCON, and Clean Currents 2025. Insights gathered during these demonstrations informed refinements to the game’s user experience and strengthened its effectiveness as an educational tool for teaching hydropower concepts. This report outlines the game’s design philosophy, educational objectives, technical implementation, user experience insights, and potential for broader deployment within educational and workforce development contexts. It emphasizes how gamified learning can demystify complex hydropower science and inspire interest in water-power technologies. The project’s success demonstrates the value of Seedling funding in fostering creative, lowcost educational tools that support DOE’s mission to advance energy literacy and innovation. The report concludes with recommendations for expanding the tool by adding new levels, integrating assessment metrics, and exploring commercialization or deployment pathways through future Sapling funding. The official webpage of the Hydropower Game, which includes a link to the educational tool, can be accessed at www.anl.gov/hydropower/hydropower-game.

13 HYDRO ENERGY↗

Generating MCNP Input Files for Unstructured Mesh Geometries

The Los Alamos National Laboratory’s (LANL) Monte Carlo N-Particle (MCNP)1 transport code version 6.3 (also known as MCNP6.3) has the capability for tracking particles on unstructured mesh (UM) geometry models embedded into constructive solid geometry (CSG) cells. This feature has been developed for performing calculations of complex geometry models because manually creating CSG models is time-consuming and error-prone as the complexities of geometries increase. A UM geometry model is a collection of finite elements representing a solid geometry. The first step of the MCNP UM calculation is using other software packages to create a finite element mesh representation of a solid 3D geometry because the MCNP code cannot be used to generate a UM model. Computer-aided design (CAD) software is typically used to create a solid geometry model, which is later imported into mesh generation software to create a UM model. Some mesh generation software packages may also be used to create solid geometries and thus CAD files are not needed. The MCNP UM feature was originally designed for models generated by the Abaqus/CAE software suite. The MCNP code version 6.0 and later can process UM models formatted as Abaqus input files. Starting with a 6.3 version, the MCNP code can process HDF5 mesh input files. We only focus on the UM models formatted as Abaqus input files in this report since currently no external software can be used to generate HDF5 mesh input files for MCNP UM calculations. The MCNP code version 6.3 can be used to convert the Abaqus mesh input files into the HDF5 mesh input files, but this option is typically used by the MCNP code development team to test the HDF5 mesh input file feature. Several software packages (such as Abaqus, Attila4MC, or Cubit) can be used to create the Abaqus input files for MCNP UM calculations. An MCNP UM calculation using an Abaqus model requires two input file types: MCNP and Abaqus input files. The Abaqus input files needed for MCNP UM calcu lations must have the correct Abaqus syntax and meet the additional requirements by the MCNP code. The MCNP code can process only Abaqus input files that make use of part and assembly definitions, where elements in each part must be grouped into one or more element sets (i.e., elset) using *Elset keyword lines with specified naming formats. The MCNP and Abaqus input files required for MCNP UM simulations must be related; pseudo-cells in an MCNP input file must be constructed from mesh model data from an Abaqus input file. For large complex UM models, it is tedious to manually create MCNP UM input files. The um pre op (unstructured mesh pre operations) program with the -m option can be used to create a skeleton MCNP input file from an Abaqus input file [6]. Since the um pre op program was written in Fortran and was not written for optimized performance, this program is a deprecated feature in the MCNP code version 6.3 and may be removed in the next release of the code. To improve calculation flow of multiphysics calculations, a Python3 code called write mcnp um input has been developed to generate an MCNP input file instead of using the um_pre_op -m option. This Python code was initially released to the public in 2020. We have updated this Python code for MCNP6.3 and it was used to generate the MCNP input files used to verify the MCNP6.3 code. The write_mcnp_um_input code is included with the MCNP6.3 code package which will be released to the public through the Radiation Safety Information Computational Center (RSICC) at Oak Ridge National Laboratory. This report is a revision of LA-UR-20-27139 report.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Stochastic Models, Indices & Optimization Algorithms for Pricing & Hedging Reliability Risks in Modern Power Grids: Data Plan - Princeton

We collected and cleaned the synthetic grid data produced by NREL for the Texas and New York synthetic grids. We developed a high dimensional joint stochastic model for load at the zone level, and solar and wind power productions at the asset level, capturing the spatial and temporal dependencies between all the variables, and demonstrated how such a model could be fitted to historical data. We designed and implemented a simulation engine which can produce Monte Carlo scenarios for the hourly day-ahead values of load, and solar and wind power productions at the spatial and temporal resolutions of the historical data used to fit the model. Finally we developed an open-source Python package which can, from an input grid model, efficiently use forecasts and large numbers of Monte Carlo scenarios to provide unit commitment and economic dispatch for each of these scenarios. The high dimensional stochastic model and the subsequent Monte Carlo simulation engine were implemented in the package PGscen and the corresponding UC and ED optimization programs in the package Vatic.

14 SOLAR ENERGY↗

Extending molecular simulation time scales: Parallel in time integrations for high-level quantum chemistry and complex force representations

Parallel in time simulation algorithms are presented and applied to conventional molecular dynamics (MD) and ab initio molecular dynamics (AIMD) models of realistic complexity. Assuming that a forward time integrator, f (e.g., Verlet algorithm), is available to propagate the system from time t{sub i} (trajectory positions and velocities x{sub i} = (r{sub i}, v{sub i})) to time t{sub i+1} (x{sub i+1}) by x{sub i+1} = f{sub i}(x{sub i}), the dynamics problem spanning an interval from t{sub 0}…t{sub M} can be transformed into a root finding problem, F(X) = [x{sub i} − f(x{sub (i−1})]{sub i} {sub =1,M} = 0, for the trajectory variables. The root finding problem is solved using a variety of root finding techniques, including quasi-Newton and preconditioned quasi-Newton schemes that are all unconditionally convergent. The algorithms are parallelized by assigning a processor to each time-step entry in the columns of F(X). The relation of this approach to other recently proposed parallel in time methods is discussed, and the effectiveness of various approaches to solving the root finding problem is tested. We demonstrate that more efficient dynamical models based on simplified interactions or coarsening time-steps provide preconditioners for the root finding problem. However, for MD and AIMD simulations, such preconditioners are not required to obtain reasonable convergence and their cost must be considered in the performance of the algorithm. The parallel in time algorithms developed are tested by applying them to MD and AIMD simulations of size and complexity similar to those encountered in present day applications. These include a 1000 Si atom MD simulation using Stillinger-Weber potentials, and a HCl + 4H{sub 2}O AIMD simulation at the MP2 level. The maximum speedup ((serial execution time)/(parallel execution time) ) obtained by parallelizing the Stillinger-Weber MD simulation was nearly 3.0. For the AIMD MP2 simulations, the algorithms achieved speedups of up to 14.3. The parallel in time algorithms can be implemented in a distributed computing environment using very slow transmission control protocol/Internet protocol networks. Scripts written in Python that make calls to a precompiled quantum chemistry package (NWChem) are demonstrated to provide an actual speedup of 8.2 for a 2.5 ps AIMD simulation of HCl + 4H{sub 2}O at the MP2/6-31G* level. Implemented in this way these algorithms can be used for long time high-level AIMD simulations at a modest cost using machines connected by very slow networks such as WiFi, or in different time zones connected by the Internet. The algorithms can also be used with programs that are already parallel. Using these algorithms, we are able to reduce the cost of a MP2/6-311++G(2d,2p) simulation that had reached its maximum possible speedup in the parallelization of the electronic structure calculation from 32 s/time step to 6.9 s/time step.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Program Fuzzing on High Performance Computing Resources

American Fuzzy Lop (AFL) is an evolutionary fuzzer that is strategically implemented as a tool for discovering bugs in software during vulnerability research. This work seeks to understand how to best implement AFL on the High-Performance Computing resources available on the unclassified network at Sandia National Laboratories. We investigate various methods of executing AFL, requesting varying numbers of tasks on single compute nodes with 36 physical cores and 72 total threads. A Python script called Blue Claw is presented as an automated testbed generator tool to assist in the tedious process of creating and executing experiments of any scale and duration.

97 MATHEMATICS AND COMPUTING↗

PcapDB: Search Optimized Packet Capture, Version 0.1.0.0

PcapDB is a packet capture system designed to optimize the captured data for fast search in the typical (network incident response) use case. The technology involved in this software has been submitted via the IDEAS system and has been filed as a provisional patent. It includes the following primary components: capture: The capture component utilizes existing capture libraries to retrieve packets from network interfaces. Once retrieved the packets are passed to additional threads for sorting into flows and indexing. The sorted flows and indexes are passed to other threads so that they can be written to disk. These components are written in the C programming language. search: The search components provide a means to find relevant flows and the associated packets. A search query is parsed and represented as a search tree. Various search commands, written in C, are then used resolve this tree into a set of search results. The tree generation and search execution management components are written in python. interface: The PcapDB web interface is written in Python on the Django framework. It provides a series of pages, API's, and asynchronous tasks that allow the user to manage the capture system, perform searches, and retrieve results. Web page components are written in HTML,CSS and Javascript.

Ferrell, Paul↗

Software Development and Testing Support for the Avionics Systems Telemetry Tool Suite

The Customer Avionics Interface Development and Analysis (CAIDA) team helps to provide modeling and simulation software for the verification of the Launch Control System (LCS). With a new iteration of telemetry tools being developed, extensive work must be done to ensure features are implemented in an efficient manner. The authors worked to develop new functionalities in the telemetry tools, update documentation, and perform various tests on the CAIDA Advanced Telemetry Tool (CATT). This was accomplished with Python through built-in library frameworks. In addition, work needed to be performed to set up a training document for new engineers and interns joining the team in the future. The outcome of this internship was the completion of several new features, unit and functional tests on CATT, thorough documentation, and a developer’s guide to programming under CAIDA.

Negron-Martinez, Antonio Jose↗

PcapDB: Search Optimized Packet Capture, Version 0.1.0.0

PcapDB is a packet capture system designed to optimize the captured data for fast search in the typical (network incident response) use case. The technology involved in this software has been submitted via the IDEAS system and has been filed as a provisional patent. It includes the following primary components: capture: The capture component utilizes existing capture libraries to retrieve packets from network interfaces. Once retrieved the packets are passed to additional threads for sorting into flows and indexing. The sorted flows and indexes are passed to other threads so that they can be written to disk. These components are written in the C programming language. search: The search components provide a means to find relevant flows and the associated packets. A search query is parsed and represented as a search tree. Various search commands, written in C, are then used resolve this tree into a set of search results. The tree generation and search execution management components are written in python. interface: The PcapDB web interface is written in Python on the Django framework. It provides a series of pages, API's, and asynchronous tasks that allow the user to manage the capture system, perform searches, and retrieve results. Web page components are written in HTML,CSS and Javascript.

Ferrell, Paul↗

Data-parallel Python for High Energy Physics Analyses

In this paper, we explore features available in Python which are useful for data reduction tasks in High Energy Physics (HEP). Highlevel abstractions in Python are convenient for implementing data reduction tasks. However, in order for such abstractions to be practical, the efficiency of their performance must also be high. Because the data sets we process are typically large, we care about both I/O performance and in-memory processing speed. In particular, we evaluate the use of data-parallel programming, using MPI and numpy, to process a large experimental data set (42 TiB) stored in an HDF5 file. We measure the speed of processing of the data, distinguishing between the time spent reading data and the time spent processing the data in memory, and demonstrate the scalability of both, using up to 1200 KNL nodes (76800 cores) on Cori at NERSC

Paterno, Marc↗