SEARCH · Engineering Papers

Results for “Software Library”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58

All sources NASA NTRS DOE OSTI

DOE OSTI2021

PyKrev: A Python Library for the Analysis of Complex Mixture FT-MS Data

In this study, we present PyKrev, a Python library for the analysis of complex mixture Fourier transform mass spectrometry (FT-MS) data. PyKrev is a comprehensive suite of tools for analysis and visualization of FT-MS data after formula assignment has been performed. These comprise formula manipulation and calculation of chemical properties, intersection analysis between multiple lists of formulas, calculation of chemical diversity, assignment of compound classes to formulas, multivariate analysis, and a variety of visualization tools producing van Krevelen diagrams, class histograms, PCA score, and loading plots, biplots, scree plots, and UpSet plots. The library is showcased through analysis of hot water green tea extracts and Scotch whisky FT-ion cyclotron resonance-MS data sets. PyKrev addresses the lack of a single, cohesive toolset for researchers to perform FT-MS analysis in the Python programming environment encompassing the most recent data analysis techniques used in the field.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Metagenome-assembled genome extraction and analysis from microbiomes using KBase

Uncultivated Bacteria and Archaea account for the vast majority of species on Earth, but obtaining their genomes directly from the environment, using shotgun sequencing, has only become possible recently. In order to realize the hope of capturing Earth’s microbial genetic complement and to facilitate the investigation of the functional roles of specific lineages in a given ecosystem, technologies that accelerate the recovery of high-quality genomes are necessary. We present a series of analysis steps and data products for the extraction of high-quality metagenome-assembled genomes (MAGs) from microbiomes using the U.S. Department of Energy Systems Biology Knowledgebase (KBase) platform (http://www.kbase.us/). Overall, these steps take about a day to obtain extracted genomes when starting from smaller environmental shotgun read libraries, or up to about a week from larger libraries. In KBase, the process is end-to-end, allowing a user to go from the initial sequencing reads all the way through to MAGs, which can then be analyzed with other KBase capabilities such as phylogenetic placement, functional assignment, metabolic modeling, pangenome functional profiling, RNA-Seq and others. While portions of such capabilities are available individually from other resources, the combination of the intuitive usability, data interoperability and integration of tools in a freely available computational resource makes KBase a powerful platform for obtaining MAGs from microbiomes. While this workflow offers tools for each of the key steps in the genome extraction process, it also provides a scaffold that can be easily extended with additional MAG recovery and analysis tools, via the KBase software development kit (SDK).

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Vectorization of CMSSW offline software

The CMS experiment has been utilizing vectorization, or SIMD, in parts of its data processing applications for over a decade. On x86 platforms the vectorization level is still SSE3. In the past attempts to use wider vector instruction sets such as AVX or AVX-512 have, in practice, not resulted in improvements in the overall event processing throughput, because the CPUs scale down their frequency when processing AVX instructions. In addition, a notable part of the global pool of CMS’ resources has been old systems either not supporting AVX, or where the CPU frequency downscaling impacts all cores of the CPU. CMS has nevertheless continued to vectorize more of its application code, and in this work we review profiling methods we have found effective to find out pieces of code that would benefit from vectorization, and techniques to transform those codes such that the GCC compiler is able to auto-vectorize those codes. The build system used for CMSSW, Scram, has also been enha nced to be able to build code for multiple CPU microarchitectures such that the shared libraries of desired microarchitecture level can be loaded based on the CPU of the system. This multi-microarchitecture setup is invisible to the workflow management system, which makes its deployment straightforward. We describe in detail how this multi-microarchitecture build is set up, and measure the impact of using wider vector units than SSE3 on the event processing throughput of CMS applications such as simulation and reconstruction on recent x86 CPUs.

Gartung, Patrick↗

DOE OSTI2018

Low-noise, radiation tolerant SiCpms

Over the last 10 years, our R&D effort has resulted in a steady progress on improving Geiger detector array technology performance, verifying their reliability and developing three cell layout libraries containing more than 2200 primitive cells for Si and SiC Geiger based single-photon detectors and silicon ROIC array fabrication. Awards from DOE, NASA and DOD, as wells as commercial R&D orders supported much progress, relevant to this program. The expertise gained and the technologies developed or qualified during these programs provided the design database and expertise to carry out the development of this new SiC photomultiplier (SiCpm) technology, successful fabrication of SiCpm detectors, and evaluation of their radiation hardness. After irradiation with protons and neutrons at 1015cm-2 and 1014cm-2 respectively, SiCpms proved radiation hardness, exceeding the expected radiation tolerance The defects induced by irradiation were mainly point defects. A new improved DLTS method (fDLTS – full wave DLTS) hardware and data processing software specific to investigating radiation damage in SiC was developed. Further work is needed to better characterize and control defects induced by charged particles. We recommend further funding of radiation hardened technology development and validation; qualification of device-specific characterization methods like f-DLTS, development of more methods to evaluate defects in SiC, more active involvement of the national labs in evaluating the SiCpm devices, and DOE support of technology transfer to a qualified foundry.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2021

Method for Rapid Modeling of Distortion in Laser Powder Bed Fusion Metal Additive Manufacturing Parts

The simulation and modeling of part-level distortion and residual stress in diverse metal additive manufacturing (AM) geometries has great potential to enable the rapid adoption of this technology in engineering design. Moreover, the use of additive manufacturing component libraries (CLs) offer a computationally efficient means of quantifying these part-level defects resultant from AM processing. We report on how the individual simulations of simple shapes, potential entries in a CL, can be superimposed to provide an indication of distortion and residual stresses in complex geometries. Laser powder bed fusion AM was used to construct test geometries of varied shapes and their combinations in the form of CLs in an effort to characterize location-dependent and feature-dependent distortion distributions. Blue light scanning was used to experimentally measure 3D distortions in order to investigate the interaction between the component shapes and local boundary conditions. Overall, part-level distortions were highly dependent on test component geometry, local boundary conditions, and shape combination. Commercial finite element software was used to verify experimental trends and to make predictions of distortion. The use of CLs resulted in over 20 times savings in computational cost while reproducing overall trends in distortion for test geometry assemblies. Therefore, it is anticipated that the use of CLs for L-PBF AM geometries has demonstrated potential to facilitate efficient simulations of full component AM assemblies, thereby reducing the need for costly trial-and-error-type experimental analysis.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Keras2c: A library for converting Keras neural networks to real-time compatible C

With the growth of machine learning models and neural networks in measurement and control systems comes the need to deploy these models in a way that is compatible with existing systems. Existing options for deploying neural networks either introduce very high latency, require expensive and time consuming work to integrate into existing code bases, or only support a very limited subset of model types. We have therefore developed a new method called Keras2c, which is a simple library for converting Keras/TensorFlow neural network models into real-time compatible C code. It supports a wide range of Keras layers and model types including multidimensional convolutions, recurrent layers, multi-input/output models, and shared layers. Keras2c re-implements the core components of Keras/TensorFlow required for predictive forward passes through neural networks in pure C, relying only on standard library functions considered safe for real-time use. The core functionality consists of ~ 1500 lines of code, making it lightweight and easy to integrate into existing codebases. Keras2c has been successfully tested in experiments and is currently in use on the plasma control system at the DIII-D National Fusion Facility at General Atomics in San Diego.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2015

Reconstructing householder vectors from Tall-Skinny QR

The Tall-Skinny QR (TSQR) algorithm is more communication efficient than the standard Householder algorithm for QR decomposition of matrices with many more rows than columns. However, TSQR produces a different representation of the orthogonal factor and therefore requires more software development to support the new representation. Further, implicitly applying the orthogonal factor to the trailing matrix in the context of factoring a square matrix is more complicated and costly than with the Householder representation. We show how to perform TSQR and then reconstruct the Householder vector representation with the same asymptotic communication efficiency and little extra computational cost. We demonstrate the high performance and numerical stability of this algorithm both theoretically and empirically. The new Householder reconstruction algorithm allows us to design more efficient parallel QR algorithms, with significantly lower latency cost compared to Householder QR and lower bandwidth and latency costs compared with Communication-Avoiding QR (CAQR) algorithm. Experiments on supercomputers demonstrate the benefits of the communication cost improvements: in particular, our experiments show substantial improvements over tuned library implementations for tall-and-skinny matrices. Furthermore, we also provide algorithmic improvements to the Householder QR and CAQR algorithms, and we investigate several alternatives to the Householder reconstruction algorithm that sacrifice guarantees on numerical stability in some cases in order to obtain higher performance.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

A general approach to seismic inversion with automatic differentiation

Imaging Earth structure or seismic sources from seismic data involves minimizing a target misfit function, and is commonly solved through gradient-based optimization. The adjoint-state method has been developed to compute the gradient efficiently; however, its implementation can be time-consuming and difficult. We develop a general seismic inversion framework to calculate gradients using reverse-mode automatic differentiation. The central idea is that adjoint-state methods and reverse-mode automatic differentiation are mathematically equivalent. Here, the mapping between numerical PDE simulation and deep learning allows us to build a seismic inverse modeling library, ADSeismic, based on deep learning frameworks, which supports high performance reverse-mode automatic differentiation on CPUs and GPUs. We demonstrate the performance of ADSeismic on inverse problems related to velocity model estimation, rupture imaging, earthquake location, and source time function retrieval. ADSeismic has the potential to solve a wide variety of inverse modeling applications within a unified framework.

58 GEOSCIENCES↗

DOE OSTI2013

Stochastic PArallel Rarefied-gas Time-accurate Analyzer

The SPARTA package is software for simulating low-density fluids via the Direct Simulation Monte Carlo (DSMC) method, which is a particle-based method for tracking particle trajectories and collisions as a model of a multi-species gas. The main component of SPARTA is a simulation code which allows the user to specify a simulation domain, populate it with particles, embed triangulated surfaces as boundary conditions for the flow, overlay a grid for finding pairs of collision partners, and evolve the system in time via explicit timestepping. The package also includes various pre- and post-processing tools, useful for setting up simulations and analyzing the results. The simulation code runs either in serial on a single processor or desktop machine, or can be run in parallel using the MPI message-passing library, to enable faster performance on large problems.

Plimpton, Steve↗

DOE OSTI2024

Control And Optimization Modular Modeling Application For Nuclear Deployment

The purpose of the COMMAND code is to provide a flexible, scalable tool for use in developing, integrating, and testing the technologies necessary for achieving autonomous operations of advanced nuclear reactors. The code enables users to efficiently implement custom simulations and experiments by combining key methods from different software modules. These modules are focused on: modeling and simulation tools, such as nuclear simulation tools used for high-fidelity modeling (e.g., Reactor Excursion and Leak Analysis Program [RELAP5-3D] and Monte Carlo N-Particle [MCNP]); machine learning and optimization tools (e.g., anomaly detection and data-driven modeling techniques); advanced control in its digital, high-performance, and supervisory control forms (e.g., proportional integral derivative (PID) control and model predictive control (MPC); and integration with hardware through industrial communication protocols. To ensure flexibility and scalability, COMMAND was designed to be both modular—the software “pieces” all inherit from generic building blocks and can be combined and connected to create complicated simulations—and high performing—designed for parallel processing, enabling simulations and experiments to take advantage of multi-core computers, servers, and nodes. The code is written in the Python programming language due to the language's popularity, active community, and open-source and cross-platform nature. Maintaining consistency with other simulation tools used within the nuclear energy community, users implement simulations and experiments through text input files, which define components, parameters, connections, etc., through lines of text. Given that COMMAND is written in Python, these input files are native Python scripts, and so use the standard Python structure and formatting. This also enables users to take advantage of Python's extensive package library to develop custom capabilities for their specific use cases.

Faber, Jacob [Idaho National Laboratory (INL), Ida↗

DOE OSTI2014

Stochastic PArallel Rarefied-gas Time-accurate Analyzer

Michael Gallis, Steve Plimpton↗

DOE OSTI2021

Jas4pp — A data-analysis framework for physics and detector studies

This paper describes the Jas4pp framework for exploring physics cases and for detector-performance studies of future particle collision experiments. Jas4pp is a multi-platform Java program for numeric calculations, scientific visualization in 2D and 3D, storing data in various file formats and displaying collision events and detector geometries. It also includes complex data-analysis algorithms for function minimization, regression analysis, event reconstruction (such as jet reconstruction), limit settings and other libraries widely used in particle physics. The framework can be used with several scripting languages, such as Python/Jython, Groovy and JShell. Several benchmark tests discussed in the paper illustrate significant improvements in the performance of the Groovy and JShell scripting languages compared to the standard Python implementation in C. Furthermore, the improvements for numeric computations in Java are attributed to recent enhancements in the Java Virtual Machine.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Quantitative Uncertainty Analysis for Diffraction

Current software uses a least squares statistical method. For each model parameter, current software yields a point estimate and estimated standard error for each point estimate. In contrast, we have employed a Bayesian approach with sampling algorithm to provide a full posterior probability density function on each model parameter. This yields a different type of solution (probabilistic) with better fidelity. The Quantitative Uncertainty Analysis for Diffraction (QUAD) package returns a matrix of affect model parameters as a function of iteration through the MCMC analysis. QUAD is a Python code that allows for analysis of X-ray and neutron diffraction data to infer the structure of materials with quantifiable certainty. The code uses Bayesian statistics and Markov chain sampling algorithms to create posterior probability distributions on all material structure parameters that are modeled by a researcher. The statistical packages use the open source GSASII project as a library to calculate model diffraction patterns.

Fancher, ChristopherM↗

DOE OSTI2016

Sandbox for Mac Malware v 1.0

This software is an analyzer for automated sandbox analysis of malware on the OS X operating system. It runs inside an OS X virtual machine to collect data about what happens when a given file is opened or run. As of August 2014, there was no sandbox software for Mac OS X malware, as it requires different methods from those used on the Windows OS (which most sandboxes are written for). This software adds OS X analysis capabilities to an existing open-source sandbox, Cuckoo Sandbox (http://cuckoosandbox.org/), which previously only worked for Windows. The analyzer itself can take many different types of files as input: the traditional Mach-O and FAT executables, .app files, zip files, Python scripts, Java archives, and web pages, as well as PDFs and other documents. While the file is running, the analyzer also simulates rudimentary human interaction with clicks and mouse movements in order to bypass the tests some malware use to see if they are being analyzed. The analyzer outputs several different kinds of data: function call traces, network captures, screenshots, and all created and modified files. This work also includes a static analysis Cuckoo module for Mach-O binary files. It extracts file structures, code library imports and exports, and signatures. This data can be used along with the analyzer results to create signatures for malware.

Walkup, Elizabeth↗

DOE OSTI2016

NDMAS implementation and data qualification for accident tolerant fuel experiments

In order to meet quality and regulatory requirements associated with the U.S. Department of Energy Accident Tolerant Fuels (ATF) experiments, all data and documentation associated with ATF component fabrication, characterization, traceability, testing and post irradiation examination are being managed using the Nuclear Data Management and Analysis System (NDMAS). NDMAS melds commercial off-the-shelf software to provide the full range of needed functionality. The core component of NDMAS is the SAS Enterprise Business Intelligence software, which provides data manipulation, statistical and graphical capabilities, and delivery of live content to web pages. Content delivery to both Idaho National Laboratory researchers and external users via a web interface is managed using SharePoint, which is access-controlled. Additionally, scripts, macros, and commercial wrappers may be used or written by the NDMAS team to control interfaces between SAS and the structured query language server, where much of the data reside. ATF experiment data and documentation stored in the NDMAS database may be utilized to support Nuclear Regulatory Commission licensing of ATF concepts for commercial Light Water Reactor demonstration. Therefore, Data Qualification Plans are being developed to screen and tag all document and data input for ATR water loop testing, Transient Reactor Testing (TREAT), and post irradiation examination. A project-wide document library has been created in the ATF NDMAS SharePoint Site for general unrestricted document and information access. Restricted libraries have also been created to allow individual customer access to restricted documents. Data from ATR water loop instrumented lead experimentation, TREAT, and post-irradiation examination will be available to program participants on the SharePoint site as they are collected. This offers a unique capability for team collaboration while these tests are being conducted. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2020

toughio v1.2.2

TOUGH is a widely used general purpose numerical simulator designed for fluid and heat flows of multiphase, multicomponent fluid mixtures in porous and fractured media, which has been applied to many real-world problems such as underground geological storage, geothermal reservoir engineering or nuclear waste disposal, to name a few. `toughio` is a lightweight, object-oriented and vectorized Python library that aims to provide user-friendly routines to facilitate pre- and post-processing of a TOUGH simulation. Although its primary target is the latest version TOUGH3, `toughio` can also be used to read/write TOUGH2 input/output files. Currently, to the best of our knowledge, only PyTOUGH offers an exhaustive number of features to carry out a complete TOUGH simulation using a scripting language. `toughio` and PyTOUGH share the same objectives, yet with different approaches and philosophies. On the one hand, a PyTOUGH mesh is represented as a MULGRAPH geometry where elements can be unstructured horizontally but only layered vertically. It also checks that the model parameters are valid to ensure that a TOUGH simulation runs smoothly. On the other hand, although it provides basic meshing features, `toughio` mostly relies on common third-party softwares (e.g. Abaqus, FLAC3D, Gmsh, LaGriT) to generate the mesh by importing and converting it to a TOUGH mesh, which also conveniently facilitates the coupling of TOUGH with any other simulator that also supports the same mesh formats. Besides, `toughio` is designed to be just a TOUGH input and output file reader/writer. It assumes that the user knows what he is doing and does not verify the validity of the model parameters (`toughio` does verify the functional arguments).

Luu, Keurfon↗

DOE OSTI2022

Next generation experimental data access at NSLS-II

The NSLS-II network and computing infrastructure has been significantly updated recently. The re-IP process in 2020-2021 enabled the NSLS-II network to be routable to the rest of the BNL campus. Then, standardization of the operating systems and deployment procedures helped to deliver a consistent environment to workstations and servers used by all NSLS-II beamlines. In particular, the RedHat Enterprise Linux 8 was deployed to 700+ machines using the RedHat Satellite infrastructure management product, and all critical services (IOCs, databases, etc.) were migrated to the new OS. NFS users’ home directories are consistent across all of the machines, which eliminates the need for the individual configuration of the user environment on each host. The standard suite of software packages is available to the beamline staff and users, which includes the system packages (deployed via RPM) as well as the conda environments for data acquisition and analysis. Security measures were implemented to comply with the industry standards, which include multi-factor authentication (using Duo), secure screen lock for the beamline machines, and advanced access control to the experimental data that is stored in shared central storage available on all hosts. These major enhancements facilitated sharing the experimental data (currently for a number of selected beamlines, with a plan to extend it to the whole facility in the nearest future) with the users via an externally facing JupyterHub instance. The beamlines keep using the Bluesky data acquisition framework to orchestrate their experiments, and the new infrastructure enabled them to use a next-generation data access library called tiled.

36 MATERIALS SCIENCE↗

DOE OSTI2016

Exploring versioned distributed arrays for resilience in scientific applications: Global view resilience

Exascale studies project reliability challenges for future HPC systems. We present the Global View Resilience (GVR) system, a library for portable resilience. GVR begins with a subset of the Global Arrays interface, and adds new capabilities to create versions, name versions, and compute on version data. Applications can focus versioning where and when it is most productive, and customize for each application structure independently. This control is portable, and its embedding in application source makes it natural to express and easy to maintain. The ability to name multiple versions and “partially materialize” them efficiently makes ambitious forward-recovery based on “data slices” across versions or data structures both easy to express and efficient. Using several large applications (OpenMC, preconditioned conjugate gradient (PCG) solver, ddcMD, and Chombo), we evaluate the programming effort to add resilience. The required changes are small (< 2% lines of code (LOC)), localized and machine-independent, and perhaps most important, require no software architecture changes. We also measure the overhead of adding GVR versioning and show that overheads < 2% are generally achieved. This overhead suggests that GVR can be implemented in large-scale codes and support portable error recovery with modest investment and runtime impact. Our results are drawn from both IBM BG/Q and Cray XC30 experiments, demonstrating portability. We also present two case studies of flexible error recovery, illustrating how GVR can be used for multi-version rollback recovery, and several different forward-recovery schemes. GVR’s multi-version enables applications to survive latent errors (silent data corruption) with significant detection latency, and forward recovery can make that recovery extremely efficient. Lastly, our results suggest that GVR is scalable, portable, and efficient. GVR interfaces are flexible, supporting a variety of recovery schemes, and altogether GVR embodies a gentle-slope path to tolerate growing error rates in future extreme-scale systems.

97 MATHEMATICS AND COMPUTING↗