Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Github”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

ATLAS (Automatic Tool for Local Assembly Structures) - A Comprehensive Infrastructure for Assembly, Annotation, and Genomic Binning of Metagenomic and Metaranscripomic Data

ATLAS (Automatic Tool for Local Assembly Structures) is a comprehensive multiomics data analysis pipeline that is massively parallel and scalable. ATLAS contains a modular analysis pipeline for assembly, annotation, quantification and genome binning of metagenomics and metatranscriptomics data and a framework for reference metaproteomic database construction. ATLAS transforms raw sequence data into functional and taxonomic data at the microbial population level and provides genome-centric resolution through genome binning. ATLAS provides robust taxonomy based on majority voting of protein coding open reading frames rolled-up at the contig level using modified lowest common ancestor (LCA) analysis. ATLAS provides robust taxonomy based on majority voting of protein coding open reading frames rolled-up at the contig level using modified lowest common ancestor (LCA) analysis. ATLAS is user-friendly, easy install through bioconda maintained as open-source on GitHub, and is implemented in Snakemake for modular customizable workflows.

genome binning↗
DOE OSTI2022

WaterTAP3 Model Results for NAWI's Baseline Analyses

Description: This folder contains the results for the WaterTAP3 model that was used for the eight NAWI (National Alliance for Water Innovation) baseline studies published in the Environmental Science and Technology special issue: Technology Baselines and Innovation Priorities for Water Treatment and Supply. The data structure and content are described in a README.txt file. For more details on how to use the data and interpret the results please refer to the model documentation and GitHub site linked in the submission.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

DISCo-microbe: design of an identifiable synthetic community of microbes

Microbiomes are extremely important for their host organisms, providing many vital functions and extending their hosts’ phenotypes. Natural studies of host-associated microbiomes can be difficult to interpret due to the high complexity of microbial communities, which hinders our ability to track and identify individual members along with the many factors that structure or perturb those communities. For this reason, researchers have turned to synthetic or constructed communities in which the identities of all members are known. However, due to the lack of tracking methods and the difficulty of creating a more diverse and identifiable community that can be distinguished through next-generation sequencing, most such in vivo studies have used only a few strains. To address this issue, we developed DISCo-microbe, a program for the design of an identifiable synthetic community of microbes for use in in vivo experimentation. The program is composed of two modules; (1) create, which allows the user to generate a highly diverse community list from an input DNA sequence alignment using a custom nucleotide distance algorithm, and (2) subsample, which subsamples the community list to either represent a number of grouping variables, including taxonomic proportions, or to reach a user-specified maximum number of community members. As an example, we demonstrate the generation of a synthetic microbial community that can be distinguished through amplicon sequencing. The synthetic microbial community in this example consisted of 2,122 members from a starting DNA sequence alignment of 10,000 16S rRNA sequences from the Ribosomal Database Project. We generated simulated Illumina sequencing data from the constructed community and demonstrate that DISCo-microbe is capable of designing diverse communities with members distinguishable by amplicon sequencing. Using the simulated data we were able to recover sequences from between 97–100% of community members using two different post-processing workflows. Furthermore, 97–99% of sequences were assigned to a community member with zero sequences being misidentified. We then subsampled the community list using taxonomic proportions to mimic a natural plant host–associated microbiome, ultimately yielding a diverse community of 784 members. DISCo-microbe can create a highly diverse community list of microbes that can be distinguished through 16S rRNA gene sequencing, and has the ability to subsample (i.e., design) the community for the desired number of members and taxonomic proportions. Although developed for bacteria, the program allows for any alignment input from any taxonomic group, making it broadly applicable. The software and data are freely available from GitHub and Python Package Index (PYPI).

16S rRNA↗
DOE OSTI2022

Exploring MDSplus data-acquisition software and custom devices

MDSplus is a software tool designed for data acquisition, storage, and analysis of complex scientific experiments. Over the years, MDSplus has primarily been used for data management for fusion experiments. This paper demonstrates that MDSplus can be used for a much wider variety of systems and experiments. We present a step-by-step tutorial describing how to create a simple experiment, manage the data, and analyze it using MDSplus and Python. To this end, a custom example device was developed to be used as the data source. This device was built on an opensource electronic hardware platform, and it consists of a microcontroller and two sensors. We read data from these sensors, store it in MDSplus, and use JupyterLab to visualize and process it. This project and code demo are available on the GitHub site at this URL: https://github.com/santorofer/MDSplusAndCustomeDevices

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Y1000 gene family

All relative analysis data and codes used in this study are available on the Figshare data repository. You can also find the codes on the Github repository at https://github.com/vnuii/Y1000_gene_family.

budding yeast↗
DOE OSTI2024

Generic and ML Workloads in an HPC Datacenter: Node Energy, Job Failures, and Node-Job Analysis

HPC datacenters offer a backbone to the modern digital society. Increasingly, they run Machine Learning (ML) jobs next to generic, compute-intensive workloads, supporting science, business, and other decision-making processes. However, understanding how ML jobs impact the operation of HPC datacenters, relative to generic jobs, remains desirable but understudied. In this work, we leverage long-term operational data, collected from a national-scale production HPC datacenter, and statistically compare how ML and generic jobs can impact the performance, failures, resource utilization, and energy consumption of HPC datacenters. Our study provides key insights, e.g., ML-related power usage causes GPU nodes to run into temperature limitations, median/mean runtime and failure rates are higher for ML jobs than for generic jobs, both ML and generic jobs exhibit highly variable arrival processes and resource demands, significant amounts of energy are spent on unsuccessfully terminating jobs, and concurrent jobs tend to terminate in the same state. We open-source our cleaned-up data traces on Zenodo (https://doi. org/10.5281/zenodo.13685426), and provide our analysis toolkit as software hosted on GitHub (https://github.com/atlarge-research/2024-icpads-hpc-workload-characterization). This study offers multiple benefits for data center administrators, who can improve operational efficiency, and for researchers, who can further improve system designs, scheduling techniques, etc.

crossanalysis↗
DOE OSTI2016

Scraper

This application is a set of utility scripts for obtaining the public data available via the GitHub REST API.

Source record↗
DOE OSTI2016

CIVET: Continuous Integration, Verification, Enhancement, and Testing

A Git server (GitHub, GitLab, BitBucket) sends event notifications to the Civet server. These are either a " Pull Request" or a "Push" notification. Civet then checks the database to determine what tests need to be run and marks them as ready to run. Civet clients, running on dedicated machines, query the server for available jobs that are ready to run. When a client gets a job it executes the scripts attached to the job and report back to the server the output and exit status. When the client updates the server, the server will also update the Git server with the result of the job, as well as updating the main web page.

Alger, Brian↗
DOE OSTI2019

Cosmic Microwave Background Anisotropy Numerical Solution (CMBAns) I: An Introduction to $C_l$ Calculation

Cosmological Boltzmann codes are often used by researchers for calculating the CMB angular power spectra from different theoretical models, for cosmological parameter estimation, etc. Therefore, the accuracy of a Boltzmann code is of utmost importance. Different Markov Chain Monte Carlo based parameter estimation algorithms typically require 10^3 - 10^4 iterations of Boltzmann code. This makes the time complexity of such codes another critical factor. In the last two decades, several Boltzmann packages, such as CMBFAST, CAMB, CMBEasy, CLASS etc., have been developed. In this paper, we present a new cosmological Boltzmann code, CMBAns, that can be used for accurate calculation of the CMB power spectrum. At present, CMBAns is developed for a flat background matrix. It is mostly written in the C language. However, we borrowed the concept of class from C++. This gives researchers the flexibility to develop their own independent package based on CMBAns, without an in-depth understanding of the source code. We also develop multiple stand-alone facilities which can be directly compiled and run on a given parameter set. In this paper, we discuss all the mathematical formulation, approximation schemes, integration methods etc., that are used in CMBAns. The package will be made available through github for public use in the near future.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

The ESIF-HPC-2 Benchmark Suite: Preprint

We describe the development of the ESIF-HPC-2 benchmark suite, a collection of kernel and application benchmark codes for measuring computational and I/O performance from single nodes to full HPC systems that was used for acceptance testing in our recent HPC procurement. The configurations of the benchmarks used for our system is presented. We also describe a set of 'dimensions' that can be used to classify benchmarks and assess coverage of a suite systematically. The collection is offered cost-free as a GitHub repository for general usage and further development.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

OpenCGRA: An Open-Source Unified Framework for Modeling,Testing, and Evaluating CGRAs

Coarse-grained reconfigurable arrays (CGRAs),loosely defined as arrays of functional units (e.g, adder, sub-tractor, multiplier, divider, or larger multi-operation units, butsmaller than a general-purpose core) interconnected through aNetwork-on-Chip, provide higher flexibility than domain-specificASIC accelerators while offering increased hardware efficiencywith respect to fine-grained reconfigurable devices, such as FieldProgrammable Gate Arrays (FPGAs). The fast evolving fieldsof machine learning and edge computing, which are seeing acontinuous flow of novel algorithms and larger models, makeCGRAs ideal target architectures to allow domain specializationwithout loosing too much generality. They also generally offerquicker and more effective reconfigurability than FPGAs, po-tentially allowing adaptation during actual algorithm execution,and implement a dataflow programming paradigm that adaptswell to these emerging workloads. Designing and generating aCGRA, however, still requires to define the type and number ofthe specific functional units, implement their interconnect andthe network topology, and perform its simulation and validation,given a variety of workloads of interest.In this paper, we propose OpenCGRA, a Python-based unifiedframework that integrates generation, modeling, testing and eval-uation for CGRAs. OpenCGRA is the first open-source integratedframework able to support the full top-to-bottom design flow forspecializing and implementing CGRAs: modeling at different ab-straction levels (functional level, cycle level, register-transfer level),generation, simulation, testing at different granularities (unit test-ing, integration testing, property-based testing), and characteriza-tion (area, power, and timing). OpenCGRAs will be made availableon GitHub.

CGRA, synthesis↗
DOE OSTI2021

Open-Source PSCAD Grid-Following and Grid-Forming Inverters and a Benchmark for Zero-Inertia Power System Simulations: Preprint

This paper presents open-source, flexible, and easily-scalable models of grid following and grid forming inverters for the PSCAD software platform. The models are intended for system integration studies, particularly stability analyses of power systems with high penetration of inverter-based generation. To verify the model functionality, they are implemented in a IEEE9-bus system in a zero-inertia operational scenario of 100% inverter-based generation. The models have been made available open source at the PyPSCAD NREL GitHub page.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2021

DESC DC2 Data Release Note

In preparation for cosmological analyses of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), the LSST Dark Energy Science Collaboration (LSST DESC) has created a 300 deg$^2$ simulated survey as part of an effort called Data Challenge 2 (DC2). The DC2 simulated sky survey, in six optical bands with observations following a reference LSST observing cadence, was processed with the LSST Science Pipelines (19.0.0). In this Note, we describe the public data release of the resulting object catalogs for the coadded images of five years of simulated observations along with associated truth catalogs. We include a brief description of the major features of the available data sets. To enable convenient access to the data products, we have developed a web portal connected to Globus data services. We describe how to access the data and provide example Jupyter Notebooks in Python to aid first interactions with the data. We welcome feedback and questions about the data release via a GitHub repository.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Updates to the Instant Online PV LCOE Calculator Tool

Levelized cost of energy (LCOE) is an important metric for techno-economic analysis of PV modules, systems, and performance. An NREL team within DuraMAT has developed a simplified PV-specific LCOE calculator as an alternative to the NREL System Advisor Model (SAM) for applications that do not have the degree of detail required by SAM. This webinar will review recent data and new features that have been added to the simplified calculator this year, including breakeven buttons, commercial system options, and inverter loading ratios. It will also review the structure of the source code on GitHub, how to edit a local version of the calculator, and walk through some example use cases for the calculator.

calculator↗
DOE OSTI2021

Introducing PRIMRE’s MRE Software Knowledge Hub

This paper focuses on the role of the Marine Renewable Energy (MRE) Software Knowledge Hub on the Portal and Repository for Information on Marine Renewable Energy (PRIMRE). The MRE Software Knowledge Hub provides online services for MRE software users and developers, and seeks to develop assessments and recommendations for improving MRE software in the future. Online software discovery platforms, known as the Code Hub and the Code Catalog, are provided. The Code Hub is a collection of open-source MRE software that includes a landing page with search functionality, linked to files hosted on the MRE Code Hub GitHub organization. The Code Catalog is a searchable online platform for discovery of useful (open-source or commercial) software packages, tools, codes, and other software products. To gather information about the existing MRE software landscape, a software survey is being performed, the preliminary results of which are presented herein. Initially, the data collected in the MRE software survey will be used to populate the MRE Software knowledge hub on PRIMRE, and future work will use data from the survey to perform a gap analysis and develop a vision for future software development. Additionally, as one of PRIMRE’s roles is to support development of MRE software within project partners, a silo of knowledge relating to best practices has been gathered.

marine, energy, open source software, MHKiT, MRE C↗
DOE OSTI2021

Introducing PRIMRE's MRE Software Knowledge Hub (February 2021)

This paper focuses on the role of the Marine Renewable Energy (MRE) Software Knowledge Hub on the Portal and Repository for Information on Marine Renewable Energy (PRIMRE). The MRE Software Knowledge Hub provides online services for MRE software users and developers, and seeks to develop assessments and recommendations for improving MRE software in the future. Online software discovery platforms, known as the Code Hub and the Code Catalog, are provided. The Code Hub is a collection of open-source MRE software that includes a landing page with search functionality, linked to files hosted on the MRE Code Hub GitHub organization. The Code Catalog is a searchable online platform for discovery of useful (open-source or commercial) software packages, tools, codes, and other software products. To gather information about the existing MRE software landscape, a software survey is being performed, the preliminary results of which are presented herein. Initially, the data collected in the MRE software survey will be used to populate the MRE Software knowledge hub on PRIMRE, and future work will use data from the survey to perform a gap analysis and develop a vision for future software development. Additionally, as one of PRIMRE's roles is to support development of MRE software within project partners, a silo of knowledge relating to best practices has been gathered. An early draft of new guidance developed from this knowledge is presented.

gap analysis↗
DOE OSTI2022

Developing an Interactive Landscape for Mobility Resources: Preprint

As the world continues to be increasingly driven by data, the ways researchers and professionals sort and collect this data is critical. In the world of mobility data, new levels of data from public transportation systems, location services, and other means are being lost due to how little organization exists. Much of the data is proprietary, and there are few if any de jure or even de facto standards connecting data. There is also little knowledge about the gaps that exist in the data. In this project, we created an interactive landscape where mobility resources are categorized and organized in an easy to use, living document. We made this landscape with open-source code from the CNCF Cloud Native Landscape and repurposed it to the mobility data's needs. Additionally, unlike previous sources that organize mobility data, this document can be updated through GitHub by those in the field to keep its sources relevant. Following the creation of a beta version of the landscape, we conducted several interviews with industry researchers and professionals to ensure the landscape would be useful. The result is an online hub where mobility researchers and resource creators can easily access research and collaborate.

ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Pele: An Exascale-Ready Suite of Combustion Codes

High fidelity simulations of realistic combustion devices are extremely demanding computationally because of the requirements to capture complex fuel chemical decomposition, its intricate interactions with turbulent, often multiphase, flows, and the wide separation of space and time scales between the thin flame and the device boundaries. Software required to carry out such computations tends to be extremely complex, particularly when designed to exploit hardware accelerators, and can be difficult to port and maintain. We present Pele, a performance portable suite of tools for the simulation of combustion systems, including codes to evolve reactive multiphase configurations in the low Mach number and compressible flow regimes, along with a set of inter-compatible post processing and in situ analysis tools. The Pele suite of tools is built on top of the AMReX framework for block-structured adaptive mesh refinement, which provides efficient data structures and algorithms that enable the development of a wide variety of efficient mesh and particle based PDE integration schemes. A hierarchical MPI+X parallelism scheme supports CPU-only and accelerated architectures, where X can be OpenMP, CUDA, and HIP based approaches for intra-node computational work distribution. The algorithms and data structures underlying the Pele simulation and analysis tools are highly scalable and performant across a wide variety of high-performance computing platforms, including DOEs newest exascale-class machines, Frontier and Aurora. The simulation and analysis tools are fully documented and freely distributed as open source via GitHub. We present key algorithmic and software challenges, solution strategies, performance and resulting set of capabilities.

AMReX↗