webspinner
Python utilities for working with data source types used by NREL's dsgrid (Demand-Side Grid Model) project (i.e. AWS, PostgreSQL, and .parquet). https://www.nrel.gov/analysis/dsgrid.html
Hale, Elaine↗
SEARCH · Engineering Papers
Results for “Data Sources”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
BuildStockQuery [SWR-23-58]
BuildStockQuery is a python library designed to simplify and streamline the process of querying massive, terabyte-scale datasets generated by ResStock(TM). ResStock (SWR-19-15) is a U.S. DOE-supported, NREL-built, national residential building energy stock model that enables a new approach to large-scale residential energy analysis across the U.S. by combining large public and private data sources, statistical sampling, detailed sub-hourly building simulations, and high-performance computing. BuildStockQuery offers an intuitive Object-Oriented Programming (OOP) interface to the ResStock output dataset allowing users to easily perform common queries and receive results in familiar pandas DataFrame format, abstracting away the need for complex SQL query. By initializing a query object with the pertinent Athena database and table names, users can easily query for various kinds of insights, for example, timeseries electricity for an end use for a given state grouped by building types.
Adhikari, Rajendra↗
ResStock™ v3.2.0 [SWR-19-15 and SWR-20-07]
The ResStock™ analysis tool was built on NREL's OpenStudio® platform, and is a project geared at modeling existing residential building stocks at national, regional, or local scales with a high-degree of granularity (e.g., one physics-based simulation model for every 200 dwelling units), using the EnergyPlus® simulation engine. Information about ComStock™, a sister tool for modeling the commercial building stock, can be found here: https://www.nrel.gov/buildings/comstock.html This repository contains: Housing characteristics of the U.S. residential building stock, in the form of conditional probability distributions stored as tab-separated value (.tsv) files. Comments at the bottom of each file document data sources and assumptions for each. A library of housing characteristic "options" that translate high-level characteristic parameters into arguments for OpenStudio measures, and which are referenced by the housing characteristic .tsv files and building energy upgrades defined in project definition files Project definition files: v2.3.0 and later: buildstockbatch YML files openable in any text editor v2.2.5 and prior: Project folder openable in PAT Unit-level OpenStudio Measures for automatically constructing OpenStudio Models of each representative dwelling unit model: v3.0.0 and later: OpenStudio-HPXML Measures v2.5.0 and prior: OpenStudio Measures Higher-level OpenStudio Measures for controlling simulation inputs and outputs This repository does not contain software for running ResStock simulations, which can be found as follows: Versions 2.3.0 and later only support the use of buildstockbatch for deploying simulations on high-performance or cloud computing. Version 2.3.0 also removed separate projects for single-family detached and multifamily buildings, in lieu of a combined project_national representing the U.S. residential building stock. See the changelog for more details. Versions 2.2.5 and prior support the use of the publicly available OpenStudio-PAT software as an interface for deploying simulations on cloud computing. Read the documentation for v2.2.5.
Horowitz, Scott↗
Firm Synthesizer and Supply-chain Simulator (SynthFirm) v1.0
SynthFirm is a large-scale agent-based freight demand model which generates a complete synthetic population of firms in the U.S. and the business-to-business commodity flows between them. Using publicly available data sources as inputs, SynthFirm simulates detailed firm and fleet characteristics, commodity production and consumption, formation of supply chains, and selection of shipping modes, all of which are essential drivers of commodity flow at a disaggregate level.
Xu, Xiaodan↗
A Data Processing Pipeline To Extract A Knowledge Graph From Sec Documents For Socio-technical Analysis Of Critical Infrastructure Influence
The code is written in Python and consists of the following pipeline that is implemented in Apache Airflow. This pipeline intends to understand the companies that are directly or indirectly involved with a type of critical infrastructure system at some point in that system's lifecycle. The pipeline takes a configuration file that specifies a list of initial companies to consider, a geographic region of interest (disk) expressed as a latitude/longitude point and distance, and a set of SEC form types from which to extract entities and relations. There are three main components to this pipeline as currently implemented: Social Network Extraction, Critical Infrastructure Network Extraction, and Inference and Fusion. First, Social Network Extraction, implemented as the `organizations_sec` component of the workflow graph queries the SEC EDGAR webservice using the list of initial companies from the configuration file. Given this, it extracts metadata that documents the number of each type of form for the given set of companies and their location. This forms metadata represents a catalog of data sources for the extracted social network knowledge graph. The pipeline then downloads these forms from the website and saves them in a build directory for further processing. These documents are then parsed for entities and relations. Second, the Critical Network Extraction component extracts entities and relations for a critical infrastructure sector. Currently, we focus on Electric Vehicle charging stations and this information is available via the Department of Energy (DOE) database on fueling stations maintained by NREL. Third, the Inference and Fusion component relates the social network graph to the critical infrastructure graph in order to understand the impact of a company within a geographic region. Relations include ownership of the EV Charging Station asset as well as maintenance/ownership of the EV payment networks. The fused network can be represented in many ways and currently we emit a knowledge graph.
Weaver, GabrielA.↗
lanl-ansi/MG-RAVENS
The MG-RAVENS project with the DOE Office of Electricity Microgrid R&D Program is a project to develop a completely free, open-source data exchange standard (API) for the Department of Energy, targeted at software tools related to infrastructure modeling, particularly the modeling of microgrids and electric power distribution systems that are created with funding from the Microgrid R&D Program. This software produces formal definitions of an API, documentation, contains supporting functions for parsing, validating, etc., and will contain examples of workflows enabled by the developed API.
Fobes, David M↗
Accessible Content Optimization for Research Needs (ACORN)
ACORN employs a set of automated processes for informing and/or enforcing defined content schemas to create standardized and highly structured data. Because of its standardized data source, ACORN easily applies computer automation to generate communication assets such as PDFs, Powerpoint presentations, and web pages. Built using the memory-safe Rust programming language, ACORN is portable and accessible for use on any Windows, Mac, or Linux machine.
Wohlgemuth, JasonHoward [Oak Ridge National Labora↗
Firm Synthesizer and Supply-chain Simulator (SynthFirm) v2.0
SynthFirm is a national-scale agent-based freight demand model which generates a complete synthetic population of firms in the U.S. and the business-to-business commodity flows between them. Using publicly available data sources as inputs, SynthFirm simulates detailed firm and fleet characteristics, commodity production and consumption, formation of supply chains, and selection of shipping modes, all of which are essential drivers of commodity flow at a disaggregate level. The SynthFirm 2.0 version includes national commercial vehicle fleet generation, international trade simulation and automized model validation pipeline, which allows seemless deployment across the nation and build a comprehensive freight inventories at national scale or for selected region.
Yang, Hung-Chia [Lawrence Berkeley National Labora↗
Dynamic_Syringe_Pump_Controller
A LabVIEW based programmable user interface for operating ISCO D-series and SyriXus-series syringe pumps that enables advanced user operations and integration with auxiliary data sources.
Frash, Luke↗
Dose-finding studies in drug development for rare genetic diseases
The small patient populations inherent to rare genetic diseases present many challenges to the traditional drug development paradigm. One major challenge is generating sufficient data in early phase studies to inform dose selection for later phase studies and dose optimization for clinical use of the drug. However, optimizing the benefit-risk profile of drugs through appropriate dose selection during drug development is critical for all drugs, including those being developed to treat rare diseases. Recognizing the challenges of conducting dose finding studies in rare disease populations and the importance of dose selection and optimization for successful drug development, we assessed the dose-finding studies and analyses conducted for drugs recently approved for rare genetic diseases. Of the 40 marketing applications for new molecular entity (NME) drugs and biologics approved by the United States Food and Drug Administration for rare genetic diseases from 2015 to 2020, 21 (53%) of the development programs conducted at least one dedicated dose-finding study. In addition, the majority of drug development programs conducted clinical studies in healthy subjects and included population pharmacokinetic and exposure–response analyses; some programs also conducted clinical studies in patient populations other than the disease for which the drug was initially approved. The majority of primary endpoints utilized in dedicated dose-finding studies were biomarkers, and the primary endpoint of the safety and efficacy study matched the primary endpoint used in the dose finding study in 9 of 13 (69%) drug development programs where primary study endpoints were assessed. Our study showed that NME drug development programs for rare genetic diseases utilize multiple data sources for dosing information, including studies in healthy subjects, population pharmacokinetic analyses, and exposure–response analyses. In addition, our results indicate that biomarkers play a key role in dose-finding studies for rare genetic disease drug development programs. Our findings highlight the need to develop study designs and methods to allow adequate dose-finding efforts within rare disease drug development programs that help overcome the challenges presented by low patient prevalence and other factors. Furthermore, the frequent reliance on biomarkers as endpoints for dose-finding studies underscores the importance of biomarker development in rare diseases.
60 APPLIED LIFE SCIENCES↗
Evaluation of the 2022 West Nile virus forecasting challenge, USA
Abstract Background West Nile virus (WNV) is the most common cause of mosquito-borne disease in the continental USA, with an average of ~1200 severe, neuroinvasive cases reported annually from 2005 to 2021 (range 386–2873). Despite this burden, efforts to forecast WNV disease to inform public health measures to reduce disease incidence have had limited success. Here, we analyze forecasts submitted to the 2022 WNV Forecasting Challenge, a follow-up to the 2020 WNV Forecasting Challenge. Methods Forecasting teams submitted probabilistic forecasts of annual West Nile virus neuroinvasive disease (WNND) cases for each county in the continental USA for the 2022 WNV season. We assessed the skill of team-specific forecasts, baseline forecasts, and an ensemble created from team-specific forecasts. We then characterized the impact of model characteristics and county-specific contextual factors (e.g., population) on forecast skill. Results Ensemble forecasts for 2022 anticipated a season at or below median long-term WNND incidence for nearly all (> 99%) counties. More counties reported higher case numbers than anticipated by the ensemble forecast median, but national caseload (826) was well below the 10-year median (1386). Forecast skill was highest for the ensemble forecast, though the historical negative binomial baseline model and several team-submitted forecasts had similar forecast skill. Forecasts utilizing regression-based frameworks tended to have more skill than those that did not and models using climate, mosquito surveillance, demographic, or avian data had less skill than those that did not, potentially due to overfitting. County-contextual analysis showed strong relationships with the number of years that WNND had been reported and permutation entropy (historical variability). Evaluations based on weighted interval score and logarithmic scoring metrics produced similar results. Conclusions The relative success of the ensemble forecast, the best forecast for 2022, suggests potential gains in community ability to forecast WNV, an improvement from the 2020 Challenge. Similar to the previous challenge, however, our results indicate that skill was still limited with general underprediction despite a relative low incidence year. Potential opportunities for improvement include refining mechanistic approaches, integrating additional data sources, and considering different approaches for areas with and without previous cases. Graphical Abstract
54 ENVIRONMENTAL SCIENCES↗
Toward a Smart Metaverse City: Immersive Realism and 3D Visualization of Digital Twin Cities
Metaverse and its related extended reality technologies can enable immersive, realistic, and participatory visualization of 3D data, and their use and the potential within a smart city can be effective for supporting urban research and urban operations management. This book chapter describes a vision for prototyping a “Smart Metaverse City” to combine the unique advantage of the Metaverse technology with the two-way connectivity of a digital twin city application. This synergy aims to create a virtual environment for immersive geovisualization to help researchers and the public understand the complex urban system through science-based and data-driven approaches. This book chapter selectively reviews past technological and paradigm advancements for collecting, analyzing, and visualizing 3D urban big data. Then we present a prototyping Geographic Information System (GIS) framework, together with some relevant data sources and open-source web technologies, to help researchers create a smart Metaverse city. We demonstrate our vision and discuss its application opportunities through a real-world example, a digital twin city developed at the Oak Ridge National Laboratory, to facilitate participatory, smart, and sustainable campus management.
Xu, Haowen↗
On Mixed-Integer Programming Formulations for the Unit Commitment Problem
We provide a comprehensive overview of mixed-integer programming formulations for the unit commitment (UC) problem. UC formulations have been an especially active area of research over the past 12 years due to their practical importance in power grid operations, and this paper serves as a capstone for this line of work. We additionally provide publicly available reference implementations of all formulations examined. We computationally test existing and novel UC formulations on a suite of instances drawn from both academic and real-world data sources. Driven by our computational experience from this and previous work, we contribute some additional formulations for both generator production upper bounds and piecewise linear production costs. By composing new UC formulations using existing components found in the literature and new components introduced in this paper, we demonstrate that performance can be significantly improved—and in the process, we identify a new state-of-the-art UC formulation.
97 MATHEMATICS AND COMPUTING↗
Knowledge Graph Embedding using Large Language Models for COVID-19
CoVID-19 (cord19) open data source for training knowledge graph with LLM
Cord19↗
Advanced Terrestrial Simulator (ATS) evaluation dataset at 7 catchments across the continental United States
This dataset comprises of the input files and other files required for Advanced Terrestrial Simulator (ATS) simulations at 7 catchments across the continental United States. ATS is an integrated surface-subsurface hydrology model. We include Jupyter notebooks (within scripts folder) for individual catchments showing information (including data sources, river network, soil, geology, landuse types etc.) on preparing the machine readable input files. ATS observation output files are provided in the output folder. Figures and analyses (.xlsx sheets) are also provided. The catchments include, Taylor River Upstream (Colorado); (b) Cossatot River (Arkansas); (c) Panther Creek (Alabama); (d) Little Tennessee River (North Carolina and Georgia); (e) Mayo River (Virginia); (f) Flat Brook (New Jersey); (g) Neversink River headwaters (New York). Readme files are provided inside the directories providing more details. Files types include: .xml, .h5, .xlsx, .png, .ipynb, .py, .nc, .txt. All of the files types can be accessed by open source software, details on software requirements are following: .xml (any text editors including notepad and textedit), .h5 (in python using hdf libraries), .xlsx (WPS Office Spreadsheets, OpenOffice Calc, LibreOffice Calc, Microsoft Office etc.), .png (any image viewer), .ipynb (Jupyter notebook), .py (any text editors including notepad and textedit), .nc (using python or other open source software).
54 ENVIRONMENTAL SCIENCES↗
PastoralScape: An Environment-Driven Model of Vaccination Decision Making Within Pastoralist Groups in East Africa
Economic and cultural resilience among pastoralists in East Africa is threatened by the interconnected forces of climate change, contagious diseases spread and evolving national and international trade. A key factor in the resilience of livestock that communities depend on is human decision making regarding vaccination against prevalent diseases such as Rift Valley fever and Contagious Bovine Pleuropneumonia. This paper describes an agent-based model that couples models of disease propagation, animal health, human decision making, and external GIS data sources capturing measures of foraging condition. We describe the design of the sub-models, their coupling, and demonstrate the sensitivity of the model to parameters that relate to controllable factors such as government and NGO information sources that can influence human decision making patterns. This model is intended to form the basis upon which richer economic and human factor models can be built.
97 MATHEMATICS AND COMPUTING↗
Parametric Vehicle Choice Modeling: ParaChoice
Assess the evolving integration potential of light-duty (LDV) and heavy-duty vehicle (HDV) technologies, fuels, and infrastructure and their contributions to lowering emissions and petroleum consumption. Leverage existing LDV and build HDV ParaChoice capability to conduct parametric analyses that explore the trade-space for key factors that influence consumer choice and technology, fuel and infrastructure development. ParaChoice provides the unique capability to examine tipping points and tradeoffs, and can help quantify the effects of and mitigate uncertainty introduced by data sources and assumptions.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
A Comprehensive System of Energy Intensity Indicators for the U.S.: Methods, Data and Key Trends
This report describes a comprehensive system of energy intensity indicators for the United States that has been developed for the Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) over the past decade. This system of indicators is hierarchical in nature, beginning with detailed indexes of energy intensity for various sectors of the economy, which are ultimately aggregated to an overall energy intensity index for the economy as a whole. The aggregation of energy intensity indexes to higher levels in the hierarchy is performed with a version of the Log Mean Divisia index (LMDI) method. Based upon the data and methods in the system of indicators, the economy-wide energy intensity index shows a decline of about 16% in 2014 relative to a 1985 base year. Discussion of energy intensity indicators for each of the broad end-use sectors of the economy—residential, commercial, industrial, and transportation—is presented in the report. An analysis of recent changes in the efficiency of electricity generation in the U.S. is also included. A detailed appendix describes the data sources and methodology behind the energy intensity indicators for each sector.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗