Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “attribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

A comparison between residential relocation timing of Sydney and Chicago residents: A Bayesian survival analysis

We report that understanding households' behaviour in residential relocation timing is of great importance in the field of transport engineering and economics. This research aims to develop a residential relocation model by considering the potential dynamic impacts of other households' decisions and variables, including economic and demographic attributes, housing features, intra-household decision-making structures, travel mode choice, and other life-course attributes. A multivariate parametric survival model with both fixed and time-varying covariates is developed. To the best of the authors' knowledge, this study is the first paper in the literature of residential relocation timing to propose the use of a Bayesian model in contrast to the widely used classic frequentist approach and have conducted a discussion on its advantages. An emerging residential relocation dataset collected for two cities in Australia and the USA (Sydney and Chicago cities) has been used, which covers residence, vehicle ownership, occupation, education, economic and demographic attributes of respondents. A comprehensive comparison between the results of two cities and a comparison between two Bayesian and frequentist approaches are made. This study confirms the impact of life-course variables, intra-household decision-making behaviours, and sociodemographic attributes on home mobility. According to the model outputs, the accelerating or decelerating impact of explanatory variables on the relocation timing has been almost the same in the two cities. The Bayesian model was confirmed to have some advantages over the frequentist model, including being straightforward to interpret, availability of making inferences on the results, and ease of handling complex models, and optimisation convergence complexities.

99 GENERAL AND MISCELLANEOUS↗

Discovering hidden geothermal signatures using non-negative matrix factorization with customized k-means clustering

Discovery of hidden geothermal resources is challenging. It requires the mining of large datasets with diverse data attributes representing subsurface hydrogeological and geothermal conditions. The commonly used play fairway analysis approach typically incorporates subject-matter expertise to analyze regional data to estimate geothermal characteristics and favorability. We demonstrate an alternative approach based on machine learning (ML) to process a geothermal dataset from southwest New Mexico (SWNM). The study region includes low- and medium-temperature hydrothermal systems. Several of these systems are not well characterized because of insufficient existing data and limited past explorative work. This study discovers hidden patterns and relations in the SWNM geothermal dataset to improve our understanding of the regional hydrothermal conditions and energy-production favorability. This understanding is obtained by applying an unsupervised ML algorithm based on non-negative matrix factorization coupled with customized k-means clustering (NMFk). NMFk can automatically identify (1) hidden signatures characterizing analyzed datasets, (2) the optimal number of these signatures, (3) the dominant data attributes associated with each signature, and (4) the spatial distribution of the extracted signatures. Here, in this study, NMFk is applied to analyze 18 geological, geophysical, hydrogeological, and geothermal attributes at 44 locations in SWNM. Using NMFk, we find data patterns and identify the spatial associations of hydrothermal signatures within two physiographic provinces (Colorado Plateau and Basin and Range) and two sub-regions of these provinces (the Mogollon-Datil volcanic field and the Rio Grande rift) in SWNM. The ML algorithm extracted five hydrothermal signatures in the SWNM datasets that differentiate between low (<90°C) and medium (90-150°C)-temperature hydrothermal systems. The algorithm also suggests that the Rio Grande rift and northern Mogollon-Datil volcanic field are the most favorable regions for future geothermal resource discovery. NMFk also identified critical attributes to identify medium-temperature hydrothermal systems in the study area. The resulting NMFk model can be applied to predict geothermal conditions and their uncertainties at new SWNM locations based on limited data from unexplored regions. The code to execute the performed analyses as well as the corresponding data can be found at https://github.com/SmartTensors/GeoThermalCloud.jl.

15 GEOTHERMAL ENERGY↗

Elucidating Cycling Rate-Dependent Electrochemical Strains in Sodium Iron Phosphate Cathodes for Na-ion Batteries

Battery electrodes materials undergo significant mechanical instabilities which affects their longevity and exert rate-limitations during the cycling process. In this study, we investigate the rate-dependent mechanical response of sodium iron phosphate (NaFePO4, NFP) cathodes during Na intercalation via galvanostatic cycling at different rates by employing digital image correlation, electrochemical methods, and mathematical model. The mechanical behaviour of the electrode shows strong dependance on the applied scan rate. At slower rates, electrode shows asymmetrical strain generation between anodic and cathodic cycles, which is attributed to the formation of cathode-electrolyte interface layers. The electrode undergoes smaller strain generation when cycled at slower rates when the same amount of Na ions is removed or inserted into the electrode. A mathematical model was developed to predict strain evolution in the composite electrode as well as the concentration profile of the Na ions in the electrode particles. Rate-dependent and time dependent factors on the strain generation in the electrode are attributed to the capacity-dependent intercalation strains, rate-dependent mismatch strains, and time-dependent irreversible strains. The combination of in situ strain measurements with the analytical model provided new insight into the electrochemically induced mechanical deformations in Na-ion cathode electrodes.

Ozdogru, Bertan↗

Measurement and analysis of GPU-accelerated applications with HPCToolkit

To address the challenge of performance analysis on the US DOE’s forthcoming exascale supercomputers, Rice University has been extending its HPCToolkit performance tools to support measurement and analysis of GPU-accelerated applications. To help developers understand the performance of accelerated applications as a whole, HPCToolkit’s measurement and analysis tools attribute metrics to calling contexts that span both CPUs and GPUs. To measure GPU-accelerated applications efficiently, HPCToolkit employs a novel wait-free data structure to coordinate monitoring and attribution of GPU performance. To help developers understand the performance of complex GPU code generated from high-level programming models, HPCToolkit constructs sophisticated approximations of call path profiles for GPU computations. To support fine-grained analysis and tuning, HPCToolkit uses PC sampling and instrumentation to measure and attribute GPU performance metrics to source lines, loops, and inlined code. To supplement fine-grained measurements, HPCToolkit can measure GPU kernel executions using hardware performance counters. To provide a view of how an execution evolves over time, HPCToolkit can collect, analyze, and visualize call path traces within and across nodes. Finally, on NVIDIA GPUs, HPCToolkit can derive and attribute a collection of useful performance metrics based on measurements using GPU PC samples. Here, we illustrate HPCToolkit’s new capabilities for analyzing GPU-accelerated applications with several codes developed as part of the Exascale Computing Project.

97 MATHEMATICS AND COMPUTING↗

Machine learning and shallow groundwater chemistry to identify geothermal prospects in the Great Basin, USA

This study discovers various geothermal prospects in the Great Basin, USA based on shallow groundwater chemical (geochemical) data. The geochemical data are expected to include hidden (latent) information that is a proxy for geothermal prospectivity. We processed the sparse geochemical data in the Great Basin at 14,341 locations including 18 attributes. Next, a non-negative matrix factorization with customized k-means clustering is applied to the geochemical data matrix that automatically finds three hidden geothermal signatures representing modestly, moderately, and highly confident geothermal prospects. The algorithm also evaluated the probability of occurrence of these types of resources through the studied region. There is a consistency between regional geothermal prospectivity as estimated by our ML methodology and the traditional play fairway analysis conducted over a portion of the study area. We also identify the dominant data attributes associated with each signature. Finally, our ML analyses allow us to reconstruct attributes from sparse into continuous over the study domain. The predicted continuous attributes can be used for future detailed geothermal explorations in the Great Basin.

15 GEOTHERMAL ENERGY↗

Ultrafast Exciton Dynamics of CH 3 NH 3 PbBr 3 Perovskite Nanoclusters

Exciton dynamics of perovskite nanoclusters has been investigated for the first time using femtosecond transient absorption (TA) and time-resolved photoluminescence (TRPL) spectroscopy. The TA results show two photoinduced absorption signals at 420 and 461 nm and a photoinduced bleach (PB) signal at 448 nm. The analysis of the PB recovery kinetic decay and kinetic model uncovered multiple processes contributing to electron-hole recombination. The fast component (~8 ps) is attributed to vibrational relaxation within the initial excited state, and the medium component (~60 ps) is attributed to shallow carrier trapping. The slow component is attributed to deep carrier trapping from the initial conduction band edge (~666 ps) and the shallow trap state (~40 ps). The TRPL reveals longer time dynamics, with modeled lifetimes of 6.6 and 93 ns attributed to recombination through the deep trap state and direct band edge recombination, respectively. In conclusion, the significant role of exciton trapping processes in the dynamics indicates that these highly confined nanoclusters have defect-rich surfaces.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Validation of the Community Land Model Version 5 over the Contiguous United States (CONUS) using in situ and remote sensing data sets

The Community Land Model (CLM) is an effective tool to simulate the biophysical and biogeochemical processes and their interactions with the atmosphere. Although CLM Version 5 (CLM5) constitutes various updates in these processes, its performance in simulating energy, water and carbon cycles over the Contiguous United States (CONUS) at scales which land surface changes and hydrometeorological and hydroclimatological applications are more locally relevant is yet to be assessed. In this study, we conducted three simulations at 0.125? during 1979-2018 over the CONUS using different configurations of CLM, namely CLM5-biogeochemistry (CLM5BGC), CLM4.5BGC, and CLM5-satellite phenology (CLM5SP). We validated and compared their simulations against multiple remote-sensed and in-situ datasets. Overall, the parametric and structural updates (e.g., carbon cost for nitrogen uptake, variable soil thickness, dry surface layer) in CLM5 improve its ability in capturing terrestrial biogeochemical dynamics. The low evapotranspiration in CLM5BGC is associated with biases in simulating vegetation phenological characteristics rather than soil water limitations. The mismatch between CLM5BGC-simulated peak leaf area index and reference data can be attributed to CLM5BGC's inability in simulating phenology of trees and grasses. The differences between CLM-simulated irrigation and reference estimates can be attributed to differences between processes represented in models and in reality, and uncertainties in input and validation datasets. Evaluation against observations at small catchments suggest that hydrologic parameters needed to be calibrated to improve simulations of runoff, especially subsurface runoff. Additional efforts are needed to incorporate spatially-distributed plant phenology and physiology parameters and regional-specific agricultural management practices (e.g., planting, harvest).

Cheng, Yanyan↗

Outer membrane vesicles catabolize lignin-derived aromatic compounds in Pseudomonas putida KT2440

Lignin is an abundant and recalcitrant component of plant cell walls. While lignin degradation in nature is typically attributed to fungi, growing evidence suggests that bacteria also catabolize this complex biopolymer. However, the spatiotemporal mechanisms for lignin catabolism remain unclear. Improved understanding of this biological process would aid in our collective knowledge of both carbon cycling and microbial strategies to valorize lignin to value-added compounds. Here, we examine lignin modifications and the exoproteome of three aromatic–catabolic bacteria: Pseudomonas putida KT2440, Rhodoccocus jostii RHA1, and Amycolatopsis sp. ATCC 39116. P. putida cultivation in lignin-rich media is characterized by an abundant exoproteome that is dynamically and selectively packaged into outer membrane vesicles (OMVs). Interestingly, many enzymes known to exhibit activity toward lignin-derived aromatic compounds are enriched in OMVs from early to late stationary phase, corresponding to the shift from bioavailable carbon to oligomeric lignin as a carbon source. In vivo and in vitro experiments demonstrate that enzymes contained in the OMVs are active and catabolize aromatic compounds. Taken together, this work supports OMV-mediated catabolism of lignin-derived aromatic compounds as an extracellular strategy for nutrient acquisition by soil bacteria and suggests that OMVs could pote

biological funneling↗

Improving streamflow predictions across CONUS by integrating advanced machine learning models and diverse data

Accurate streamflow prediction is crucial to understand climate impacts on water resources and develop effective adaption strategies. A global long short-term memory (LSTM) model, using data from multiple basins, can enhance streamflow prediction, yet acquiring detailed basin attributes remains a challenge. To overcome this, we introduce the Geo-vision transformer (ViT)-LSTM model, a novel approach that enriches LSTM predictions by integrating basin attributes derived from remote sensing with a ViT architecture. Applied to 531 basins across the Contiguous United States, our method demonstrated superior prediction accuracy in both temporal and spatiotemporal extrapolation scenarios. Geo-ViT-LSTM marks a significant advancement in land surface modeling, providing a more comprehensive and effective tool for better understanding the environment responses to climate change.

Tayal, Kshitij↗

How different power plant types contribute to electric grid reliability, resilience, and vulnerability: a comparative analytical framework

Abstract This work explores the dependability tradeoffs provided by the most common types of central power plants in the United States. Historically, the electricity sector has lacked consensus on how reliability , resilience , and vulnerability differ and how those metrics change depending on the power plant fleet composition. We propose distinct definitions for these metrics and an analytical framework to evaluate power plant fleet dependability. Using data analysis and literature review, we identify fifteen dependability attributes across which we rank eleven power plant types relative to natural gas combined-cycle (NGCC) plants. We use NGCC as the benchmark because it is common to many locations and is of relatively recent vintage. The framework shows that each power plant type has unique dependability benefits and drawbacks. We provide examples of how researchers may use the framework to evaluate grid dependability qualitatively under different scenarios. We find that assuming all attributes that contribute to grid dependability are equally important and additive, electric grid dependability is best supported when power plant fleets include a mixture of power generation technologies. Then, we discuss scenario characteristics that could alter the prioritization and relationships of attributes. We also find that if current capacity installation trends continue to favor low- and zero-carbon power plants, US power grids may benefit from increased resilience and reduced vulnerability at the cost of decreased reliability. We conclude by recommending methods for adapting the framework and quantifying relationships between attributes in individual scenarios.

Ramirez-Meyers, K. (ORCID:0000000291216952)↗

Laboratory investigation of hydraulic fracturing in granitic rocks using active and passive seismic monitoring

SUMMARY Knowledge of the fracturing processes can be important for the optimization of pressurized fluid injection operations in the deep underground rock mass. Active and passive seismic monitoring techniques have been used in the field for tracking or mapping the propagating hydraulic fracture. Although both these monitoring techniques provide valuable information about the generated fracture network, it is difficult for either technique to comprehensibly identify the different processes associated with hydraulic fracturing. The combined active and passive monitoring has the potential for better characterization of the complex hydraulic fracturing phenomena. In this study, laboratory hydraulic fracturing experiments with combined active and passive seismic monitoring were conducted on true triaxially loaded Barre granite cubes with different fluid injection rates. The seismic inelastic fracturing was detected by 16 passive acoustic emission sensors, where 3678 and 2370 seismic source events were detected for the high and low injection rate experiments, respectively. For active monitoring, strong variations in the attributes of signals were observed which were transmitted through four source–receiver pairs, placed both perpendicular and parallel to the generated hydraulic fracture. Positive velocity changes were observed for active sensor pairs with ray paths passing through the generated hydraulic fracture indicating fluid permeation, whereas isolated dry deformation was characterized by a slight but permanent velocity decrease. Compared to velocity, the energy of the active signals was 1–2 orders of magnitude more sensitive to different hydraulic fracturing processes. However, the sensitivity and signatures of the active signal attributes were found to be dependent on the frequency range and direction of ray path with respect to the location of the generated fracture network. Using the coupled evaluation of the active and passive signals we were able to systematically identify various hydraulic fracturing processes including: (1) aseismic deformation, (2) fracture initiation and fluid permeation, (3) pressure build-up, (4) fracture propagation and (5) pressure release and leak-off. The results of this study showed that combining the respective advantages of active and passive seismic techniques and using both of them to monitor the failure processes can facilitate a more comprehensive understanding and better control of the hydraulic stimulations in subsurface operations.

Geochemistry & Geophysics↗

Efficient graph representation framework for chemical molecule similarity tasks

Graph data has emerged in numerous scientific domains and machine learning techniques have been widely used for analysis and learning of diverse data for prediction and decision. Machine learning techniques can readily address complex problems by leveraging their structural information. But graphs cannot be directly used for existing machine learning algorithms unless encoded as vectors. The problem of efficient representation of graphs is a substantial challenge in graph machine learning. In this paper, we propose a novel two-stage framework for the representation of chemical molecule graphs based on the strengths of Graph Isomorphism Networks (GINs) and Siamese autoencoders. In the first stage, the GIN model is constructed and trained using the structural information of chemical molecule graphs. Node attributes, edge attributes, and edge indices are used as input data, while graph attributes are used as labels. The GIN model effectively captures the structural characteristics of graphs and can accurately predict graph attributes, i.e., molecular properties. It also generates Graph Embeddings, represented as vectors that encode the structural information of graphs. In the second stage, Graph Embedding vectors are further optimized for downstream similarity tasks while preserving the graph structural information. The Siamese autoencoder is constructed and trained, which reduces the dimensionality of the Graph Embedding vectors, while maximizing the preservation of structural information in the original high-dimensional vectors. The resulting low-dimensional Graph Embeddings can be effectively utilized for tasks such as approximate nearest neighbor search. The experimental results demonstrate the effectiveness of our proposed framework in accurately predicting graph similarity.

Ma, Jiaji↗

FeederGAN: Synthetic Feeder Generation via Deep Graph Adversarial Nets

This paper presents a novel, automated, generative adversarial networks (GAN) based synthetic feeder generation mechanism, abbreviated as FeederGAN. FeederGAN digests real feeder models represented by directed graphs via a deep learning framework powered by GAN and graph convolutional networks (GCN). Information of a distribution feeder circuit is extracted from its model input files so that the device connectivity is mapped onto the adjacency matrix and the device characteristics, such as circuit types (i.e., 3-phase, 2-phase, and 1-phase) and component attributes (e.g., length and current ratings), are mapped onto the attribute matrix. Then, Wasserstein distance is used to optimize the GAN and GCN is used to discriminate the generated graphs from the actual ones. A greedy method based on graph theory is developed to reconstruct the feeder using the generated adjacency and attribute matrices. Furthermore, our results show that the GAN generated feeders resemble the actual feeder in both topology and attributes verified by visual inspection and by empirical statistics obtained from actual distribution feeders.

42 ENGINEERING↗

Effects of forest degradation classification on the uncertainty of aboveground carbon estimates in the Amazon

Tropical forests are critical for the global carbon budget, yet they have been threatened by deforestation and forest degradation by fire, selective logging, and fragmentation. Existing uncertainties on land cover classification and in biomass estimates hinder accurate attribution of carbon emissions to specific forest classes. In this study, we used textural metrics derived from PlanetScope images to implement a probabilistic classification framework to identify intact, logged and burned forests in three Amazonian sites. We also estimated biomass for these forest classes using airborne lidar and compared biomass uncertainties using the lidar-derived estimates only to biomass uncertainties considering the forest degradation classification as well. Our classification approach reached overall accuracy of 0.86, with accuracy at individual sites varying from 0.69 to 0.93. Logged forests showed variable biomass changes, while burned forests showed an average carbon loss of 35%. We found that including uncertainty in forest degradation classification significantly increased uncertainty and decreased estimates of mean carbon density in two of the three test sites. Our findings indicate that the attribution of biomass changes to forest degradation classes needs to account for the uncertainty in forest degradation classification. By combining very high-resolution images with lidar data, we could attribute carbon stock changes to specific pathways of forest degradation. This approach also allows quantifying uncertainties of carbon emissions associated with forest degradation through logging and fire. Both the attribution and uncertainty quantification provide critical information for national greenhouse gas inventories.

54 ENVIRONMENTAL SCIENCES↗

Seismic response to paleo-sand dunes in the Nugget Sandstone Formation, southwestern Wyoming

We have analyzed a 3D seismic survey acquired for a carbon sequestration project on top of the Moxa Arch in southwestern Wyoming. We observed a zone of discontinuous reflectors on vertical slices of seismic amplitude volume, whereas, the northwest–southeast lineations were observed on the time slices. We performed a seismic to well tie that suggested that the lineations occur within the Nugget Sandstone. The Nugget Sandstone is an eolian sandstone deposit of Early Jurassic age, deposited as a subtropical dune field, and equivalent to the Navajo Sandstone of southwestern Utah. Petrophysical analysis indicates that the Nugget Sandstone is dominated by clean sandstone (70%–80%), whereas evaporites, including halite and anhydrite, are present in certain zones. Previous outcrop studies on the Navajo Sandstone indicate the wind direction to be predominantly northeast–southwest. Seismic attributes, including coherence and curvature, displayed on stratal slices within the Nugget Sandstone interval indicate the presence of lineations in the northwest–southeast direction with irregular spacing. These lineations are approximately perpendicular to the inferred dominant wind direction. We computed the dominant wind direction from the average azimuth of the lineations as seen on the curvature attribute in the Nugget Sandstone interval. Geological feature: Eolian sand dunes with interdunal evaporites Seismic appearance: Parallel lineations with irregular spacing on seismic attribute horizon slices Alternative interpretations: Canyons at continental slopes; slope failures Features with a similar appearance: Marine bars; contourites Formation: The Nugget Sandstone — equivalent to the Navajo Sandstone Age: Early Jurassic Location: Moxa Arch, Wyoming Seismic data: Obtained by the University of Wyoming with U.S. DOE funding Contributors: Dhruv Agrawal, Brady Lujan, Sumit Verma, Shuvajit Bhattacharya, and Subhashis Mallick Analysis tools: Coherence and curvature attributes; seismic inversion; petrophysical inversion

Geochemistry & Geophysics↗

Explosion Discrimination Using Seismic Gradiometry and Spectral Filtering of Data

Here, we present a new method to discriminate between earthquakes and buried explosions using observed seismic data. The method is different from previous seismic discrimination algorithms in two main ways. First, we use seismic spatial gradients, as well as the wave attributes estimated from them (referred to as gradiometric attributes), rather than the conventional three-component seismograms recorded on a distributed array. The primary advantage of this is that a gradiometer is only a fraction of a wavelength in aperture compared with a conventional seismic array or network. Second, we use the gradiometric attributes as input data into a machine learning algorithm. The resulting discrimination algorithm uses the norms of truncated principal components obtained from the gradiometric data to distinguish the two classes of seismic events. Using high-fidelity synthetic data, we show that the data and gradiometric attributes recorded by a single seismic gradiometer performs as well as a conventional distributed array at the event type discrimination task.

58 GEOSCIENCES↗

Bay Area Regional Energy: Network Integrated Commercial Retrofits (BRICR) Project. Final Report

The BRICR project applied large-scale building energy modeling concepts with the aim of reducing the cost of energy efficiency targeting, design, and project development, and measurement of energy savings for energy efficiency programs implemented by local governments that serve small and medium commercial buildings (SMB). The project leveraged the services and resources of existing local government energy programs serving disadvantaged and hard-to-reach SMB customers. In contrast to programs run by utilities, local government programs generally do not have direct access to energy billing records for an entire class of customers in a geographic area, which prior research demonstrated useful for large-scale building energy model baseline development and calibration. , However, local governments are rich in public records that offer important clues about physical attributes and uses that, along with behavior, determine energy use. Relying only on public records, BRICR demonstrated development of credible baseline energy models for 3,792 office, retail, and hotel buildings. Publicly disclosed annual energy use data from a local energy benchmarking program and anonymized data from the Building Performance Database, the nation’s largest dataset about energy-related characteristics of buildings, were utilized to validate and calibrate energy models via an innovative method comparing distributions of energy intensity by fuel type for portfolios of buildings of similar size, vintage, and use. Portfolio calibration does not provide certainty that an energy model fits an individual building; the method is useful when billing data is not accessible – a common situation for researchers, energy service providers and ESCOs, local governments, and any party other than a utility. A software component was developed, the BRICR gem, which automates simulation when relevant data is added or edited by the user to a file saved in the standardized BuildingSync XML schema for energy audit data. The component was demonstrated as a simplified means to generate a mass of energy models corresponding to public records containing basic attributes such as building scale, location, use, year built, and aspect ratio in combination with building energy code prototype data corresponding to use and vintage. The component was also demonstrated as a simplified means to automate energy simulation when attributes are revised; the intention was to enable iterative improvement of the baseline model and energy savings estimates for common energy conservation measures as users revise relevant attributes based on their observations. In the context of institutional change and uncertainty for the participating local government energy programs, 13 whole building retrofits were completed. Impacts were measured by applying the CalTRACK2.0 methods to standardize measurement of normalized metered energy consumption. The GRIDMeter methods of stratified sampling and individual load shape analysis were applied to adjust for impacts of the effect of COVID-19 on retrofitted buildings in the context of all local buildings of similar size and use. Excluding impacts of the pandemic, retrofitted buildings demonstrated between 1.6% and 25.1% reduction in energy use. The project contributed use cases and feedback that helped inform evolution of the software tools and data formats that were combined for the first time in the BRICR project.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE COVID-19 Data Curation Effort: Overview of Initial Data Collection Coverage (March - June 2020)

During the COVID-19 pandemic of 2020, major case reporting outlets quickly coalesced around two or three primary vendors. Johns Hopkins University and The New York Times were among the more prominent, and all were of great value to the nation, particularly during the uncertain early stages of the pandemic. They primarily focused on three major attributes: number of new cases, deaths, and recovery, but only at the state level. Recognizing that many states were reporting very detailed data sets (e.g., hospital beds) at a count level or finer, the ORNL Pandemic Modeling team embarked on a major data curation effort from March to June 2020 for the purpose of capturing this wealth of detailed data. The challenge of curating this data was daunting. The number of attributes reported by the states grew on almost on a weekly basis. States were routinely shifting their web tool strategies away from easily parsable HTML-based formatting to new Tableau and ArcGIS content. This growth in the sheer number of attributes combined with the unpredictable shifts in data format meant an aggressive and agile combination of automated scripting and manual scraping was required to capture new daily streams. To keep up, the team had to scale up staff and widen its approach for capture and storage. The DOE COVID-19 data collection effort resulted in over 11 million data points being collected, covering over 13,000 unique geographies and over 2,000 unique attributes that spanned predominantly from early March through the end of June 2020.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗