Autodataman
The autodataman tool is intended to be a very lightweight climate data database and data distribution engine.
Ordonez, AnaC↗
SEARCH · Engineering Papers
Results for “Distributed databases”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Theory and Simulation of Ultrafast Multidimensional Nonlinear X-ray Spectroscopy of Molecules (Final Report)
Emerging X-ray free electron laser (XFEL) beam sources offer new types of probes of matter with unprecedented spatial and temporal resolutions. These experimental advances must be met by robust theoretical and computational tools that provide predictive modeling capacity of the underlining electronic and structural dynamics. The latter will be essential for the design of sophisticated multi-pulse experiments and for their interpretation. The proposed research effort will focus on developing cutting- edge simulation tools for nonlinear multidimensional X-ray/optical spectroscopies and aims to address key questions in Priority Research Opportunities 1 (Probing and controlling electron motion within a molecule) and 3 (Capturing rare events and intermediate states in the transformation of matter) as mentioned in the BES Roundtable Report “Opportunities for Basic Research at the Frontiers of XFEL Ultrafast Science”. XFEL multidimensional nonlinear techniques, which combine sequences of X-ray and possibly optical pulses, provide a unique experimental toolbox for probing the dynamics of core and valence electronic excitations, as well as material structure. Predictive modeling of these dynamical processes requires the combination of analytical theory for nonlinear interactions of light and matter, robust quantum-chemical methodologies for the accurate description of electronic structure of various materials, and multiscale ab initio electron and nuclear dynamics techniques operating beyond Born-Oppenheimer approximation. These challenges will be addressed with three research thrusts (i) Develop and implement theoretical apparatus for modeling a broad range of multidimensional spectroscopic techniques enabled by present and upcoming XFEL facilities. This thrust also includes the incorporation of a computational module in the DOE supported open-source NWChem computational chemistry package as well as the development of other open-source codes ready for dissemination across a broad user base; (ii) Propose and design new multi-pulse experiments that make use of the capabilities of the incoming LCLS-II facility; (iii) Perform selected applications to specific molecular systems that can be carried out at LCLS-II and demonstrate how these X-ray sources may be used to study nonadiabatic dynamics through conical interactions, electronic correlations in multi-core excitons, and charge transfer/energy transfer processes. The proposed research will be carried out by a multi-disciplinary four-institution research team which combines academia and national laboratories and spans the broad and necessary expertise in theoretical spectroscopy, nonlinear optics, quantum chemistry, molecular non-adiabatic dynamics and code development. The work will be performed in a highly interactive team environment with junior researchers shared between institutions thus cementing cross- disciplinary interactions. The developed simulation tools will be immediately deployed for XFEL facility users, both experimentalists and theorists, via freely distributed codes and databases. Altogether, this project will facilitate establishing XFEL-based multidimensional spectroscopies as a novel diagnostic tool for monitoring electronic and structural dynamics in molecular materials.
74 ATOMIC AND MOLECULAR PHYSICS↗
Land Use and Land Cover Affect the Depth Distribution of Soil Carbon: Insights From a Large Database of Soil Profiles
Not Available
54 ENVIRONMENTAL SCIENCES↗
GriddingMachine, a database and software for Earth system modeling at global and regional scales
Land and Earth system modeling is moving towards more explicit biophysical representations, requiring increasing variety of datasets for initialization and benchmarking. However, researchers often have difficulties in identifying and integrating non-standardized datasets from various sources. We aim towards a standardized database and one-stop distribution method of global datasets. Here, we present the GriddingMachine as (1) a database of global-scale datasets commonly used to parameterize or benchmark the models, from plant traits to vegetation indices and geophysical information and (2) a cross-platform open source software to download and request a subset of datasets with only a few lines of code. The GriddingMachine datasets can be accessed either manually through traditional HTTP, or automatically using modern programming languages including Julia, Matlab, Octave, Python, and R. The GriddingMachine collections can be used for any land and Earth modeling framework and ecological research at the regional and global scales, and the number of datasets will continue to grow to meet the increasing needs of research communities.
58 GEOSCIENCES↗
Analysis of Spectral Lines in Large Databases of Synthetic Spectra for Massive Stars
In this paper, we describe a program that identifies in the optical spectrum the main parameters of a spectral line, namely the initial and final wavelengths, and the line depth. Moreover, using numerical calculations, it identifies and removes adjacent lines. Next, the program calculates the equivalent width and the FWHM. The software was tested in a sample of 300 lines in two databases of synthetic spectra generated by the CMFGEN and PoWR codes, and 300 lines in observed spectra from the IACOB database, showing a Gaussian distribution of relative errors, from which it is inferred that 80% of the measured lines have errors less than 17% and only 5% of the lines have errors greater than 26%. The program was also run on the entire database of 45,000 CMFGEN and 202 POWR synthetic spectra, generating a library of H i, He i, and He ii lines necessary to feed the FITspec code for the derivation of stellar parameters: effective temperature, surface gravity, and luminosity.
47 OTHER INSTRUMENTATION↗
Phase Identification in Real Distribution Networks with High PV Penetration Using Advanced Metering Infrastructure Data
Many distribution network monitoring and control applications - including state estimation, volt/VAR optimization, and network reconfiguration - rely on accurate network models; however, the network models maintained by utilities can become outdated because of restoration activities, network reconfiguration, and missing data. With the widespread deployment of advanced metering infrastructure (AMI), abundant measurement data from low-voltage secondary networks are available. The AMI measurement data can be used for phase identification to improve the network models. Although the existing phase identification techniques work well in passive distribution feeders that do not have photovoltaic (PV) generation, they can fail to accurately identify the phases in the presence of PV. This paper proposes a robust phase identification algorithm based on supervised machine learning that accurately identifies the AMI meter phase connectivity in the presence of significant PV generation. The proposed algorithm does not require network topology information or feeder head measurement data. The algorithm is validated using the AMI measurement data collected in the field and the field-validated phase connectivity database on two real distribution feeders from San Diego Gas & Electric Company that have significant PV generation.
advanced metering infrastructure↗
Phase Identification in Real Distribution Networks with High PV Penetration Using Advanced Metering Infrastructure Data: Preprint
Many distribution network monitoring and control applications - including state estimation, volt/VAR optimization, and network reconfiguration - rely on accurate network models; however, the network models maintained by utilities can become outdated because of restoration activities, network reconfiguration, and missing data. With the widespread deployment of advanced metering infrastructure (AMI), abundant measurement data from low-voltage secondary networks are available. The AMI measurement data can be used for phase identification to improve the network models. Although the existing phase identification techniques work well in passive distribution feeders that do not have photovoltaic (PV) generation, they can fail to accurately identify the phases in the presence of PV. This paper proposes a robust phase identification algorithm based on supervised machine learning that accurately identifies the AMI meter phase connectivity in the presence of significant PV generation. The proposed algorithm does not require network topology information or feeder head measurement data. The algorithm is validated using the AMI measurement data collected in the field and the field-validated phase connectivity database on two real distribution feeders from San Diego Gas & Electric Company that have significant PV generation.
advanced metering infrastructure↗
Phase Identification in Real Distribution Networks with High PV Penetration Using Advanced Metering Infrastructure Data
Many distribution network monitoring and control applications - including state estimation, Volt/VAr optimization, and network reconfiguration - rely on accurate network models; however, the network models maintained by utilities can become outdated because of restoration activities, network reconfiguration, and missing data. With the widespread deployment of advanced metering infrastructure (AMI), abundant measurement data from low-voltage secondary networks are available. The AMI measurement data can be used for phase identification to improve the network models. Although the existing phase identification techniques work well in passive distribution feeders that do not have photovoltaic (PV) generation, they can fail to accurately identify the phases in the presence of PV. This paper proposes a robust phase identification algorithm based on supervised machine learning that accurately identifies the AMI meter phase connectivity in the presence of significant PV generation. The proposed algorithm does not require network topology information or feeder-head measurement data. The algorithm is validated using the AMI measurement data collected in the field and the field-validated phase connectivity database on two real distribution feeders from San Diego Gas & Electric Company that have significant PV generation.
advanced metering infrastructure (AMI)↗
New Architecture to Support Integration and Processing of Seismic Data from Heterogeneous Sources
The Geophysical Monitoring Program (GMP) at Lawrence Livermore National Lab (LLNL) maintains a database and supporting infrastructure for geophysical data used in support of the Nuclear Detonation Detection mission. This database includes data from multiple sources, many of which do not distribute data to the public or for which there is no automated means of access. For example, Figure 1 shows (left) the distribution of waveform data in our database by source. The Incorporated Research Institutions for Seismology Data Management Center (IRISDMC) is our major source of waveform data and those data may be retrieved at will using the Federated Digital Seismograph Networks FDSN web Application Programming Interface (API). However, the next 6 most important sources of waveform data have no or only limited automated access to waveforms. As Figure 1 (right) shows, it is very common for waveform records associate with an event in our database to come from two or more sources, and in some cases data come from 10 sources. This diversity of data sources drives our need for efficient and correct integration of metadata, parametric data, and waveform data.
58 GEOSCIENCES↗
Enabling Modular Autonomous Feedback‐Loops in Materials Science through Hierarchical Experimental Laboratory Automation and Orchestration
Abstract Materials acceleration platforms (MAPs) operate on the paradigm of integrating combinatorial synthesis, high‐throughput characterization, automatic analysis, and machine learning. Within a MAP, one or multiple autonomous feedback loops may aim to optimize materials for certain functional properties or to generate new insights. The scope of a given experiment campaign is defined by the range of experiment and analysis actions that are integrated into the experiment framework. Herein, the authors present a method for integrating many actions within a hierarchical experimental laboratory automation and orchestration (HELAO) framework. They demonstrate the capability of orchestrating distributed research instruments that can incorporate data from experiments, simulations, and databases. HELAO interfaces laboratory hardware and software distributed across several computers and operating systems for executing experiments, data analysis, provenance tracking, and autonomous planning. Parallelization is an effective approach for accelerating knowledge generation provided that multiple instruments can be effectively coordinated, which the authors demonstrate with parallel electrochemistry experiments orchestrated by HELAO. Efficient implementation of autonomous research strategies requires device sharing, asynchronous multithreading, and full integration of data management in experimental orchestration, which to the best of the authors’ knowledge, is demonstrated for the first time herein.
36 MATERIALS SCIENCE↗
North American Lithium-Ion Battery Supply Chain Database Development - Phase II
Lithium-ion batteries (LIBs) are used in a wide range of applications, including cell phones, laptops, power tools, electric vehicles, and grid storage, and are essential for economic growth and addressing climate change. However, the significant demand for LIBs has led to supply chain issues for the United States, as China dominates the processing of battery materials and battery production. To address this concern, NAATBatt International, a trade association of North American battery companies, supported the National Renewable Energy Laboratory in developing a database of companies that mine, process, manufacture, reuse, and recycle batteries in North America. The purpose of this database was to identify strengths and gaps in the supply chain, so that private-government partnerships could develop strategies to create a competitive LIB supply chain in the US. NREL published the first version of this database in 2021 and the second version in 2022. The database includes companies that have a manufacturing facility in North America and are engaged in materials, cells, packs, end-of-life management, as well as those involved in LIB battery modeling, distribution, service and repair, and R&D. In this presentation, we will discuss our approach to collecting data and categorizing various segments and products. We will also provide a summary of the data and present various maps to illustrate the distribution of companies in the database.
ADVANCED PROPULSION SYSTEMS,ENERGY STORAGE↗
Database development and exploration of process–microstructure relationships using variational autoencoders
The paper demonstrates graphical representation of a large database containing process–microstructure relationships using an unsupervised machine learning algorithm. Correlating microstructural features to processing is an essential first step to answer the difficult problem of process sequence design. Here, a large database of 346,200 orientation distribution functions resulting from a variety of process sequences is constructed, where each sequence comprises up to four stages of tension, compression and rolling along different directions in various permutations. This open-source database is constructed for collaborative development of process design algorithms. The paper demonstrates a novel application of the large database: graphical representation of texture–process relationships. A variational autoencoder is used to reduce the entire database to a two dimensional latent space where variations in processes and properties can be visualized. Using proximity analysis in this latent space, we can quickly unearth multiple process solutions to the problem of texture or property design.
36 MATERIALS SCIENCE↗
Modeling Nanoconfinement Effects Using Active Learning
Predicting the spatial configuration of gas in nanopores of is relevant in applications such as fluid flow forecasting and hydrocarbon reserves estimation. For example, shale reservoirs have suffered from computationally intractable multiscale problems, since fluid properties such as viscosity, density, and adsorption must be calculated by using expensive molecular dynamics (MD) simulations within each nanopore, whereas flow through these connected nanopores must be simulated at the micrometer scale. We utilize machine learning techniques to quickly and accurately model nanoscale confinement effects as an important step toward bridging the nano and micro scales. Our workflow is based on building and training physics-based deep-neural-networks models by learning from a database of MD calculations. The model accounts for the adsorption phenomenon by predicting the statistical distribution of gas inside nanopores. Because large databases of MD calculations are expensive to create, we investigate active learning (AL) as a data set construction strategy. In this workflow, new data are selected based on the model uncertainty via the query-by-committee approach. We show that our workflow obtains accurate models that generalize to real scanning electron microscopy geometries with 1/10th of the number of MD calculations required vs random data set generation. Our method enables the possibility of modeling nanoconfinement effects at the mesoscale, where complex connected sets of nanopores affect flow.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
The Materials Provenance Store
Abstract We present a database resulting from high throughput experimentation, primarily on metal oxide solid state materials. The central relational database, the Materials Provenance Store (MPS), manages the metadata and experimental provenance from acquisition of raw materials, through synthesis, to a broad range of materials characterization techniques. Given the primary research goal of materials discovery of solar fuels materials, many of the characterization experiments involve electrochemistry, along with optical, structural, and compositional characterizations. The MPS is populated with all information required for executing common data queries, which typically do not involve direct query of raw data. The result is a database file that can be distributed to users so that they can independently execute queries and subsequently download the data of interest. We propose this strategy as an approach to manage the highly heterogeneous and distributed data that arises from materials science experiments, as demonstrated by the management of over 30 million experiments run on over 12 million samples in the present MPS release.
36 MATERIALS SCIENCE↗
A catastrophic tropical drought kills hydraulically vulnerable tree species
Drought-related tree mortality is now a widespread phenomenon predicted to increase in magnitude with climate change. However, the patterns of which species and trees are most vulnerable to drought, and the underlying mechanisms have remained elusive, in part due to the lack of relevant data and difficulty of predicting the location of catastrophic drought years in advance. In this work, we used long-term demographic records and extensive databases of functional traits and distribution patterns to understand the responses of 20–53 species to an extreme drought in a seasonally dry tropical forest in Costa Rica, which occurred during the 2015 El Niño Southern Oscillation event. Overall, species-specific mortality rates during the drought ranged from 0% to 34%, and varied little as a function of tree size. By contrast, hydraulic safety margins correlated well with probability of mortality among species, while morphological or leaf economics spectrum traits did not. This firmly suggests hydraulic traits as targets for future research.
59 BASIC BIOLOGICAL SCIENCES↗
Crowd-based spatial risk assessment of urban flooding: Results from a municipal flood hotline in Detroit, MI
Climate change is increasing the frequency and intensity of extreme precipitation events, raising the risk of urban flood disasters. This study uses a crowd-sourced municipal call database to characterize the spatial distribution of flood risk in Detroit, MI. Call data including dates and addresses were obtained from the City of Detroit Department of Public Works for 2021. Calls were mapped and aggregated to census tract counts and merged with neighborhood-level data. Associations of predictors with flood calls were tested using spatial regression models. Flooding calls were located throughout the city but were concentrated in specific areas. Multivariate models of census tract level call counts indicated that increased poverty and Black, immigrant, and older residents were positively associated with flood calls, while increased elevation was associated with protective effects. Longer distances from waste water interceptors were associated with higher risk for calls. Crowd-sourced flood hotline call data can be used for effective spatial flood risk assessment. Though flooding occurs throughout the city of Detroit, infrastructural, neighborhood, and household factors influence flooding extent. Limitations included the self-reported nature of calls. Future modeling efforts might include input from local stakeholders to improve spatial risk assessment.
54 ENVIRONMENTAL SCIENCES↗
Software and computing for Run 3 of the ATLAS experiment at the LHC
The ATLAS experiment has developed extensive software and distributed computing systems for Run 3 of the LHC. These systems are described in detail, including software infrastructure and workflows, distributed data and workload management, database infrastructure, and validation. The use of these systems to prepare the data for physics analysis and assess its quality are described, along with the software tools used for data analysis itself. An outlook for the development of these projects towards Run 4 is also provided.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
ERAD (Energy Resilience Analysis for Electric Distribution Systems) [SWR-22-75]
ERAD is a software product for computing resiliency metrics in power distribution systems. It uses graph database technology to store, query and compute metrics making it highly scalable and available. Metrics can be computed for various resiliency scenarios such as fire, flooding, earthquake.
Palmintier, Bryan↗