Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Using gap-filled MAIAC AOD and WRF-Chem to estimate daily PM 2.5 concentrations at 1 km resolution in the Eastern United States

To link short-term exposures of air pollutants to health outcomes, scientists must use high temporal and spatial resolution estimates of PM 2.5 concentrations. We develop a daily PM 2.5 product at 1 × 1 km 2 spatial resolution across the eastern United States (east of 90° W) with the aid of 1 × 1 km 2 MAIAC aerosol optical depth (AOD) data, 36 × 36 km 2 WRF-Chem output, 1 × 1 km 2 land-use type from the National Land Cover Database, and 0.125° × 0.125° ERA-Interim re-analysis meteorology. A gap-filling technique is applied to MAIAC AOD data to construct robust daily estimates of AOD when the satellite data are missing (e.g., areas obstructed by clouds or snow cover). The input data are incorporated into a multiple-linear regression model trained to surface observations of PM 2.5 from the EPA Air Quality System (AQS) monitoring network. The model generates a high-fidelity estimate (r 2 = 0.75 using a 10-fold random cross-validation) of daily PM 2.5 throughout the eastern United States. Of the inputs to the statistical model, WRF-Chem output (r 2 = 0.66) is the most important contributor to the skill of the model. MAIAC AOD is also a strong contributor (r 2 = 0.52). Daily PM 2.5 output from our statistical model can be easily integrated into county-level epidemiological studies. The novelty of this project is that we are able to simulate PM 2.5 in a computationally efficient manner that is constrained to ground monitors, satellite data, and chemical transport model output at high spatial resolution (1 × 1 km 2 ) without sacrificing the temporal resolution (daily) or spatial coverage (>2,000,000 km 2 ).

47 OTHER INSTRUMENTATION↗

Leveraging Application Data Constraints to Optimize Database-Backed Web Applications

Exploiting the relationships among data is a classical query optimization technique. As persistent data is increasingly being created and maintained programmatically, prior work that infers data relationships from data statistics misses an important opportunity. We present Coco, the first tool that identifies data relationships by analyzing database-backed applications. Once identified, Coco leverages the constraints to optimize the application's physical design and query execution. Instead of developing a fixed set of predefined rewriting rules, Coco employs an enumerate-test-verify technique to automatically exploit the discovered data constraints to improve query execution. Each resulting rewrite is provably equivalent to the original query. Using 14 real-world web applications, our experiments show that Coco can discover numerous data constraints from code analysis and improve real-world application performance significantly.

Computer Science↗

Efficient telemetry format

Format would simplify ground processing of telemetry data. Also, missing minor frame would create error in only one set of source data instead of disrupting all sets. Format organizes data from various sources into autonomous blocks. Data are pre-processed, in effect, so main computer only needs to determine block type and process data set as batch.

Greenberg, E.↗

Modeling Ionosphere Environments: Creating an ISS Electron Density Tool

The International Space Station (ISS) maintains an altitude typically between 300 km and 400 km in low Earth orbit (LEO) which itself is situated in the Earth's ionosphere. The ionosphere is a region of partially ionized gas (plasma) formed by the photoionization of neutral atoms and molecules in the upper atmosphere of Earth. It is important to understand what electron density the spacecraft is/will be operating in because the ionized gas along the ISS orbit interacts with the electrical power system resulting in charging of the vehicle. One instrument that is already operational onboard the ISS with a goal of monitoring electron density, electron temperature, and ISS floating potential is the Floating Potential Measurement Unit (FPMU). Although this tool is a valuable addition to the ISS, there are limitations concerning the data collection periods. The FPMU uses the Ku band communication frequency to transmit data from orbit. Use of this band for FPMU data runs is often terminated due to necessary observation of higher priority Extravehicular Activities (EVAs) and other operations on ISS. Thus, large gaps are present in FPMU data. The purpose of this study is to solve the issue of missing environmental data by implementing a secondary electron density data source, derived from the COSMIC satellite constellation, to create a model of ISS orbital environments. Extrapolating data specific to ISS orbital altitudes, we model the ionospheric electron density along the ISS orbit track to supply a set of data when the FPMU is unavailable. This computer model also provides an additional new source of electron density data that is used to confirm FPMU is operating correctly and supplements the original environmental data taken by FPMU.

Gurgew, Danielle N.↗

Thermal Control Materials on MISSE-5 with Comparison to Earlier Flight Data

A variety of thermal control materials were flown on the Materials on International Space Station Experiment (MISSE)-5. Several types of beta cloth, as used in multi-layer insulation blankets, were flown, including samples from the same batch as used on the International Space Station. Two candidate sunshade materials for the James Webb Space Telescope were also exposed on MISSE-5. The white thermal control coating AZ93 was applied to Kapton instead of aluminum; this sample maintained good solar absorptance and did not indicate any significant level of contamination to the MISSE-5 experiment. Marker coatings maintained their color. Thermo-optical properties are discussed, along with comparable data from MISSE-2 and the Passive Optical Sample Assembly (POSA) - I experiments.

Finckenor, Miria↗

MPS dose reconstruction for internal emitters: some site-specific issues and approaches

Background: As part of the Million Person Study (MPS), dose reconstructions for internal emitters have been performed for several U.S. facilities where large quantities of radionuclides were handled. The main challenges and dominant sources of potential error in retrospective dose estimates for internally exposed workers have been found to vary from site to site. Here in this paper, we discuss some important issues encountered in dose reconstructions performed for selected MPS sites and the approaches used to address those issues. The focus is on some foundational components of retrospective dose assessments that have received little attention in the literature. Methods: The discussion is built around illustrative exposure data and dose reconstructions for workers at selected facilities addressed in the MPS. Related findings at some non-MPS sites are also discussed. Results: Each of the following items has been found to be a major source of potential error in reconstructed tissue doses for some MPS sites: identification of all dosimetrically important internal emitters; the time pattern of intake; the mode(s) of intake; reliability of bioassay measurements; application of surrogate (coworker) information in lieu of, or in conjunction with, worker-specific monitoring data; the chemical and physical forms of inhaled radionuclide; and the relation of air monitoring data to actual intake. Conclusions: (1) Much of the dose reconstruction effort for internal emitters should be devoted to development of best feasible exposure scenarios. (2) Coworker data should be used to assign exposure scenarios or dose estimates to workers with missing exposure data only if there is compelling evidence of similar coworker exposure. (3) Bioassay data for some radionuclides and periods of operation at MPS sites are of questionable reliability due to sizable uncertainties associated with contamination, recovery, or background issues. (4) Dose estimates derived solely from air monitoring data should be treated as highly uncertain values in the absence of site-specific information demonstrating that the data are reasonably predictive of intake. (5) For intakes known or assumed to be via inhalation, the uncertainty in lung dose typically is much greater than the uncertainty in dose to systemic tissues, when dose estimates are based on urinary excretion data. (6) The lung dose estimate often can be improved through development of site-specific respiratory absorption parameter values. (7) There is generally insufficient site-specific information to justify development of site-specific systemic models.

61 RADIATION PROTECTION AND DOSIMETRY↗

An Approach for Setting Priorities for Experimental Studies in Support of Safety Assessments - 19393

Many experiments have been conducted, in both academia and industry, to support the understanding of the disposal of radioactive waste. For the disposal of low-level radioactive waste (LLW) into radioactive waste repositories, it is important to develop an approach to tackle which datasets are able to directly underpin safety assessments, and what future research experiments could address the gaps in this understanding. In this paper, we present an approach to identify which future experiments should be considered to underpin safety assessments for disposals of LLW in a near-surface repository. The approach involves 6-steps: define the disposal problem (e.g. the disposal of LLW in engineered vaults); determine what key processes may affect the disposal of the waste; establish the assumptions underpinning the key processes; determine how sensitive the key process parameters are to variation; determine whether key data are available and assess the quality of that data with respect to the disposal problem; and determine attributes for experiments needed to supply missing key data. We show, in this paper, how this approach can be used to develop a better understanding of C-14 gas generation, under varying degrees of groundwater saturation, for the disposal of LLW into engineered vaults, in the near-surface. Furthermore, we have developed an approach that identifies the types of experiments and their attributes to aid researchers to help select appropriate types of experiments to underpin safety assessments. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Efficient species identification for Pacific salmon genetic monitoring programs

Abstract Genetic monitoring of Pacific salmon in the Columbia River basin provides crucial information to fisheries managers that is otherwise challenging to obtain using traditional methods. Monitoring programs such as genetic stock identification (GSI) and parentage‐based tagging (PBT) involve genotyping tens of thousands of individuals annually. Although rare, these large sample collections inevitably include misidentified species, which exhibit low genotyping success on species‐specific Genotyping‐in‐Thousands by sequencing (GT‐seq) panels. For laboratories involved in large‐scale genotyping efforts, diagnosing non‐target species and reassigning them to the appropriate monitoring program can be costly and time‐consuming. To address this problem, we identified 19 primer pairs that exhibit consistent cross‐species amplification among salmonids and contain 51 species informative variants. These genetic markers reliably discriminate among 11 salmonid species and two subspecies of Cutthroat Trout and have been included in species‐specific GT‐seq panels for Chinook Salmon, Coho Salmon, Sockeye Salmon, and Rainbow Trout commonly used for Pacific salmon genetic monitoring. The majority of species‐informative amplicons (16) were newly identified from the four existing GT‐seq panels, thus demonstrating a low‐cost approach to species identification when using targeted sequencing methods. A species‐calling script was developed that is tailored for routine GT‐seq genotyping pipelines and automates the identification of non‐target species. Following extensive testing with empirical and simulated data, we demonstrated that the genetic markers and accompanying script accurately identified species and are robust to missing genotypic data and low‐frequency, shared polymorphisms among species. Finally, we used these tools to identify Coho Salmon incidentally caught in the Columbia River Chinook Salmon sport fishery and used PBT to determine their hatchery of origin. These molecular and computing resources provide a valuable tool for Pacific salmon conservation in the Columbia River basin and demonstrate a cost‐effective approach to species identification for genetic monitoring programs.

Robinson, Zachary L.↗

Physical-chemical evaluation of hydraulic fracturing chemicals in the context of produced water treatment

Produced water is a significant waste stream that can be treated and reused; however, the removal of production chemicals—such as those added in hydraulic fracturing—must be addressed. One motivation for treating and reusing produced water is that current disposal methods—typically consisting of deep well injection and percolation in infiltration pits—are being limited. Furthermore, oil and gas production often occurs in arid regions where there is demand for new water sources. Here in this paper, hydraulic fracturing chemical additive data from California are used as a case study where physical-chemical and biodegradation data are summarized and used to screen for appropriate produced water treatment technologies. The data indicate that hydraulic fracturing chemicals are largely treatable; however, data are missing for 24 of the 193 chemical additives identified. More than one-third of organic chemicals have data indicating biodegradability, suggesting biological treatment would be effective. Adsorption-based methods and partitioning of chemicals into oil for subsequent separation is expected to be effective for approximately one-third of chemicals. Volatilization-based treatment methods (e.g. air stripping) will only be effective for approximately 10% of chemicals. Reverse osmosis is a good catch-all with over 70% of organic chemicals expected to be removed efficiently. Other technologies such as electrocoagulation and advanced oxidation are promising but lack demonstration. Chemicals of most concern due to prevalence, toxicity, and lack of data include propargyl alcohol, 2-mercaptoethyl alcohol, tetrakis hydroxymethyl-phosphonium sulfate, thioglycolic acid, 2-bromo-3-nitrilopropionamide, formaldehyde polymers, polymers of acrylic acid, quaternary ammonium compounds, and surfactants (e.g. ethoxylated alcohols). Future studies should examine the fate of hydraulic fracturing chemicals in produced water treatment trains to demonstrate removal and clarify interactions between upstream and downstream processes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

WE-EF-207-08: Improve Cone Beam CT Using a Synchronized Moving Grid, An Inter-Projection Sensor Fusion and a Probability Total Variation Reconstruction

Purpose: To present a cone beam computed tomography (CBCT) system, which uses a synchronized moving grid (SMOG) to reduce and correct scatter, an inter-projection sensor fusion (IPSF) algorithm to estimate the missing information blocked by the grid, and a probability total variation (pTV) algorithm to reconstruct the CBCT image. Methods: A prototype SMOG-equipped CBCT system was developed, and was used to acquire gridded projections with complimentary grid patterns in two neighboring projections. Scatter was reduced by the grid, and the remaining scatter was corrected by measuring it under the grid. An IPSF algorithm was used to estimate the missing information in a projection from data in its 2 neighboring projections. Feldkamp-Davis-Kress (FDK) algorithm was used to reconstruct the initial CBCT image using projections after IPSF processing for pTV. A probability map was generated depending on the confidence of estimation in IPSF for the regions of missing data and penumbra. pTV was finally used to reconstruct the CBCT image for a Catphan, and was compared to conventional CBCT image without using SMOG, images without using IPSF (SMOG + FDK and SMOG + mask-TV), and image without using pTV (SMOG + IPSF + FDK). Results: The conventional CBCT without using SMOG shows apparent scatter-induced cup artifacts. The approaches with SMOG but without IPSF show severe (SMOG + FDK) or additional (SMOG + TV) artifacts, possibly due to using projections of missing data. The 2 approaches with SMOG + IPSF removes the cup artifacts, and the pTV approach is superior than the FDK by substantially reducing the noise. Using the SMOG also reduces half of the imaging dose. Conclusion: The proposed technique is promising in improving CBCT image quality while reducing imaging dose.

60 APPLIED LIFE SCIENCES↗

Handling Large and Complex Data in a Photovoltaic Research Institution Using a Custom Laboratory Information Management System

Twenty-five years ago the desktop computer started becoming ubiquitous in the scientific lab. Researchers were delighted with its ability to both control instrumentation and acquire data on a single system, but they were not completely satisfied. There were often gaps in knowledge that they thought might be gained if they just had more data and they could get the data faster. Computer technology has evolved in keeping with Moore’s Law meeting those desires; however those improvements have of late become both a boon and bane for researchers. Computers are now capable of producing high speed data streams containing terabytes of information; capabilities that evolved faster than envisioned last century. Software to handle large scientific data sets has not kept up. How much information might be lost through accidental mismanagement or how many discoveries are missed through data overload are now vital questions. An important new task in most scientific disciplines involves developing methods to address those issues and to create the software that can handle large data sets with an eye towards scalability. This software must create archived, indexed, and searchable data from heterogeneous instrumentation for the implementation of a strong data-driven materials development strategy. At the National Center for Photovoltaics in the National Renewable Energy Laboratory, we began development a few years ago on a Laboratory Information Management System (LIMS) designed to handle lab-wide scientific data acquisition, management, processing and mining needs for physics and materials science data, and with a specific focus towards future scalability for new equipment or research focuses. We will present the decisions, processes, and problems we went through while building our LIMS system for materials research, its current operational state and our steps for future development.

14 SOLAR ENERGY↗

Graph identification of proteins in tomograms ( GRIP‐Tomo )

Abstract In this study, we present a method of pattern mining based on network theory that enables the identification of protein structures or complexes from synthetic volume densities, without the knowledge of predefined templates or human biases for refinement. We hypothesized that the topological connectivity of protein structures is invariant, and they are distinctive for the purpose of protein identification from distorted data presented in volume densities. Three‐dimensional densities of a protein or a complex from simulated tomographic volumes were transformed into mathematical graphs as observables. We systematically introduced data distortion or defects such as missing fullness of data, the tumbling effect, and the missing wedge effect into the simulated volumes, and varied the distance cutoffs in pixels to capture the varying connectivity between the density cluster centroids in the presence of defects. A similarity score between the graphs from the simulated volumes and the graphs transformed from the physical protein structures in point data was calculated by comparing their network theory order parameters including node degrees, betweenness centrality, and graph densities. By capturing the essential topological features defining the heterogeneous morphologies of a network, we were able to accurately identify proteins and homo‐multimeric complexes from 10 topologically distinctive samples without realistic noise added. Our approach empowers future developments of tomogram processing by providing pattern mining with interpretability, to enable the classification of single‐domain protein native topologies as well as distinct single‐domain proteins from multimeric complexes within noisy volumes.

59 BASIC BIOLOGICAL SCIENCES↗

National Energy Water Treatment & Speciation (NEWTS): A Water & Critical Mineral Database and Dashboard

The scarcity of water resources, the need for beneficial water reuse, and the challenges of wastewater treatment are becoming increasingly pressing in economic, social, and environmental domains. Addressing these concerns requires effective treatment strategies to manage wastewater streams and tackle environmental and economic issues. Furthermore, the recovery of critical minerals from the waste streams associated with energy production holds the promise of offsetting treatment costs and securing local sources of valuable minerals. However, relevant data on these waste streams are dispersed and challenging to locate. The process of ingesting such data into modeling software often involves multiple steps, requiring data restructuring to meet software-input requirements. The non-standardized reporting of water data makes data aggregation and reformatting a time-consuming process. Additionally, essential attributes necessary for modeling water treatment and mineral scale formation are frequently missing. Moreover, data gaps vary depending on the region of interest. Consequently, there is a pressing need for high-quality energy-water composition data that can be easily imported into water chemistry modeling software. To address this need, the National Energy Technology Laboratory has created the National Energy Water Treatment and Speciation (NEWTS) Database and Dashboard—a free online tool catering to community leaders and water researchers. NEWTS facilitates a comprehensive understanding of the composition of energy-related wastewater streams in the United States. The datasets provide detailed concentrations and speciation of major and minor aqueous compounds in energy-related wastewater streams, including power plant leachate, acid mine drainage, brackish water, and oil and gas produced water across the United States. Many of the aqueous species are critical minerals (Li, REEs) in high demand to modernize the world’s energy infrastructure. Many of the datasets also contain volumetric flow-rates needed to model the treatment and reuse scenarios in advanced aqueous chemistry software programs. The NEWTS Database and Dashboard offer public access to hitherto challenging-to-access datasets, presented in a standardized format that is tailored for easy input into aqueous chemistry modeling software. By performing the work needed to transform dispersed, disparate data sources into unified, model-ready datasets, NEWTS serves as an essential resource in advancing water treatment research and sustainable water resource management.

produced water management↗

Long-term missing value imputation for time series data using deep neural networks

We present an approach that uses a deep learning model, in particular, a MultiLayer Perceptron, for estimating the missing values of a variable in multivariate time series data. We focus on filling a long continuous gap (e.g., multiple months of missing daily observations) rather than on individual randomly missing observations. Our proposed gap filling algorithm uses an automated method for determining the optimal MLP model architecture, thus allowing for optimal prediction performance for the given time series. We tested our approach by filling gaps of various lengths (three months to three years) in three environmental datasets with different time series characteristics, namely daily groundwater levels, daily soil moisture, and hourly Net Ecosystem Exchange. We compared the accuracy of the gap-filled values obtained with our approach to the widely used R-based time series gap filling methods ImputeTS and mtsdi. The results indicate that using an MLP for filling a large gap leads to better results, especially when the data behave nonlinearly. Thus, our approach enables the use of datasets that have a large gap in one variable, which is common in many long-term environmental monitoring observations.

97 MATHEMATICS AND COMPUTING↗

Constructing a Simulation Surrogate with Partially Observed Output

Gaussian process surrogates are a popular alternative to directly using computationally expensive simulation models. When the simulation output consists of many responses, dimension-reduction techniques are often employed to construct these surrogates. However, surrogate methods with dimension reduction generally rely on complete output training data. This article proposes a new Gaussian process surrogate method that permits the use of partially observed output while remaining computationally efficient. The new method involves the imputation of missing values and the adjustment of the covariance matrix used for Gaussian process inference. The resulting surrogate represents the available responses, disregards the missing responses, and provides meaningful uncertainty quantification. In conclusion, the proposed approach is shown to offer sharper inference than alternatives in a simulation study and a case study where an energy density functional model that frequently returns incomplete output is calibrated.

42 ENGINEERING↗

Bayesian High-Rank Hankel Matrix Completion for Nonlinear Synchrophasor Data Recovery

Phasor measurement units (PMUs) provide high temporal-resolution synchrophasor measurements for power system monitoring and control. The frequent data quality issues, such as missing and bad data, prevent the incorporation of synchrophasor data in real-time operations. Most existing data-driven data recovery methods assume the power system dynamics can be approximated by a linear dynamical system, and the recovery performance degrades significantly when the power system is experiencing nonlinear dynamics during significant events. Here, this paper proposes a data-driven Bayesian nonlinear synchrophasor data recovery method (Ba-NSDR) that can recover a consecutive time period of simultaneous data losses or errors across all channels, even when the underlying system is highly nonlinear. The idea is to lift the Hankel matrix of the spatial-temporal synchrophasor data to a higher dimension such that the lifted Hankel matrix is low-rank in that space and can be processed with the kernel trick. Our proposed Bayesian method then infers the probabilistic distributions of synchrophasor from the partial observations. Some distinctive features of Ba-NSDR include an uncertainty index to measure the accuracy of the recovery result and the robustness to parameter selections. Our method is verified on both synthetic and recorded event datasets.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗