Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Snow Depth and Density at End-of-Winter for NGEE Areas A, B, C and D, Utqiagvik (Barrow), Alaska, 2012-2014

End-of-winter snow depth and average snow density from area A, B, C and D, which include 1000's of point depth measurement located between ~20 and ~50 cm apart. The data was originally provided in excel workbooks, these files have been preserved within the zip Original_Files. The files were transformed for archiving with images pulled out into jpegs and data preserved as csv files. Some functions, features, and formatting was lost during the transformation. Missing data was defined with blank cells. Additional documentation can be found in files labelled with readme.csv and information.csv file endings.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1999

A Probabilistic Approach of Incorporating Safety and Reliability in System Designs for a Manned Mission to Mars

Conceptual stages in mission design often lack the input of quantitative safety and reliability assessments, simply because failure rates or other data are not yet available for systems that have not yet been designed. Absence of such data should not, however, prevent the development of a quantitative risk models with placeholders for missing data. Functions (that is, actions the systems must perform) in mission design will eventually require system probabilities of success, and there could be much learned from surrogate data, adequately bounded in uncertainty, used in a large event tree model of a complex mission.

Railsback, Jan W.↗
NASA NTRS2010

Automated Discovery and Modeling of Sequential Patterns Preceding Events of Interest

The integration of emerging data manipulation technologies has enabled a paradigm shift in practitioners' abilities to understand and anticipate events of interest in complex systems. Example events of interest include outbreaks of socio-political violence in nation-states. Rather than relying on human-centric modeling efforts that are limited by the availability of SMEs, automated data processing technologies has enabled the development of innovative automated complex system modeling and predictive analysis technologies. We introduce one such emerging modeling technology - the sequential pattern methodology. We have applied the sequential pattern methodology to automatically identify patterns of observed behavior that precede outbreaks of socio-political violence such as riots, rebellions and coups in nation-states. The sequential pattern methodology is a groundbreaking approach to automated complex system model discovery because it generates easily interpretable patterns based on direct observations of sampled factor data for a deeper understanding of societal behaviors that is tolerant of observation noise and missing data. The discovered patterns are simple to interpret and mimic human's identifications of observed trends in temporal data. Discovered patterns also provide an automated forecasting ability: we discuss an example of using discovered patterns coupled with a rich data environment to forecast various types of socio-political violence in nation-states.

Rohloff, Kurt↗
DOE OSTI2026

Data mining the missing ordered phases of Li/Na metal oxides

Data-driven discovery of Li-ion and Na-ion battery materials has been pioneered by generic materials data platforms such as the Materials Project. After decades of progress, it is timely to ask whether there remain underexplored compositional spaces. Here, in this work, we present a systematic data-mining effort to uncover missing ordered binary, ternary and quaternary Li/Na-containing metal oxides using high-throughput density functional theory (DFT). Building on 19,120 stable and metastable oxides entries from the Materials Project, we performed 13,245 additional calculations through isovalent substitutions of known ground states, experimentally reported compounds, and specific prototype structures. Our study identifies 36 new ground states within the GGA/GGA + U convex hull and 45 within the r 2 SCAN convex hull. Additionally, we identified 840 metastable compounds from GGA/GGA + U and 979 from r 2 SCAN that are absent in the present Materials Project databases. Moreover, we have tripled the metastable materials in compositional spaces with a molar ratio of cation/anion >1, highlighting the overlooked opportunities in this compositional space.

25 ENERGY STORAGE↗
NASA NTRS1976

DSN telemetry system data records

The DSN telemetry system now includes the capability to provide a complete magnetic tape record, within 24 hours of reception, of all telemetry data received from a spacecraft. This record, the intermediate data record, is processed and generated almost entirely automatically, and provides a detailed accounting of any missing data.

Gatz, E. C.↗
DOE OSTI2020

Provenance infrastructure for multimodal x-ray experiments and reproducible analysis

Exploring relationships between material structures and desired properties for material discovery and synthesis requires mining machine-readable databases and linking experimental data with sufficiently rich annotations. There are vast gaps both in these data sources and in the connections between them. We present a provenance-based data management and analysis framework that enables capturing, persisting, and reanalyzing experimental data from the BNL National Synchrotron Light Source II (NSLS-II). Our system leverages NSLS-II Bluesky for experimental data acquisition, captures analysis parameters, and enriches the search space with results from the scientific literature. We describe in detail the challenges and benefits of data-driven approaches when comparing computational and experimental data, such as the multiple possible scenarios of interest in often ill-posed inverse problems and missing data. Our infrastructure utilizes the NSLS-II event model and includes discovery mechanisms across multiple beamlines. A text and data mining module using natural language processing classification techniques extracts relevant information from the literature that is added to data sources. We describe a provenance acquisition and analysis rerun workflow based on a directed acyclic graph that keeps track of process and data provenance and allows to replay analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

2016 Floodplain Geochemistry from the East River Watershed, Colorado

This dataset includes groundwater geochemistry data from floodplain piezometers collected as a part of the Watershed Function Scientific Focus Area (SFA) located in the Upper Colorado River Basin. The data were collected in order to investigate the role of hyporheic exchange and other river corridor processes on riverine export of solutes. Floodplain piezometers installed in several meanders (Meander C, Meander D, and Meander O) in the vicinity of the Pumphouse were sampled at several dates in 2016. Data includes in-field measurements (pH, electrical conductivity, oxidation reduction potential, dissolved oxygen, dissolved Fe(II), dissolved sulfide, and groundwater level) along with laboratory measurements (dissolved inorganic carbon, dissolved organic carbon, metals and major cations). Five files are included in this dataset, including: sample locations and depths in a csv file (metadata.csv), aqueous geochemistry data in a csv file, analytical detection limits in a csv file, a file-level metadata (flmd.csv) file, and a data dictionary (dd.csv) file. Data that was below the detection limit is reported as one half of the detection limit and missing data is listed as "-9999".

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Verification and Validation Tests of Gamma Library of MC2-3 for Coupled Neutron and Gamma Heating Calculation

For the accurate assessment of the heat generation rate in fast reactors, the gamma library of MC2 -3 and the MC2 -3 + GAMSOR procedure employing the coupled neutron and gamma heating calculation has been thoroughly verified against Monte Carlo results and validated using the ZPPR-15D gamma dose measurement data. NJOY outputs are post-processed in a consistent way with the NJOY procedure to avoid any missing data or double counting of data. Prompt heating for 379 out of 391 isotopes in the gamma library was verified against MCNP6.2 to within 1% relative error in total heating for most isotopes. For both a simple one-dimensional slab problem representing a sodium cooled fast reactor and the Experimental Breeder Reactor II (EBR-II) benchmark problem, the root-mean-square values of assembly power error were less than 0.5% for fuel assemblies, ~1% in blankets and ~1 to ~3% in reflectors compared to MCNP6.2 results. The most plausible cause for the 3% error in a reflector assembly is believed to be the error in the multigroup neutron cross sections for the reflector assembly. For validation, gamma doses measured with thermoluminiscent dosimeters (TLDs) in the ZPPR-15D experiment were calculated using GAMSOR. Due to the uncertainty in the TLD measurement with regards to the energy deposition of photons and neutrons, the validation data leads to a 12.7% uncertainty on the experimental measurement. With this uncertainty bound, the calculated doses all fell within one standard deviation of the measured value. Combined with the accurate calculation of reaction rate distributions and neutron spectrum measurements, these results indicate good agreement for the neutron and gamma heating calculations that were performed.

coupled neutron and gamma heating↗
DOE OSTI2022

Random Forest Regressor-Based Approach for Detecting Fault Location and Duration in Power Systems

Power system failures or outages due to short-circuits or “faults” can result in long service interruptions leading to significant socio-economic consequences. It is critical for electrical utilities to quickly ascertain fault characteristics, including location, type, and duration, to reduce the service time of an outage. Existing fault detection mechanisms (relays and digital fault recorders) are slow to communicate the fault characteristics upstream to the substations and control centers for action to be taken quickly. Fortunately, due to availability of high-resolution phasor measurement units (PMUs), more event-driven solutions can be captured in real time. In this paper, we propose a data-driven approach for determining fault characteristics using samples of fault trajectories. A random forest regressor (RFR)-based model is used to detect real-time fault location and its duration simultaneously. This model is based on combining multiple uncorrelated trees with state-of-the-art boosting and aggregating techniques in order to obtain robust generalizations and greater accuracy without overfitting or underfitting. Four cases were studied to evaluate the performance of RFR: 1. Detecting fault location (case 1), 2. Predicting fault duration (case 2), 3. Handling missing data (case 3), and 4. Identifying fault location and length in a real-time streaming environment (case 4). A comparative analysis was conducted between the RFR algorithm and state-of-the-art models, including deep neural network, Hoeffding tree, neural network, support vector machine, decision tree, naive Bayesian, and K-nearest neighborhood. Experiments revealed that RFR consistently outperformed the other models in detection accuracy, prediction error, and processing time.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2015

Systematic Review of Radiation Therapy Toxicity Reporting in Randomized Controlled Trials of Rectal Cancer: A Comparison of Patient-Reported Outcomes and Clinician Toxicity Reporting

The use of multimodal treatments for rectal cancer has improved cancer-related outcomes but makes monitoring toxicity challenging. Optimizing future radiation therapy regimens requires collection and publication of detailed toxicity data. This review evaluated the quality of toxicity information provided in randomized controlled trials (RCTs) of radiation therapy in rectal cancer and focused on the difference between clinician-reported and patient-reported toxicity. Medline, EMBASE, and the Cochrane Library were searched (January 1995-July 2013) for RCTs reporting late toxicity in patients treated with regimens including preoperative (chemo)radiation therapy. Data on toxicity measures and information on toxicity reported were extracted using Quantitative Analyses of Normal Tissue Effects in the Clinic recommendations. International Society for Quality of Life Research standards on patient-reported outcomes (PROs) were used to evaluate the quality of patient-reported toxicity. Twenty-one RCT publications met inclusion criteria out of 4144 articles screened. All PRO studies reported higher rates of toxicity symptoms than clinician-reported studies and reported on a wider range and milder symptoms. No clinician-reported study published data on sexual dysfunction. Of the clinician-reported studies, 55% grouped toxicity data related to an organ system together (eg “Bowel”), and 45% presented data only on more-severe (grade ≥3) toxicity. In comparison, all toxicity grades were reported in 79% of PRO publications, and all studies (100%) presented individual symptom toxicity data (eg bowel urgency). However, PRO reporting quality was variable. Only 43% of PRO studies presented baseline data, 28% did not use any psychometrically validated instruments, and only 29% of studies described statistical methods for managing missing data. Analysis of these trials highlights the lack of reporting standards for adverse events and reveals the differences between clinician and patient reporting of toxicity. Recommendations for improving the quality of adverse event data collection are provided, with the aim of improving critical appraisal of outcomes for future studies.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2022

QA/QC of the East River, Colorado, discharge and geochemical time series datasets (Almont, BCC, and Pump House) to be used for modeling of hydrogeochemical balance

The following datasets were QA/QC-ed (Quality Assurance/Quality Control): 1. Brush Creek Confluence (BCC) discharge data (from Helen Malenda, USGS, Colorado School of Mines), which were calculated using the pressure transducer data and rating curves. The original 15 min time series data were presented as mean daily discharge. 2. Almont discharge data from United States Geological Survey (USGS). The original data were in 15 min time intervals, and were averaged to mean daily discharge time series. 3. Pump House discharge data as mean daily discharge (downloaded from the SFA portal). 4. BCC and Pump House chemistry data from SFA data portal and/or original spreadsheets provided by Roelof Versteeg. The following challenging QA/QC problems of the datasets were resolved: Missing data with the duration of gaps up to >1 month; Duplicated dates; Anomalies and outliers of discharge and concentrations; Time stamps of measurements of the discharge and concentrations are not aligned (hydrogeochemical balance calculations require the timestamps to be aligned). All QA/QC-ed datasets are given as csv files. The csv files were prepared using the xts files with multiple worksheets, which are also included in the data packages. Figures of the QA/QC-ed datasets are given in the jpeg and pdf formats. The QA/QC-ed datasets have been used to quantify discharge and chemical concentrations in river water in order to understand riverine exports of water and dissolved constituents in the East River watershed. These datasets served as a basis in the presentation given by P. Fox et al. at the 2021 Goldschmidt Conference.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2015

Informal Statistics Help Desk

Back by popular demand, the JSC Biostatistics Lab is offering an opportunity for informal conversation about challenges you may have encountered with issues of experimental design, analysis, data visualization or related topics. Get answers to common questions about sample size, repeated measures, violation of distributional assumptions, missing data, multiple testing, time-to-event data, when to trust the results of your analyses (reproducibility issues) and more.

Ploutz-Snyder, R. J.↗
NASA NTRS2017

A Network-Based Algorithm for Clustering Multivariate Repeated Measures Data

The National Aeronautics and Space Administration (NASA) Astronaut Corps is a unique occupational cohort for which vast amounts of measures data have been collected repeatedly in research or operational studies pre-, in-, and post-flight, as well as during multiple clinical care visits. In exploratory analyses aimed at generating hypotheses regarding physiological changes associated with spaceflight exposure, such as impaired vision, it is of interest to identify anomalies and trends across these expansive datasets. Multivariate clustering algorithms for repeated measures data may help parse the data to identify homogeneous groups of astronauts that have higher risks for a particular physiological change. However, available clustering methods may not be able to accommodate the complex data structures found in NASA data, since the methods often rely on strict model assumptions, require equally-spaced and balanced assessment times, cannot accommodate missing data or differing time scales across variables, and cannot process continuous and discrete data simultaneously. To fill this gap, we propose a network-based, multivariate clustering algorithm for repeated measures data that can be tailored to fit various research settings. Using simulated data, we demonstrate how our method can be used to identify patterns in complex data structures found in practice.

Koslovsky, Matthew↗
DOE OSTI2020

AUTONOMIE VID

Autonomie Vehicle Information Database (VID) offers a comprehensive list of vehicle specifications since 1990. The database details more than 65,000 vehicles with hundreds of attributes. The database is the result of the development of a general automated data collection framework as well as the development of building blocks for processing, cleaning, integrating and analyzing complex data. The data has undergone several layers of outlier detections processes, machine learning based imputations methods have been used to deal with missing data problems, and new fields have been created according to the rules of feature engineering. Thanks to this streamlined data pipelines, the resulting processed aggregated data should deliver a unique level of information to the user in which the content can be efficiently maintained and updated.

Moswd, Ayman↗
DOE OSTI2016

Online mapping and forecasting of epidemics using open-source indicators

Open-source indicators have been proposed as a way of tracking and forecasting disease outbreaks. Some, such are meteorological data, are readily available as reanalysis products. Others, such as those derived from our online behavior (web searches, media article etc.) are gathered easily and are more timely than public health reporting. In this study we investigate how these datastreams may be combined to provide useful epidemiological information. The investigation is performed by building data assimilation systems to track influenza in California and dengue in India. The first does not suffer from incomplete data and was chosen to explore disease modeling needs. The second explores the case when observational data is sparse and disease modeling complexities are beside the point. The two test cases are for opposite ends of the disease tracking spectrum. We find that data assimilation systems that produce disease activity maps can be constructed. Further, being able to combine multiple open-source datastreams is a necessity as any one individually is not very informative. The data assimilation systems have very little in common except that they contain disease models, calibration algorithms and some ability to impute missing data. Thus while the data assimilation systems share the goal for accurate forecasting, they are practically designed to compensate for the shortcomings of the datastreams. Thus we expect them to be disease and location-specific.

42 ENGINEERING↗
DOE OSTI2015

Optimizing Geothermal Drilling: Oil and Gas Technology Transfer

There is a significant amount of financial risk associated with geothermal drilling; however, there are opportunities to improve upon current practices and technologies used. The scope of this drilling operational study included 21 geothermal wells and 21 oil and gas wells. The goal was to determine a 'perfect well' using historical data to compare the best oil and gas well to the best geothermal well. Unfortunately, limitations encountered in the study included missing data (bit records, mud information, etc.), poor data collection, and difficult to ascertain handwriting. An online software database was used to format drilling data to IADC coded daily drilling reports and generate analysis figures. Six major issues have been found in geothermal drilling operations. These problems include lost circulation, rig/equipment selection, cementing, penetration rate, drilling program, and time management. As a result of these issues, geothermal drilling averages 56.4 days longer than drilling comparable oil and gas wells in the wells in this study. Roughly $13.9 million would be lost due to non-productive time in the 21 geothermal wells and only $1.3 million in the oil and gas wells, assuming a cost of $50,000 per day. Comparable events such as drilling the same sized hole, tripping in/out, cementing, and running the same size casing took substantially less time in the oil and gas wells. Geothermal wells were drilled using older and/or less advanced technology to depths less than 10,000 feet, while oil and gas wells reached 12,500 feet faster with purpose built rigs. A new approach is now underway that will optimize drilling programs throughout the drilling industry. It is the use of Mechanical Specific Energy (MSE) as a tool to realize efficient drilling processes. However, a work-flow must also be established in order for there to be an efficient drilling program. Potential improvements for current geothermal operations are: the use of electronic records, real time services, and official glossary terms to describe rig operations, and advanced drilling rigs/technology.

15 GEOTHERMAL ENERGY↗
DOE OSTI2015

Optimizing Geothermal Drilling: Oil and Gas Technology Transfer

There is a significant amount of financial risk associated with geothermal drilling. This study of drilling operations seeks opportunities to improve upon current practices and technologies. The scope of this study included analyzing 21 geothermal wells and 21 oil and gas wells. The goal was to determine a 'Perfect Well' using historical data to compare the best oil and gas well to the best geothermal well. Unfortunately, limitations encountered in the study included missing data (bit records, mud information, etc.) and poor data collection practices An online software database was used to format drilling data to IADC coded daily drilling reports and generate figures for analysis. Six major issues have been found in geothermal drilling operations. These problems include lost circulation, rig/ equipment selection, cementing, penetration rate, drilling program, and time management. As a result of these issues, geothermal drilling averaged 56.4 days longer than drilling comparable oil and gas wells in the wells in this study. Roughly $13.9 million was spent on non-productive time in the 21 geothermal wells, compared with only $1.3 million in the oil and gas wells, assuming a cost of $50,000 per day. Comparable events such as drilling the same sized hole, tripping in/out, cementing, and running the same size casing took substantially less time in the oil and gas wells. Geothermal wells were drilled using older and/or less advanced technology to depths less than 10,000 feet, while oil and gas wells reached 12,500 feet faster with purpose built rigs. A new approach is now underway that will optimize drilling programs throughout the drilling industry using Mechanical Specific Energy (MSE) as a tool to realize efficient drilling processes. Potential improvements for current geothermal operations are: the use of electronic records, real time services, and official glossary terms to describe rig operations, and advanced drilling rigs/technology.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

An iterative bidirectional gradient boosting approach for CVR baseline estimation

Here this paper presents a novel Iterative Bidirectional Gradient Boosting Model (IBi-GBM) for estimating the baseline of Conservation Voltage Reduction (CVR) programs. In contrast to many existing methods, we treat CVR baseline estimation as a missing data retrieval problem. The approach involves dividing the load and its corresponding temperature profiles into three periods: pre-CVR, CVR, and post-CVR. To restore the missing load profile during the CVR period, the method employs a three-step process. First, a forward-pass GBM is executed using data from the pre-CVR period as inputs. Subsequently, a backward-pass GBM is applied using data from the post-CVR period. The two restored load profiles are reconciled, considering pre-calculated weights derived from forecasting accuracy, and only the leftmost and rightmost points are retained. The newly restored points are then included as inputs for the subsequent iteration. This iterative procedure continues until the original load data in the CVR period is fully restored. We develop IBi-GBM using actual smart meter and Supervisory Control and Data Acquisition (SCADA) data. Our results demonstrate that IBi-GBM exhibits robust performance across various data resolutions and in different seasons and outperforms existing methods by achieving a 1-2% reduction in normalized Root Mean Square Error (nRMSE).

42 ENGINEERING↗