SEARCH · Engineering Papers

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Gaseous Ammonia Measurements at the S2 site for the CoURAGE campaign

This dataset contains measurements of gaseous ammonia (NH3(g)) at the S2 (Mt. Airy) site during the CoURAGE study from April to June 2025. This dataset includes 10-minute and 30-minute average NH3(g) concentrations in units of micrograms per cubic meter. Measurements are conducted with an AiRRmonia NH3 analyzer manufactured by RR Mechatronics. A data flag of -999 indicates missing data due to the instrument being offline (maintenance, calibration, etc.) while a data flag of -888 indicates measurements below the instrument LOD (LOD = 0.1 micrograms per cubic meter). The uncertainty of the measurements is 15%.

AiRRmonia NH3 analyzer↗

DOE OSTI2018

Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) Winter Intensive Operational Period: Aerodynamic Particle Sizer Field Campaign Report

An aerodynamic particle sizer spectrometer (APS, model 3321 by TSI Inc., Shoreview, Minnesota) was deployed at the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility Eastern North Atlantic (ENA) observatory during the second (winter) intensive operational period (IOP) of the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. Data were collected from January 9 through February 26, 2018, covering the second deployment of the ARM Aerial Facility (AAF) aircraft within ACE-ENA. The standard ENA instrument suite does not provide measurements of aerosol particle size distribution for diameters greater than one micron. Properties of coarse-mode (super-micron) aerosols are rarely measured, although super-micron particles can play an important role in the aerosol life cycle, affecting aerosol optical properties and cloud nuclei activity (e.g., Clarke et al. 2003). The APS was deployed to fill this gap in particle size measurements: it is capable of measuring size-resolved concentration of the aerosols in the 0.5-20 μm range (aerodynamic diameter). The APS was installed in the aerosol trailer to sample through the main stack. Special attention was given to selection of the instrument location (just below the main stack) to minimize particle losses in the connecting lines. Most atmospheric aerosols are hygroscopic, so the measurement of the aerosol size distributions should either be performed in a controlled humidity environment or accompanied by concurrent measurement of the temperature and humidity of the sample air. The latter strategy was used during this deployment: a high-precision humidity/temperature transmitter (model EE08 by E+E Elektonik GMBH, Engerwitzdorf, Germany) was mounted in the APS sampling line immediately upstream of the APS inlet. A dedicated laptop computer was used for data collection and communications. The APS was controlled with vendor-supplied software (Aerosol Instrument Manager, ver. 10.1, TSI); aerosol size distribution data were collected over a period of 55 seconds and converted to one-minute averages. Temperature and humidity analog signals were acquired via analog-to-digital converter (ADC) module (model WTAIN-M by Weeder Technologies, Fort Walton Beach, Florida) using vendor-supplied software (ModCom) with 1- sec resolution. In post-processing, the APS data were exported from TSI proprietary format into “comma-separated value” American Standard Code for Information Interchange (ASCII) files and matched with appropriately averaged temperature and humidity data. Two significant periods of missing data were caused by local power outages. Vendor-supplied software was not designed for unattended operation: it needs to be manually restarted including sensor configuration setup.

ACE-ENA, Azores, ultra-high-sensitivity aerosol sp↗

DOE OSTI2014

SU-E-T-524: Web-Based Radiation Oncology Incident Reporting and Learning System (ROIRLS)

Purpose: Describe a Web-based Radiation Oncology Incident Reporting and Learning system that has the potential to improve quality of care for radiation therapy patients. This system is an important facet of continuing effort by our community to maintain and improve safety of radiotherapy.Material and Methods: The VA National Radiation Oncology Program office has embarked on a program to electronically collect adverse events and near miss data of radiation treatment of over 25,000 veterans treated with radiotherapy annually. Software used for this program is deployed on the VAs intranet as a Website. All data entry forms (adverse event or near miss reports, work product reports) utilize standard causal, RT process step taxonomies and data dictionaries defined in AAPM and ASTRO reports on error reporting (AAPM Work Group Report on Prevention of Errors and ASTROs safety is no accident report). All reported incidents are investigated by the radiation oncology domain experts. This system encompasses the entire feedback loop of reporting an incident, analyzing it for salient details, and developing interventions to prevent it from happening again. The operational workflow is similar to that of the Aviation Safety Reporting System. This system is also synergistic with ROSIS and SAFRON. Results: The ROIRLS facilitates the collection of data that help in tracking adverse events and near misses and develop new interventions to prevent such incidents. The ROIRLS electronic infrastructure is fully integrated with each registered facility profile data thus minimizing key strokes and multiple entries by the event reporters. Conclusions: OIRLS is expected to improve the quality and safety of a broad spectrum of radiation therapy patients treated in the VA and fulfills our goal of Effecting Quality While Treating Safely The Radiation Oncology Incident Reporting and Learning System software used for this program has been developed, conceptualized and maintained by TSG Innovations Inc. and is deployed on the VA intranet as a Website. The Radiation Oncology Incident Reporting and Learning System software used for this program has been developed, conceptualized and maintained by TSG Innovations Inc. and is deployed on the VA intranet as a Website.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2015

SU-E-T-469: Implementation of VAs Web-Based Radiotherapy Incident Reporting and Analysis System (RIRAS)

Purpose: This Web-based Radiotherapy Incident Reporting and Analysis System (RIRAS) is a tool to improve quality of care for radiation therapy patients. This system is an important facet of continuing effort by our community to maintain and improve safety of radiotherapy.Material and Methods: VA’s National Radiation Oncology Program office has embarked on a program to electronically collect adverse events and good-catch data of radiation treatment of over 25,000 veterans treated with radiotherapy annually. This VA-Intranet based software design has made use of dataset taxonomies and data dictionaries defined in AAPM/ASTRO reports on error reporting. We used proven industrial and medical event reporting techniques to avoid several common problems faced in effective data collection such as incomplete data due to data entry fatigue by the reporters, missing data due to data difficult to obtain or not familiar to most reporters, missing reports due to fear of reprisal etc. This system encompasses the entire feedback loop of reporting an incident, analyzing it for salient details, and developing interventions to prevent it from happening again. The analysis reports with corrective, learning actions are shared with the reporter/facility and made public to the community (after deidentification) as part of the learning process. Results: Till date 50 incident/good catches have been reported in RIRAS and we have completed analysis on 100% of these reports. This is done due to the fact that each reported incidents is investigated and a complete analysis/patient-safety-work-product report is generated by radiation oncology domain-experts. Conclusions Because of the completeness of the data, the system has enabled us to analyze process steps and track trends of major errors which in the future will lead to implementing system wide process improvement steps and safe standard operating procedures for each radiotherapy treatment modality/technique and fulfills our goal of “Effecting Quality While Treating Safely”. RIRAS developed and copyrighted by TSG Innovations Inc.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2020

Generalized Canonical Polyadic Tensor Decomposition

Tensor decomposition is a fundamental unsupervised machine learning method in data science, with applications including network analysis and sensor data processing. This work develops a generalized canonical polyadic (GCP) low-rank tensor decomposition that allows other loss functions besides squared error. For instance, we can use logistic loss or Kullback--Leibler divergence, enabling tensor decomposition for binary or count data. We present a variety of statistically motivated loss functions for various scenarios. We provide a generalized framework for computing gradients and handling missing data that enables the use of standard optimization methods for fitting the model. Furthermore, we demonstrate the flexibility of the GCP decomposition on several real-world examples including interactions in a social network, neural activity in a mouse, and monthly rainfall measurements in India.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

How robust are estimates of key parameters in standard viral dynamic models?

Mathematical models of viral infection have been developed, fitted to data, and provide insight into disease pathogenesis for multiple agents that cause chronic infection, including HIV, hepatitis C, and B virus. However, for agents that cause acute infections or during the acute stage of agents that cause chronic infections, viral load data are often collected after symptoms develop, usually around or after the peak viral load. Consequently, we frequently lack data in the initial phase of viral growth, i.e., when pre-symptomatic transmission events occur. Missing data may make estimating the time of infection, the infectious period, and parameters in viral dynamic models, such as the cell infection rate, difficult. However, having extra information, such as the average time to peak viral load, may improve the robustness of the estimation. Here, we evaluated the robustness of estimates of key model parameters when viral load data prior to the viral load peak is missing, when we know the values of some parameters and/or the time from infection to peak viral load. Although estimates of the time of infection are sensitive to the quality and amount of available data, particularly pre-peak, other parameters important in understanding disease pathogenesis, such as the loss rate of infected cells, are less sensitive. Viral infectivity and the viral production rate are key parameters affecting the robustness of data fits. Fixing their values to literature values can help estimate the remaining model parameters when pre-peak data is missing or limited. We find a lack of data in the pre-peak growth phase underestimates the time to peak viral load by several days, leading to a shorter predicted growth phase. On the other hand, knowing the time of infection (e.g., from epidemiological data) and fixing it results in good estimates of dynamical parameters even in the absence of early data. While we provide ways to approximate model parameters in the absence of early viral load data, our results also suggest that these data, when available, are needed to estimate model parameters more precisely.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗

DOE OSTI2021

Cleaning Images with Gaussian Process Regression

Many approaches to astronomical data reduction and analysis cannot tolerate missing data: corrupted pixels must first have their values imputed. This paper presents astrofix, a robust and flexible image imputation algorithm based on Gaussian process regression. Through an optimization process, astrofix chooses and applies a different interpolation kernel to each image, using a training set extracted automatically from that image. It naturally handles clusters of bad pixels and image edges and adapts to various instruments and image types. For bright pixels, the mean absolute error of astrofix is several times smaller than that of median replacement and interpolation by a Gaussian kernel. We demonstrate good performance on both imaging and spectroscopic data, including the SBIG 6303 0.4 m telescope and the FLOYDS spectrograph of Las Cumbres Observatory and the CHARIS integral-field spectrograph on the Subaru Telescope.

42 ENGINEERING↗

DOE OSTI2018

A Kriging based spatiotemporal approach for traffic volume data imputation

Along with the rapid development of Intelligent Transportation Systems, traffic data collection technologies have progressed fast. The emergence of innovative data collection technologies such as remote traffic microwave sensor, Bluetooth sensor, GPS-based floating car method, and automated license plate recognition, has significantly increased the variety and volume of traffic data. Despite the development of these technologies, the missing data issue is still a problem that poses great challenge for data based applications such as traffic forecasting, real-time incident detection, dynamic route guidance, and massive evacuation optimization. A thorough literature review suggests most current imputation models either focus on the temporal nature of the traffic data and fail to consider the spatial information of neighboring locations or assume the data follow a certain distribution. These two issues reduce the imputation accuracy and limit the use of the corresponding imputation methods respectively. As a result, this paper presents a Kriging based data imputation approach that is able to fully utilize the spatiotemporal correlation in the traffic data and that does not assume the data follow any distribution. A set of scenarios with different missing rates are used to evaluate the performance of the proposed method. The performance of the proposed method was compared with that of two other widely used methods, historical average and K-nearest neighborhood. Comparison results indicate that the proposed method has the highest imputation accuracy and is more flexible compared to other methods.

Science & Technology - Other Topics↗

DOE OSTI2015

SHELXT – Integrated space-group and crystal-structure determination

SHELXT automates routine small-molecule structure determination starting from single-crystal reflection data, the Laue group and a reasonable guess as to which elements might be present. The new computer program SHELXT employs a novel dual-space algorithm to solve the phase problem for single-crystal reflection data expanded to the space group P1. Missing data are taken into account and the resolution extended if necessary. All space groups in the specified Laue group are tested to find which are consistent with the P1 phases. After applying the resulting origin shifts and space-group symmetry, the solutions are subject to further dual-space recycling followed by a peak search and summation of the electron density around each peak. Elements are assigned to give the best fit to the integrated peak densities and if necessary additional elements are considered. An isotropic refinement is followed for non-centrosymmetric space groups by the calculation of a Flack parameter and, if appropriate, inversion of the structure. The structure is assembled to maximize its connectivity and centred optimally in the unit cell. SHELXT has already solved many thousand structures with a high success rate, and is optimized for multiprocessor computers. It is, however, unsuitable for severely disordered and twinned structures because it is based on the assumption that the structure consists of atoms.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2022

A Review of the Use of Wearables in Indoor Environmental Quality Studies and an Evaluation of Data Accessibility from a Wearable Device

An understanding of indoor environmental quality (IEQ) and its effects on occupant well-being can inform building system design and operation. The use of wearables in field studies to collect subjective and objective health performance indicators (HPIs) from a large number of occupants could deliver important improvements in IEQ. To facilitate the use of wearables in IEQ studies, there is a need to identify which HPIs should be collected and to evaluate data accessibility from these devices. To address this issue, a literature review of previous IEQ studies was conducted to identify relationships between different IEQ factors and HPIs, with a focus on HPIs that were collected using wearables. A preliminary assessment of data accessibility from a selected wearable device (Fitbit Versa 2) was performed and documented. The review suggested the need to further investigate and collect sleep quality parameters, heart rate, stress response, as well as subjective ratings of comfort using wearables. The data accessibility assessment revealed issues related to missing data points and data resolution from the examined device. A set of recommendations is outlined to inform future studies.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2018

AAF_UHSAS-air_CACTI

The UHSAS-air is the bench version of the UHSAS that measures particle size from 0.06 to 1.0 micro-meters. Aerosol passes through a laser beam and the resulting intensity of scattered light is directly correlated to the diameter of the particle passing through the beam. This isntrument has been modified to volumetricaly control the Sheath Flow using an Alicat flow controller. Data is reported at 1.000s and missing data is reported as -9999 Multiple data flags are present in this version. Counts?: The coincident error is deemed minimal until the counts exceed 4000. Counts between 3000 and 4000 are flagged with a caution flag (1). After 4000 counts a bad flag (2) is triggered and the data should not be used. Noise?: Instrument noise detected in 65-75nm size range. 1: Caution slight noise observed. 2: High amount of noise observed data should not be used. Total? Sum of all the flags triggered Max Flag: Max flag triggered by the above QC test CloudFlag: Warns of the presence of a cloud. Clouds can cause shattering on the probe and result in questionable measurements. Data during these periods of thin clouds (1) or heavy clouds (2) should be used with extreme caution.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2018

3D Visualisation of Un-georeferenced, Coordinate-poor Spatial Data in Complex D and D and WM Environments - 18294

This paper describes the development of an algorithm-based utility tool using Visual Basic that examines un-georeferenced or GPS-deprived spatial datasets (3D datasets lacking one or more of the key x, y and z components). Taking into account any known geometrical parameters which are input via a user-friendly front-end, the new code automatically evaluates attributes, including any spatial coordinate components, detects invalid survey data points and fills in bad or missing data points utilising data-specific interpolation (e.g. cubic or linear spline) of valid data points, such that local extrema are not created. Previously troublesome 2D data can be easily converted to realistic 3D contour display, integrated with other time-series data to create 4D movies, and incorporated into other existing 3D datasets. More importantly those processed and 'more complete' data can be easily stored in 3D or 4D spatial database management systems for future interrogation by using clear, realistic and intuitive 3D visualisation. Test cases are presented to demonstrate the value and versatility of the newly developed utility tool which include the 3D visualisation of: surveyed wall thicknesses of a helical coil, radiation mapping of a plant and aerial survey mapping of radiation in a region near Fukushima. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2023

Pair distribution function analysis of nano-object assemblies

The atomic pair distribution function (aPDF) analysis technique, also known as the total scattering method, which considers both Bragg and diffuse scattering, has been used extensively to probe local atomic arrangements in crystalline and disordered materials. In contrast, there have been limited applications of the PDF in self-assembled nanomaterials, which represent a class of materials built from nanoscale objects, such as nano-colloids, micelles and proteins. As distinguished from atoms, nano-objects have polydispersity in size and shape, and such form-factor effects complicate the application of PDF analysis to nano-systems. Here, in this paper, the application of the PDF is extended to spherical nano-object assemblies and the formulae for the nano-PDF (nPDF) are derived, showing some differences from the aPDF. By numerical simulations, the properties of the nPDF (peak broadening and pattern profile) are studied systematically as a function of structural features, such as nano-object parameters (size and size polydispersity) and assembly structural features (size, shape, structure type and lattice disorder), and of data processing parameters ( q cut-off and `missing' data in ultra-small-angle regions). The nPDF analysis method is found to provide an effective route to revealing not only nanoscale but also mesoscale structural properties, for example the morphology of a nano-assembly.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

DOE OSTI2015

Tall Tower Wind Energy Monitoring and Numerical Model Validation in Northern Nevada

The main objectives of this project were to conduct a tall-tower and sodar field campaign in complex terrain, investigate wind properties relevant to wind energy assessment, and evaluate high-resolution models with fixed and adaptive grid structures. Two 60-m towers at Virginia Peak ridges near Washoe Valley, Nevada, were instrumented with cup and vane anemometers as well as sonic anemometers, and an acoustic sounder (hereafter sodar) was installed near one of the towers. The towers were located 2,700 m apart with a vertical distance of 140 m elevation between their bases. Each tower had a downhill exposure of rolling complex terrain, with the nearby valley floor 3,200 m to the west and 800 m below the summit. Cup anemometers were installed at both towers at 20, 40, and 60 m, wind vanes at 20 and 60 m, and sonic anemometers at 20 and 60 m. The sodar measurements were nominally provided every 10 m in vertical distance from 40 to 200 m with the quality of the data generally decreasing with height. Surface air temperature, atmospheric pressure, and radiation measurements were conducted at 1.5 m AGL at both of the towers. Although the plan was to conduct a 1-year period of data collection, we extended the period (October 5, 2012 through February 24, 2014) to cover for possible data loss from instrument or communication problems. We also present a preliminary analysis of the towers and sodar data, including a detailed inventory of available and missing data as well as outliers. The analysis additionally includes calculation of the Weibull parameters, turbulence intensity, and initial computation of wind power density at various heights.

17 WIND ENERGY↗

DOE OSTI2022

Accounting for uncertainty sources in pyrometric measurements: Variables affecting Surface Temperature measurements using the LLNL Streak Spectrometers

Here we present a detailed analysis of the error sensitivities of the LLNL temperature analysis package used in analyzing streak spectrometer data. These sensitivities include noise, temperature, wavelength, emissivity, missing bands, and missing data points. The analysis package was tested for its ability to accurately reconstruct randomly generated noisy data and found to perform within specifications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2016

A SPATIOTEMPORAL APPROACH FOR HIGH RESOLUTION TRAFFIC FLOW IMPUTATION

Along with the rapid development of Intelligent Transportation Systems (ITS), traffic data collection technologies have been evolving dramatically. The emergence of innovative data collection technologies such as Remote Traffic Microwave Sensor (RTMS), Bluetooth sensor, GPS-based Floating Car method, automated license plate recognition (ALPR) (1), etc., creates an explosion of traffic data, which brings transportation engineering into the new era of Big Data. However, despite the advance of technologies, the missing data issue is still inevitable and has posed great challenges for research such as traffic forecasting, real-time incident detection and management, dynamic route guidance, and massive evacuation optimization, because the degree of success of these endeavors depends on the timely availability of relatively complete and reasonably accurate traffic data. A thorough literature review suggests most current imputation models, if not all, focus largely on the temporal nature of the traffic data and fail to consider the fact that traffic stream characteristics at a certain location are closely related to those at neighboring locations and utilize these correlations for data imputation. To this end, this paper presents a Kriging based spatiotemporal data imputation approach that is able to fully utilize the spatiotemporal information underlying in traffic data. Imputation performance of the proposed approach was tested using simulated scenarios and achieved stable imputation accuracy. Moreover, the proposed Kriging imputation model is more flexible compared to current models.

Han, Lee↗

DOE OSTI2018

FUSRAP Case Study - Benefits and Lessons Learned from Implementation of a Modern Data Solution at Maywood - 18399

This paper focuses on data production and management practices dating back to 1999 that continue to evolve to support the project today. In 2000, the FUSRAP Maywood Project began using a software environmental data management system and laboratory information management system, developed by a single database administrator (DA) and authored in Microsoft Access{sup TM} (MS Access). The original databases included basic functionality including sample number assignment and label generation, tracking of LIMS gamma spec QC and assignment of basic qualifiers and basic database queries. The DA continued over the next fifteen years to refine the EDMS/LIMS by integrating project-specific data evaluation guidance document processes and Quality Assurance/Quality Control (QA/QC) charting and excursion warning tools. With the retirement of the original DA, the U.S. Army Corps of Engineers (USACE)-Cabrera Team procured and implemented a commercial-leaning solution using modern database tools and an established EDMS vendor. For the database transition process, the project team established and configured the new EDMS/LIMS with basic architecture and accessibility; evaluated existing workflows; established/revised LIMS and EDMS-related documentation (manuals and procedures); and conducted dry runs and validation testing. The overall team coordinated closely to facilitate a seamless transition from initial assessment and authoring, to data migration, and finally to resumption of normal ongoing operations. Key features of the new EDMS Subcontractor's software included lookup tables, an import wizard, data checking options and unique indexes. Since the database systems became operational, the technical staff gathered useful feedback from end users (e.g., lab staff, field engineers, task managers, and the third-party data validator) to refine the EDMS/LIMS operation. This feedback triggered improvements in the performance of onsite laboratory qualification forms, promoted consistency in sample ID numbers between the database sample planning screens and field chain-of-custodies, corrected bugs in the LIMS Duplicate Evaluation Report, eliminated data import issues such as blank, inconsistent or missing data from key fields. One of the advantages of the commercial database solution was that it allowed the project to cut cost over time due to the lower cost of database maintenance provided by the commercial database vendor versus the cost of an on-staff DA. Cost savings are estimated at $40,000 per year. There were also indirect cost savings resulting from implementation of the new database since it allowed better integration of the LIMS and EDMS. While there will always be challenges associated with long term records/data management for any old remediation site, the Maywood Team's approach to an upgraded EDMS/LIMS implementation serves as one element of the overall effort to enhance the integrity and reproducibility of critical data that supports the remediation program while at the same time, more effectively using available resources. These lessons-learned will enhance implementation of cost-effective solutions for similar environmental remediation sites. (authors)

97 MATHEMATICS AND COMPUTING↗