SEARCH · Engineering Papers

Results for “Synthetic Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Pycheron: A Python-Based Seismic Waveform Data Quality Control Software Package

Supplementing an existing high-quality seismic monitoring network with openly available station data could improve coverage and decrease magnitudes of completeness; however, this can present challenges when varying levels of data quality exist. Without discerning the quality of openly available data, using it poses significant data management, analysis, and interpretation issues. Incorporating additional stations without properly identifying and mitigating data quality problems can degrade overall monitoring capability. If openly available stations are to be used routinely, a robust, automated data quality assessment for a wide range of quality control (QC) issues is essential. To meet this need, we developed Pycheron, a Python-based library for QC of seismic waveform data. Pycheron was initially based on the Incorporated Research Institutions for Seismology’s Modular Utility for STAtistical kNowledge Gathering but has been expanded to include more functionality. Pycheron can be implemented at the beginning of a data processing pipeline or can process stand-alone data sets. Its objectives are to (1) identify specific QC issues; (2) automatically assess data quality and instrumentation health; (3) serve as a basic service that all data processing builds on by alerting downstream processing algorithms to any quality degradation; and (4) improve our ability to process orders of magnitudes more data through performance optimizations. This article provides an overview of Pycheron, its features, basic workflow, and an example application using a synthetic QC data set.

58 GEOSCIENCES↗

DOE OSTI2022

Tsunami Early Warning From Global Navigation Satellite System Data Using Convolutional Neural Networks

Abstract We investigate the potential of using Global Navigation Satellite System (GNSS) observations to directly forecast full tsunami waveforms in real time. We train convolutional neural networks to use less than 9 min of GNSS data to forecast the full tsunami waveforms over 6 hr at select locations, and obtain accurate forecasts on a test data set. Our training and test data consists of synthetic earthquakes and associated GNSS data generated for the Cascadia Subduction Zone using the MudPy software, and corresponding tsunami waveforms in Puget Sound computed using GeoClaw. We use the same suite of synthetic earthquakes and waveforms as in earlier work where tsunami waveforms were used for forecasting, and provide a comparison. We also explore varying the number of GNSS stations, their locations, and their observation durations.

Rim, Donsub↗

NASA NTRS2001

Calibration of Sea Ice Motion from QuikSCAT with those from SSM/I and Buoy

QuikSCAT backscatter and DMSP SSM/I radiance data are used to derive sea ice motion for both the Arctic and Antarctic region using wavelet analysis method. This technique provides improved spatial coverage over the existing array of Arctic Ocean buoys and better temporal resolution over techniques utilizing satellite data from Synthetic Aperture Radar (SAR). Sea ice motion of the Arctic for the period from October 1999 to March 2000 derived from QuikSCAT and SSM/I data agrees well with that derived from ocean buoys quantitatively. Thus the ice tracking results from QuikSCAT and SSM/I are complement to each other, Then, three sea-ice drift daily results from QuikSCAT, SSM/I, and buoy data can be merged to generate composite maps with more complete coverage of sea ice motion than those from single data source. A series of composite sea ice motion maps for December 1999 show that the major circulation patterns of sea ice motion are changing and shifting significantly within every four days and they are dominated by wind forcing. Sea-ice drift in the summer can not be derived from NSCAT and SSM/I data. In later summer of 1999 (in September), however, QuikSCAT data can provide good sea ice motion information in the Arctic. QuiksCAT can also provide at least partial sea ice motion information until June 15 in early summer 1999. For the Antarctic, case study shows that sea ice motion derived from QuikSCAT data is predominantly forced by and is consistent with wind field derived from QuikSCAT around the polar region. These calibrated/validated results indicate that QuikSCAT, SSM/I, and buoy merged daily ice motion are suitably accurate to identify and closely locate sea ice processes, and to improve our current knowledge of sea ice drift and related processes through the data assimilation of ocean-ice numerical model.

Liu, Antony K.↗

NASA NTRSDate not supplied

Target Tracking with Distributed Sensing and Optimal Data Migration

The paper presents an Extended Kalman Filter based framework for airborne target tracking using adaptive information fusion from multi-modal multi-rate distributed sensors network. First, the tracking algorithm execution location is determined using an optimal data migration strategy, which also computes the associated delays for each sensor data to arrive at the computing location. Next, the fast (zero-delay) sensors information is dynamically fused in the filter correction procedure at the arrival instance of each valid sensor reading. Finally, the target estimation is updated based on the valid slow (delayed) data, which are grouped according to the delay-time steps before application of the Larsen's method. This approach is applied to the synthetic sensor data generated by means of the ground based radar and camera models for the simulated target flight in Reflection simulation environment.

Distributed sensing↗

NASA NTRSDate not supplied

Target Tracking with Distributed Sensing and Optimal Data Migration

Distributed sensing↗

DOE OSTI2024

Coincident learning for unsupervised anomaly detection of scientific instruments

Abstract Anomaly detection is an important task for complex scientific experiments and other complex systems (e.g. industrial facilities, manufacturing), where failures in a sub-system can lead to lost data, poor performance, or even damage to components. While scientific facilities generate a wealth of data, labeled anomalies may be rare (or even nonexistent), and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called coincident learning for anomaly detection (CoAD), which is specifically designed for multi-modal tasks and identifies anomalies based on coincident behavior across two different slices of the feature space. We define an unsupervised metric, F ^ β , out of analogy to the supervised classification F β statistic. CoAD uses F ^ β to train an anomaly detection algorithm on unlabeled data , based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and our motivating task of identifying RF station anomalies in a particle accelerator.

43 PARTICLE ACCELERATORS↗

DOE OSTI2020

Efficient 3-D velocity model building using joint inline and crossline plane-wave wave-equation migration velocity analyses

SUMMARY Wave-equation migration velocity analysis (WEMVA) is an image-domain inversion method for velocity model building. Automatic plane-wave WEMVA (PWEMVA) calculates the moveouts of plane-wave common-image gathers (CIGs) by searching a best-fitting parabola with semblance analysis and backprojects residual CIG moveouts into wavefield wave paths with a reflection tomographic kernel. However, 3-D PWEMVA is very computationally expensive because 3-D reflection tomographic inversion requires at least five 3-D reverse-time migrations per iteration and stores two types of source wavefields at model boundaries. We develop a joint inline and crossline PWEMVA method for efficient 3-D velocity model building. We alternatively implement the inline and crossline PWEMVAs with a constraint for each other, in which we iteratively construct the 3-D velocity model update through 1-D spline interpolation of 2-D gradients. The inline and crossline joint inversion is practical since PWEMVA only inverts for low-wavenumber velocity perturbations along wave paths, and the method can take less than 1 per cent of the computational cost of full 3-D PWEMVA. To construct unaliased plane waves for our joint inline and crossline PWEMVA, we develop a 3-D data interpolation method in the frequency–wavenumber (FK) domain to recover regularly and randomly missing traces. The method minimizes the misfit on sufficiently localized data subsets with iterative optimal step lengths and a gradient preconditioner that iteratively selects dominant dips along different azimuths. In numerical experiments, we use a 3-D synthetic seismic data set and a land 3-D field seismic data set acquired at the Farnsworth CO2-EOR (enhanced oil recovery) field to demonstrate the efficacy of our velocity model building and data interpolation methods.

Liu, Xuejian↗

NASA NTRS1994

ASF archive issues: Current status, past history, and questions for the future

The Alaska SAR Facility (ASF) collects, processes, archives, and distributes data from synthetic aperture radar (SAR) satellites in support of scientific research. ASF has been in operation since 1991 and presently has an archive of over 100 terabytes of data. ASF is performing an analysis of its magnetic tape storage system to ensure long-term preservation of this archive. Future satellite missions have the possibility of doubling to tripling the amounts of data that ASF acquires. ASF is examining the current data systems and the high volume storage, and exploring future concerns and solutions.

Goula, Crystal A.↗

DOE OSTI2024

Generation and validation of comprehensive synthetic weather histories using auto-regressive moving-average models

As energy system design moves to more complex methods of optimization including machine learning there is a significant need for more weather data than is available. One method to solve this is using synthetic data models such as the auto-regressive moving-average (ARMA) model which has been frequently utilized to create such data. This paper looks at extending the ARMA algorithm to generate solar components through the use of clearsky detrending, maintaining vector relationships and by leveraging physical relationships. The method for the creation of entirely synthetic weather data files including key weather variables for energy system analysis is presented. Furthermore, a detailed comparison of energy system simulations utilizing both real and synthetic data is made using NREL’s System Advisor Model. Whilst good agreement is made for the solar variables, and other weather variables, ARMA methods often fail to capture the standard deviation and skew of annual weather distributions. Vector-ARMA is shown to maintain correlations between variables and thus generate data sets that perform similarly in energy system design. Here, it is finally shown that the ARMA method fails to preserve day-today correlations in weather variables and thus over-predicts optimal energy storage by 21% for a residential solar application.

42 ENGINEERING↗

NASA NTRS2013

A Parameter Estimation Scheme for Multiscale Kalman Smoother (MKS) Algorithm Used in Precipitation Data Fusion

A new approach is presented in this paper to effectively obtain parameter estimations for the Multiscale Kalman Smoother (MKS) algorithm. This new approach has demonstrated promising potentials in deriving better data products based on data of different spatial scales and precisions. Our new approach employs a multi-objective (MO) parameter estimation scheme (called MO scheme hereafter), rather than using the conventional maximum likelihood scheme (called ML scheme) to estimate the MKS parameters. Unlike the ML scheme, the MO scheme is not simply built on strict statistical assumptions related to prediction errors and observation errors, rather, it directly associates the fused data of multiple scales with multiple objective functions in searching best parameter estimations for MKS through optimization. In the MO scheme, objective functions are defined to facilitate consistency among the fused data at multiscales and the input data at their original scales in terms of spatial patterns and magnitudes. The new approach is evaluated through a Monte Carlo experiment and a series of comparison analyses using synthetic precipitation data. Our results show that the MKS fused precipitation performs better using the MO scheme than that using the ML scheme. Particularly, improvements are significant compared to that using the ML scheme for the fused precipitation associated with fine spatial resolutions. This is mainly due to having more criteria and constraints involved in the MO scheme than those included in the ML scheme. The weakness of the original ML scheme that blindly puts more weights onto the data associated with finer resolutions is overcome in our new approach.

multiscale↗

NASA NTRS2022

P- and L-Band Retrieval of Subsurface Soil Moisture and Temperature Profiles as First-Order Polynomial Function

This paper demonstrates the potential use of P and L band passive measurements to determine root zone soil moisture (SM) and soil temperature(ST). SM and ST data have been taken as a function of depth during the NASA GSFC PLEX19 experiment in the summer of 2019 at Beltsville, MD, USA. Using these data, a coherent model has been used to compute H and V brightness temperatures at frequencies of 0.8 and 1.4 GHz with an observation angle of 35 degrees. These synthetic brightness data are then used to estimate the SM and ST profiles which are represented by linear polynomials. The inversion problem is formulated as a least square problem that is solved by a global optimization method known as the Adaptive Simulated Annealing(ASA) method. Four inversion examples having different SM and ST profiles are presented. Selected results show that the standard deviation between the retrieved and measured data is less than 0.077 cm3/cm3 for SM, and 2.245 °C for ST.

Ming Li↗

DOE OSTI2021

Neural network for 3D inertial confinement fusion shell reconstruction from single radiographs

In inertial confinement fusion (ICF), x-ray radiography is a critical diagnostic for measuring implosion dynamics, which contain rich three-dimensional (3D) information. Traditional methods for reconstructing 3D volumes from 2D radiographs, such as filtered backprojection, require radiographs from at least two different angles or lines of sight (LOS). In ICF experiments, the space for diagnostics is limited, and cameras that can operate on fast timescales are expensive to implement, limiting the number of projections that can be acquired. To improve the imaging quality as a result of this limitation, convolutional neural networks (CNNs) have recently been shown to be capable of producing 3D models from visible light images or medical x-ray images rendered by volumetric computed tomography. Here, we propose a CNN to reconstruct 3D ICF spherical shells from single radiographs. We also examine the sensitivity of the 3D reconstruction to different illumination models using preprocessing techniques such as pseudo-flatfielding. To resolve the issue of the lack of 3D supervision, we show that training the CNN utilizing synthetic radiographs produced by known simulation methods allows for reconstruction of experimental data as long as the experimental data are similar to the synthetic data. We also show that the CNN allows for 3D reconstruction of shells that possess low mode asymmetries. Further comparisons of the 3D reconstructions with direct multiple LOS measurements are justified.

3D Reconstruction↗

DOE OSTI2020

Validation of finite-element models using full-field experimental data: Levelling finite-element analysis data through a digital image correlation engine

Full-field data from digital image correlation (DIC) provide rich information for finite-element analysis (FEA) validation. However, there are several inherent inconsistencies between FEA and DIC data that must be rectified before meaningful, quantitative comparisons can be made, including strain formulations, coordinate systems, data locations, strain calculation algorithms, spatial resolutions and data filtering. As such, in this paper, we investigate two full-field validation approaches: (1) the direct interpolation approach, which addresses the first three inconsistencies by interpolating the quantity of interest from one mesh to the other, and (2) the proposed DIC-levelling approach, which addresses all six inconsistencies simultaneously by processing the FEA data through a stereo-DIC simulator to ‘level’ the FEA data to the DIC data in a regularisation sense. Synthetic ‘experimental’ DIC data were generated based on a reference FEA of an exemplar test specimen. The direct interpolation approach was applied, and significant strain errorswere computed, even though therewas no model form error, because the filtering effect of theDIC enginewas neglected. In contrast, the levelling approach provided accurate validation results, with no strain error when no model form error was present. Next, model form error was purposefully introduced via a mismatch of boundary conditions. With the direct interpolation approach, the mismatch in boundary conditions was completely obfuscated, while with the levelling approach, it was clearly observed. Finally, the ‘experimental’ DIC datawere purposefully misaligned slightly fromthe FEA data. Both validation techniques suffered from the misalignment, thus motivating continued efforts to develop a robust alignment process. In conclusion, direct interpolation is insufficient, and the proposed levelling approach is required to ensure that the FEA and the DIC data have the same spatial resolution and data filtering. Only after the FEA data have been ‘levelled’ to the DIC data can meaningful, quantitative error maps be computed.

42 ENGINEERING↗

DOE OSTI2021

Directional Laplacian Centrality for Cyber Situational Awareness

Cyber operations is drowning in diverse, high-volume, multi-source data. To get a full picture of current operations and identify malicious events and actors, analysts must see through data generated by a mix of human activity and benign automated processes. Although many monitoring and alert systems exist, they typically use signature-based detection methods. We introduce a general method rooted in spectral graph theory to discover patterns and anomalies without a priori knowledge of signatures. We derive and propose a new graph-theoretic centrality measure based on the derivative of the graph Laplacian matrix in the direction of a vertex. To build intuition about our measure, we show how it identifies the most central vertices in standard network datasets and compare to other graph centrality measures. Finally, we focus our attention on studying its effectiveness in identifying important IP addresses in network flow data. Using both real and synthetic network flow data, we conduct several experiments to test our measure’s sensitivity to two types of injected attack profiles and show that vertices participating in injected attack profiles exhibit noticeable changes in our centrality measures, even when the injected anomalies are relatively small, and in the presence of simulated network dynamics.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2022

Differentially Private K -Means Clustering Applied to Meter Data Analysis and Synthesis

The proliferation of smart meters has resulted in a large amount of data being generated. It is increasingly apparent that methods are required for allowing a variety of stakeholders to leverage the data in a manner that preserves the privacy of the consumers. The sector is scrambling to define policies, such as the so called ‘15/15 rule’, to respond to the need. However, the current policies fail to adequately guarantee privacy. Here, in this paper, we address the problem of allowing third parties to apply K-means clustering, obtaining customer labels and centroids for a set of load time series by applying the framework of differential privacy. We leverage the method to design an algorithm that generates differentially private synthetic load data consistent with the labeled data. We test our algorithm’s utility by answering summary statistics such as average daily load profiles for a 2-dimensional synthetic dataset and a real-world power load dataset.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Active oversight and quality control in standard Bayesian optimization for autonomous experiments

The fusion of experimental automation and machine learning has catalyzed a new era in materials research, prominently featuring Gaussian Process (GP) Bayesian Optimization (BO) driven autonomous experiments. Here we introduce a Dual-GP approach that enhances traditional GPBO by adding a secondary surrogate model to dynamically constrain the experimental space based on real-time assessments of the raw experimental data. This Dual-GP approach enhances the optimization efficiency of traditional GPBO by isolating more promising space for BO sampling and more valuable experimental data for primary GP training. We also incorporate a flexible, human-in-the-loop intervention method in the Dual-GP workflow to adjust for unanticipated results. We demonstrate the effectiveness of the Dual-GP model with synthetic model data and implement this approach in autonomous pulsed laser deposition experimental data. This Dual-GP approach has broad applicability in diverse GPBO-driven experimental settings, providing a more adaptable and precise framework for refining autonomous experimentation for more efficient optimization.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Polarimetric Geometric Modeling for mm-VLBI Observations of Black Holes

The Event Horizon Telescope (EHT) is a millimeter very long baseline interferometry (VLBI) array that has imaged the apparent shadows of the supermassive black holes M87* and Sagittarius A*. Polarimetric data from these observations contain a wealth of information on the black hole and accretion flow properties. In this work, we develop polarimetric geometric modeling methods for mm-VLBI data, focusing on approaches that fit data products with differing degrees of invariance to broad classes of calibration errors. We establish a fitting procedure using a polarimetric “m-ring” model to approximate the image structure near a black hole. By fitting this model to synthetic EHT data from general relativistic magnetohydrodynamic models, we show that the linear and circular polarization structure can be successfully approximated with relatively few model parameters. We then fit this model to EHT observations of M87* taken in 2017. In total intensity and linear polarization, the m-ring fits are consistent with previous results from imaging methods. In circular polarization, the m-ring fits indicate the presence of event-horizon-scale circular polarization structure, with a persistent dipolar asymmetry and orientation across several days. The same structure was recovered independently of observing band, used data products, and model assumptions. Despite this broad agreement, imaging methods do not produce similarly consistent results. Our circular polarization results, which imposed additional assumptions on the source structure, should thus be interpreted with some caution. Polarimetric geometric modeling provides a useful and powerful method to constrain the properties of horizon-scale polarized emission, particularly for sparse arrays like the EHT.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2022

Analysis of the MODIS Above-Cloud Aerosol Retrieval Algorithm Using MCARS

The Multi-sensor Cloud and Aerosol Retrieval Simulator (MCARS) presently produces synthetic radiance data from Goddard Earth Observing System version 5 (GEOS-5) model output as if the Moderate Resolution Imaging Spectroradiometer (MODIS) was viewing a combination of atmospheric column inclusive of clouds, aerosols and a variety of gases and land/ocean surface at a specific location. In this paper we use MCARS to study the MODIS Above-Cloud AEROsol retrieval algorithm (MOD06ACAERO). MOD06ACAERO is presently a regional research algorithm able to retrieve aerosol optical thickness over clouds, in particular absorbing biomass burning aerosols overlying marine boundary layer clouds in the Southeastern Atlantic Ocean. The algorithm’s ability to provide aerosol information in cloudy conditions makes it a valuable source of information for modeling and climate studies n an area where current clear sky-only operational MODIS aerosol retrievals effectively have a data gap between the months of June and October. We use MCARS for a verification and closure study of the MOD06ACAERO algorithm. The purpose of this study is to develop a set of constraints a model developer might use during assimilation of MOD06ACAERO data. Our simulations indicate that the MOD06ACAERO algorithm performs well for marine boundary layer clouds in the SE Atlantic provided some specific screening rules are observed. For the present study, a combination of five simulated MODIS data granules was used for a dataset of 13.5 million samples with known input conditions. When pixel retrieval uncertainty was less than 30%, optical thickness of the underlying cloud layer was greater than 4 and scattering angle range within the cloud bow was excluded, MOD06ACAERO retrievals agreed with the underlying ground truth (GEOS-5 cloud and aerosol profiles used to generate the synthetic radiances) with a slope of 0.913, offset of 0.06, and RMSE=0.107. When only near-nadir pixels were considered (view zenith angle within +/-20 degrees) the agreement with source data further improved (0.977, 0.051 and 0.096 respectively). Algorithm closure was examined using a single case out of the five 38 used for verification. For closure, the MOD06ACAERO code was modified to use GEOS-5 temperature and moisture profiles as ancillary. Agreement of MOD06ACAERO retrievals with source data for the closure study had a slope of 0.996 with offset -0.007 and RMSE of 0.097 at pixel uncertainty level of less than 40%, illustrating the benefits of high-quality ancillary atmospheric data for such retrievals.

MODIS↗