Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Noisy data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

mystic : software for autonomous discovery and design under uncertainty

Throughout the diverse range of science and engineering applications, there is a growing desire to develop computational methods that can reliably predict the behavior of complex systems. Specifically, there is a strategic need for tools that can robustly forecast the behavior of complex physical systems, where data may be high-dimensional, noisy, or sparse, and models of the system may be time-dependent or include uncertainty. We use mystic to build tools that leverage statistical learning, physics-informed learning, and active learning in the efficient generation of reliably predictive surrogates for complex physical systems. mystic is a robust, proven, open-source optimization and uncertainty quantification toolkit with over a decade of use in the design and optimization of neutron instrumentation, solar-powered drones, and gasguns, and in iterative tuning of models for Raman spectroscopy and elastoplastic materials strength. Recent developments have focused on automated learning of statistically robust surrogates under uncertainty, with applications in materials in extreme environments, nanostructures, materials simulations and strength models, and the failure of shielding under particle radiation. In 2020, McKerns demonstrated active learning of optimally robust surrogates with respect to new simulated data for molecular dynamics simulations of materials mixing in warm dense matter, and is currently applying active learning to the automated steering of particle accelerator beams and the optimal design and control of quantum optical sensor instrumentation.

42 ENGINEERING↗
DOE OSTI2022

Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5

Abstract. There is growing interest in data-driven weather prediction (DDWP), e.g., using convolutional neural networks such as U-NET that are trained on data from models or reanalysis. Here, we propose three components, inspired by physics, to integrate with commonly used DDWP models in order to improve their forecast accuracy. These components are (1) a deep spatial transformer added to the latent space of U-NET to capture rotation and scaling transformation in the latent space for spatiotemporal data, (2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and (3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit and feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the spatial-transformer-based U-NET (U-STN) clearly outperforms the U-NET, e.g., improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. This DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promise; for example, it reduces the average error by factors of 2–3. These results show the benefits and feasibility of these three components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the three components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast and assimilation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

AmeriFlux US-GL1 Stannard Rock

This is the AmeriFlux version of the carbon flux data for the site US-GL1 Stannard Rock. Site Description - Stannard Rock is located 39 km from the nearest shore (Keweenaw Peninsula) in Lake Superior, 44 miles NNE of Marquette, Michigan, and 24 miles ESE of Manitou Island. The site is located on the historic Stannard Rock Lighthouse, which was completed in 1882. Eddy covariance instrumentation was installed in 2008 by a network of scientists from both US and Canada, eventually to be called the Great Lakes Evaporation Network (GLEN). The intent of GLEN has been to provide observations of over-lake meteorology and evaporation, improve forecasting of Great Lakes water levels, and support a wide variety of stakeholders, including the National Weather Service (NWS), Environment and Climate Change Canada, National Oceanic and Atmospheric Administration, U.S. Coast Guard, recreational boaters and commercial shipping, emergency management officials, and the Great Lakes research community. The eddy covariance station along with other ancillary meteorological instrumentation is located at an approximate elevation of 39.2 meters above mean lake water level. Meteorological data from the lighthouse are sent to the National Data Buoy Center, where they can be viewed in real-time at http://www.ndbc.noaa.gov/station_page.php?station=stdm4 Uncorrected half-hour fluxes were computed directly on the logger using a 30-minute block averaging period as high-frequency data were not available. The half hour flux measurements downloaded from the datalogger were post-processed with the following filters and corrections. Latent and sensible heat and carbon dioxide fluxes were corrected with 2-D coordinate rotation. Although the primary objective of data collection at Stannard Rock was to quantify the evaporative flux, additional preliminary measurements of the carbon dioxide concentration and flux from the LI-7500 are also included in this dataset but it is advised to use the carbon data with caution. Carbon data reported in this dataset includes turbulent fluxes of CO2 with no storage correction (FC, µmol m-2 s-1) and CO2 density in mole fraction of wet air (CO2), which was originally output on the datalogger as average CO2 density (mg m-2 s-1) and converted into µmol mol-1 using air temperature and pressure in post-processing. Webb, Pearman, and Leuning terms were applied to account for density fluctuations for water vapor and CO2. Sonic path length, high-frequency attenuation and sensor separation were accounted for according to Horst and Massman, and the oxygen absorption correction for the KH2O hygrometer was also applied. Latent and sensible heat fluxes were assumed to be unrealistic above an absolute value of 1000 W m-2 and were removed. Both carbon flux (FC) and carbon dioxide mole fraction in wet air (CO2) and were assumed to be unrealistic above 1000 µmol m-2 s-1 and 1000 µmol mol-1 respectively. Spikes in latent and sensible heat and carbon fluxes and densities (often due to periods of precipitation) were identified by computing the mean and standard deviation over a moving, overlapping window of 336 half-hours (7 days), similar to Shao et al., and were removed when the flux was more than 1.5 standard deviations from the moving window’s mean. While Vickers and Mahrt use a threshold of 3.5 standard deviations from the mean, a conservative value of 1.5 was chosen due to the noisy nature of over-lake data at this particular site. This process was repeated twice for latent and sensible heat, and carbon dioxide fluxes and densities and therefore it is possible that some real, realistic data was filtered out in this process. No detrending was performed. As per AmeriFlux standards, no friction velocity (USTAR, m s-1) filtering was performed.

Spence, Chris↗
DOE OSTI2025

AmeriFlux FLUXNET-1F US-GL1 Stannard Rock

This is the AmeriFlux Management Project (AMP) created FLUXNET-1F version of the carbon flux data for the site US-GL1 Stannard Rock. This is the FLUXNET version of the carbon flux data for the site US-GL1 Stannard Rock produced by applying the standard ONEFlux (1F) software. Site Description - Stannard Rock is located 39 km from the nearest shore (Keweenaw Peninsula) in Lake Superior, 44 miles NNE of Marquette, Michigan, and 24 miles ESE of Manitou Island. The site is located on the historic Stannard Rock Lighthouse, which was completed in 1882. Eddy covariance instrumentation was installed in 2008 by a network of scientists from both US and Canada, eventually to be called the Great Lakes Evaporation Network (GLEN). The intent of GLEN has been to provide observations of over-lake meteorology and evaporation, improve forecasting of Great Lakes water levels, and support a wide variety of stakeholders, including the National Weather Service (NWS), Environment and Climate Change Canada, National Oceanic and Atmospheric Administration, U.S. Coast Guard, recreational boaters and commercial shipping, emergency management officials, and the Great Lakes research community. The eddy covariance station along with other ancillary meteorological instrumentation is located at an approximate elevation of 39.2 meters above mean lake water level. Meteorological data from the lighthouse are sent to the National Data Buoy Center, where they can be viewed in real-time at http://www.ndbc.noaa.gov/station_page.php?station=stdm4 Uncorrected half-hour fluxes were computed directly on the logger using a 30-minute block averaging period as high-frequency data were not available. The half hour flux measurements downloaded from the datalogger were post-processed with the following filters and corrections. Latent and sensible heat and carbon dioxide fluxes were corrected with 2-D coordinate rotation. Although the primary objective of data collection at Stannard Rock was to quantify the evaporative flux, additional preliminary measurements of the carbon dioxide concentration and flux from the LI-7500 are also included in this dataset but it is advised to use the carbon data with caution. Carbon data reported in this dataset includes turbulent fluxes of CO2 with no storage correction (FC, µmol m-2 s-1) and CO2 density in mole fraction of wet air (CO2), which was originally output on the datalogger as average CO2 density (mg m-2 s-1) and converted into µmol mol-1 using air temperature and pressure in post-processing. Webb, Pearman, and Leuning terms were applied to account for density fluctuations for water vapor and CO2. Sonic path length, high-frequency attenuation and sensor separation were accounted for according to Horst and Massman, and the oxygen absorption correction for the KH2O hygrometer was also applied. Latent and sensible heat fluxes were assumed to be unrealistic above an absolute value of 1000 W m-2 and were removed. Both carbon flux (FC) and carbon dioxide mole fraction in wet air (CO2) and were assumed to be unrealistic above 1000 µmol m-2 s-1 and 1000 µmol mol-1 respectively. Spikes in latent and sensible heat and carbon fluxes and densities (often due to periods of precipitation) were identified by computing the mean and standard deviation over a moving, overlapping window of 336 half-hours (7 days), similar to Shao et al., and were removed when the flux was more than 1.5 standard deviations from the moving window’s mean. While Vickers and Mahrt use a threshold of 3.5 standard deviations from the mean, a conservative value of 1.5 was chosen due to the noisy nature of over-lake data at this particular site. This process was repeated twice for latent and sensible heat, and carbon dioxide fluxes and densities and therefore it is possible that some real, realistic data was filtered out in this process. No detrending was performed. As per AmeriFlux standards, no friction velocity (USTAR, m s-1) filtering was performed.

Spence, Chris [Environment and Climate Change Cana↗
DOE OSTI2020

Automated Identification of Molecular Crystals’ Packing Motifs

Packing motifs—patterns in how molecules orient relative to one another in a crystal structure—are an important concept in many subdisciplines of materials science because of correlations observed between specific packing motifs and properties of interest. That said, packing motif data sets have remained small and noisy due to intensive manual labeling processes and insufficient labeling schemes. The most prominent labeling algorithms calculate relative interplanar angles of nearest neighbor molecules to determine the packing motif of a molecular crystal, but this simple approach can fail when neighbors are naively sampled isotropically around the crystal structure. To remedy this issue, here we propose an optimization algorithm, which rotates the molecular crystal structure to find representative molecules that inform the packing motif. We package this algorithm into an automated framework—Autopack—which both optimally rotates the crystal structure and labels the packing motif based on the appropriate neighboring molecules. In this work, we detail the Autopack framework and its performance, which shows improvements compared to previous state-of-the-art labeling methods, providing the first quantitative point of comparison for packing motif labeling algorithms. Furthermore, using Autopack (available at https://ipo.llnl.gov/technologies/software/autopack), we perform the first large-scale study of potential relationships between chemicals’ compositions and packing motifs, which shows that these relationships are more complex than previously hypothesized from studies that used only tens of polycyclic aromatic hydrocarbon molecules. Autopack’s capabilities help pose next steps for crystal engineering research focusing not only on a molecule’s adoption of a specific packing motif but also on new structure–property relationships.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Measurements of Single Event Upset in ATLAS IBL

Effects of Single Event Upsets (SEU) and Single Event Transients (SET) are studied in the FE-I4B chip of the innermost layer of the ATLAS pixel system. SEU/SET affect the FE-I4B Global Registers as well as the settings for the individual pixels, causing, among other things, occupancy losses, drops in the low voltage currents, noisy pixels, and silent pixels. Quantitative data analysis and simulations indicate that SET dominate over SEU on the load line of the memory. Operational issues and mitigation techniques are presented.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Evaluation of a generalized least squares algorithm for infrasound beamforming with coherent background noise

Infrasonic signals of interest can occur during periods with persistent, coherent, background noise, which may be natural or anthropogenic. For high signal-to-noise (SNR) ratio transient signals, an ‘overprinting’ of the coherent background may occur, and the signal may still be detected. However, this approach fails for low SNR signals of interest, which may be obscured by coherent noise. An infrasound beamforming method based on generalized least squares (GLS) is investigated for detecting transient signals of interest in the presence of coherent and incoherent background noise. This approach relies on an estimate of the noise covariance, captured in a covariance matrix, to effectively null contributions to the array response from noisy directions of arrival. Synthetic array data is used to investigate the performance of the GLS beamformer compared to the Bartlett beamformer when coherent and incoherent backgrounds are present. Additionally, the effects of array element number and relative strength of the interfering signal on the GLS estimates is investigated. GLS empirical area under the curve estimates suggest that the beamformer can recover coherent power for a signal of interest lower in amplitude than the coherent background, but this effectiveness degrades more quickly with SNR for a four element array compared to a six or eight element infrasound array. Finally, infrasound from the Forensic Surface Experiment, a bolide signal observed at IMS array I37NO, and a volcanic signal recorded at the Alaska Volcano Observatory array ADKI are used to evaluate GLS performance on recorded data. A ten minute window was used to capture the background noise, and the coherent background signal was nulled in all three examples.

58 GEOSCIENCES↗
NASA NTRS1972

Oblique shock-sound interaction at a freestream Mach number of about 20 in helium.

Detailed experimental measurements of the characteristics and level of pre- and postshock disturbances obtained for typical disturbance modes and shock angles are presented to aid in the evaluation of transition studies conducted in 'noisy' hypersonic wind tunnels. The presented data suggest that the shock is probably generating total temperature fluctuations in a fashion directly related to shock strength. Since in high supersonic and hypersonic flows the total temperature fluctuations are mainly contained in velocity fluctuations, it may be concluded that the predominant disturbances generated are vorticity as long as the postshock flow is at a sufficiently high Mach number.-

Weinstein, L. M.↗
NASA NTRS1974

Results from cascade thrust reverser noise and suppression experiments

Results from experimental work on model scale cascade reversers with cold airflow are presented. Sound power level directivity and spectral characteristics for cascade reversers are reported. Effect of cascade exit area ratio, vane profile shape, and emission arc are discussed. Model equivalent diameters varied from 3 to 5 inches, pressure ratios range from 1.15 to 3.0. Depending on the reverser type, acoustic power was proportional to the 4 1/2 to 6th power of ideal jet velocity. Reverser noise peaked at higher frequency and was more omnidirectional than nozzle-alone jet noise. Appreciable reduction in sideline noise was obtained from plane shields. Airfoil-vaned cascades were the most aerodynamically efficient and least noisy reversers. Scaling of cascade reverser data to example aircraft engines showed all cascades above the 95 PNdB sideline goal from STOL aircraft. However, the airfoil-vaned reverser has a good potential for meeting this goal for high-bypass (low pressure ratio) exhausts.

Gutierrez, O. A.↗
NASA NTRS1980

Shuttle Ku-band and S-band communications implementation study

Various aspects of the shuttle orbiter S-band network communication system, the S-band payload communication system, and the Ku-band communication system are considered. A method is proposed for obtaining more accurate S-band antenna patterns of the actual shuttle orbiter vehicle during flight because the preliminary antenna patterns using mock-ups are not realistic that they do not include the effects of additional appendages such as wings and tail structures. The Ku-band communication system is discussed especially the TDRS antenna pointing accuracy with respect to the orbiter and the modifications required and resulting performance characteristics of the convolutionally encoded high data rate return link to maintain bit synchronizer lock on the ground. The TDRS user constraints on data bit clock jitter and data asymmetry on unbalanced QPSK with noisy phase references are included. The S-band payload communication system study is outlined including the advantages and experimental results of a peak regulator design built and evaluated by Axiomatrix for the bent-pipe link versus the existing RMS-type regulator. The nominal sweep rate for the deep-space transponder of 250 Hz/s, and effects of phase noise on the performance of a communication system are analyzed.

Dodds, J. G.↗
NASA NTRS1986

Measurements of lower stratospheric/upper tropospheric water vapor by the SAGE II instrument

Preliminary, unvalidated data are discussed from early measurements of water vapor profiles in the upper troposphere/lower stratosphere by means of SAGE II satellite sensors. Zonal means for April 1985 are discussed and compared with previous, separate data sets for water vapor profiles for the 100 mbar and 300 mbar levels determined from space-based IR and rawinsonde data. Techniques employed to correct for noisy measurements due to the presence of clouds in the SAGE II scenes are described, noting the slant path transmission methods applied to generate the water vapor profiles.

Larsen, J. C.↗
NASA NTRS1988

Pattern-based fault diagnosis using neural networks

An architecture for a real-time pattern-based diagnostic expert system capable of accommodating noisy, incomplete, and possibly erroneous input data is outlined. Results from prototype systems applied to jet and rocket engine fault diagnosis are presented. The ability of a neural network-based system to be trained via the presentation of behavioral patterns associated with fault conditions is demonstrated.

Dietz, W. E.↗
NASA NTRS1991

Coordinating sensing and local navigation

Based on Navigation Templates (or NaTs), this work presents a new paradigm for local navigation which addresses the noisy and uncertain nature of sensor data. Rather than creating a new navigation plan each time the robot's perception of the world changes, the technique incorporates perceptual changes directly into the existing navigation plan. In this way, the robot's navigation plan is quickly and continuously modified, resulting in actions that remain coordinated with its changing perception of the world.

Slack, Marc G.↗
NASA NTRS1995

An Experiment in Determining Software Reliability Model Applicability

There have been few reports on the behavior of software reliability models under controlled conditions. That is to say, most of the reported experience with the models is during the testing phase of actual projects, during which researchers have little or no control over the data with which they work. Give that failure data for actual projects can be noisy, distorted, and uncertain, reported procedures for determining model applicability may be incomplete.

Software Reliability↗
NASA NTRSDate not supplied

The WFIRST CGI Integral Field Spectrograph: Requirements and Performance Predictions

The WFIRST coronagraphic instrument (CGI) will demonstrate exoplanet spectroscopy using an integral field spectrograph (IFS). The CGI IFS, being designed and built at Goddard Space Flight Center, has a spectral resolution of R50 and is designed to accommodate a 20% bandpass spanning 600- 970 nm. The IFS is principally targeting the abundance of Methane features, with the primary coronagraph band being centered around 770nm. Key to the performance estimates are the achievable signal-to-noise (SNR) ratios and the stability of the microspectra over the course of tens and hundreds of hours. As a technology demonstration for CGI, the ability to close a wavefront control loop around the IFS, maintain a stable dark hole, and provide time resolved data that simultaneously spans spatial and spectral dimensions are crucial demonstrations for future observatories. The IFS is optimized both for coronagraphs and science observations with a potential future starshade. We highlight how the long duration observations, and requirements for both starshades and coronagraphs drive the IFS requirements and the calibrations required both onorbit and on the ground. We also provide further detail on the optomechanical design, its stability based on thermal and structural predictions, anticipated performance, and operations concept of the CGI IFS. The impact of these performance metrics are projected into simulated data products, demonstrating cube extraction of noisy images and the subsequent planet spectrum that can be extracted from them. These demonstrations and performance predictions are key to future missions such as LUVOIR and HabEx, whose principal science case relies on efficient spectroscopy of exoplanets.

Tyler Groff↗
DOE OSTI2025

Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNet

Modern data-driven surrogate models for weather forecasting provide accurate short-term predictions but inaccurate and nonphysical long-term forecasts. This paper investigates online weather prediction using machine learning surrogates supplemented with partial and noisy observations. We empirically demonstrate and theoretically justify that, despite the long-time instability of the surrogates and the sparsity of the observations, filtering estimates can remain accurate in the long-time horizon. As a case study, we integrate the Fourier Forecasting Neural Network (FourCastNet), a weather surrogate model, within a variational data assimilation framework using partial, noisy ERA5 global reanalysis data from the European Centre for Medium-Range Weather Forecasts (ECMWF). Here, our results show that filtering estimates remain accurate over a year-long assimilation window and provide effective initial conditions for forecasting tasks, including extreme event prediction.

Data assimilation↗
DOE OSTI2026

Learning genetic perturbation effects with variational causal inference

Advances in sequencing technologies have enhanced the understanding of gene regulation in cells. In particular, Perturb-seq has enabled high-resolution profiling of the transcriptomic response to genetic perturbations at the single-cell level. This understanding has implications in functional genomics and potentially for identifying therapeutic targets. Various computational models have been developed to predict perturbational effects. While deep learning models excel at interpolating observed perturbational data, they tend to overfit in the lack of enough data and may not generalize well to unseen perturbations. In contrast, mechanistic models, such as linear causal models based on gene regulatory networks, hold greater potential for extrapolation, as they encapsulate regulatory information that can predict responses to unseen perturbations. However, their application has been limited to small studies due to overly simplistic assumptions, making them less effective in handling noisy, large-scale single-cell data. We propose a hybrid approach that combines a mechanistic causal model with variational deep learning, termed Single Cell Causal Variational Autoencoder (SCCVAE). The mechanistic model employs a learned regulatory network to represent perturbational changes as shift interventions that propagate through the learned network. SCCVAE integrates this mechanistic causal model into a variational autoencoder, generating rich, comprehensive transcriptomic responses. Our results indicate that SCCVAE exhibits superior performance over current state-of-the-art baselines for extrapolating to predict unseen perturbational responses. Additionally, for the observed perturbations, the latent space learned by SCCVAE allows for the identification of functional perturbation modules and simulation of single-gene knockdown experiments of varying penetrance, presenting a robust tool for interpreting and interpolating perturbational responses at the single-cell level.

59 BASIC BIOLOGICAL SCIENCES↗