Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “matrix completion”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Low-rank Tensor Completion for PMU Data Recovery

This paper proposes a tensor completion method for the recovery of missing phasor measurement unit (PMU) measurements. Tensor completion as the general case of matrix completion has attracted increasing attention in recent years. The imputation accuracy for the existing matrix completion methods may be significantly reduced when there are consecutive data losses across multiple data channels. To tackle this issue, we explore the multi-way characteristics of PMU measurements by using a tensor model. We leverage the low-rank property of the PMU measurements and formulate the missing PMU data recovery problem as a low-rank tensor completion problem. An efficient algorithm based on alternating direction method of multipliers (ADMM) is developed to solve the tensor completion problem. The experiments using the real PMU dataset show that the proposed method exhibits better imputation accuracy compared with the conventional data recovery methods.

Ghasemkhani, Amir↗
DOE OSTI2021

Analyzing the Effects of Cyberattacks on Distribution System State Estimation

Key components of power systems—such as energy management systems, automatic generation control, and state estimation—are under serious vulnerability from cyberattacks. Cyber threats in electric grids have increased significantly because of the increased interconnectivity of supervisory control and data acquisition systems and public network infrastructure. As the penetration level of distributed energy resources increases, it is imperative to employ system-monitoring techniques such as state estimation for the reliable operation of distribution systems. Recently, multiple methods have been developed that exploit the low rank property of distribution system state matrix and are robust to bad data, such as matrix completion. This paper analyzes the impact of various realistic cyberattack scenarios on matrix completion. Realistic cyberattack scenarios are converted into data corruption models that are used in an extensive simulation of a custom IEEE 123-bus system.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2020

Analyzing the Effects of Cyberattacks on Distribution System State Estimation: Preprint

Key components of power systems—such as energy management systems, automatic generation control, and state estimation—are under serious vulnerability from cyber attacks. Cyber threats in electric grids have increased significantly because of the increased interconnectivity of supervisory control and data acquisition systems and public network infrastructure. As the penetration level of distributed energy resources increases, it is imperative to employ system-monitoring techniques such as state estimation for the reliable operation of distribution systems. Recently, multiple methods have been developed that exploit the low rank property of distribution system state matrix and are robust to bad data, such as matrix completion. This paper analyzes the impact of various realistic cyber attack scenarios on matrix completion. Realistic cyber attack scenarios are converted into data corruption models that are used in an extensive simulation of a custom IEEE 123-bus system.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2022

A non-commutative Bayes' theorem

Using a diagrammatic reformulation of Bayes' theorem, we provide a necessary and sufficient condition for the existence of Bayesian inference in the setting of finite-dimensional C* -algebras. In other words, we prove an analogue of Bayes' theorem in the joint classical and quantum context. Our analogue is justified by recent advances in categorical probability theory, which have provided an abstract formulation of the classical Bayes' theorem. In the process, we further develop non-commutative almost everywhere equivalence and illustrate its important role in non-commutative Bayesian inversion. The construction of such Bayesian inverses, when they exist, involves solving a positive semidefinite matrix completion problem for the Choi matrix. This gives a solution to the open problem of constructing Bayesian inversion for completely positive unital maps acting on density matrices that do not have full support. In conclusion, we illustrate how the procedure works for several examples relevant to quantum information theory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Enhanced Tensor Completion Based Approaches for State Estimation in Distribution Systems

Grid state estimation is essential for effective control and management of distribution systems. While weighted least squares has been the conventional method for state estimation, sparsity-aware methods have become popular due to their superior performance with limited data. Matrix completion and compressed sensing-based state estimation approaches exploit the underlying smoothness in the state variables. However, classic matrix completion methods do not take into account the temporal correlation of system states. Compressed sensing methods, on the other hand, require an appropriate choice of sparsifying basis that may not be easy to identify. This paper proposes a blocktensor completion based framework which uses an alternative approach to estimate voltage phasor, power injections and branch currents. This approach utilizes the temporal correlation of the system states in a tensor trace-norm minimization formulation with power flow equations as constraints. Herein, feature scaling is introduced in the problem formulation to benefit from the improved sensitivity of the tensor trace norm to the matrix columns in the scaled unfoldings of the tensor. Weighted tensor norm is utilized to exploit the structures of the different unfoldings of the state measurement tensor to improve the voltage estimation. The estimation accuracy is further improved by alternatively estimating the tensor columns and increasing the available data at each stage in the tensor completion process. The proposed methods are evaluated on the IEEE-33, 37 test systems and a 100- node test system. The proposed methods are shown to provide significant performance gains relative to the classic matrix and tensor completion based approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Smart Scattering Scanning Near-Field Optical Microscopy

Scattering scanning near-field optical microscopy (s-SNOM) provides spectroscopic imaging from molecular to quantum materials with few nanometer deep subdiffraction limited spatial resolution. However, in its conventional implementation s-SNOM is slow to effectively acquire a series of spatio-spectral images, especially with large fields of view. This problem is further exacerbated for weak resonance contrast or when using light sources with limited spectral irradiance. Indeed, the generally limited signal-to-noise ratio prevents sampling a weak signal at the Nyquist sampling rate. Here, we demonstrate how acquisition time and sampling rate can be significantly reduced by using compressed sampling, matrix completion, and adaptive random sampling, while maintaining or even enhancing the physical or chemical image content. We use fully sampled real data sets of molecular, biological, and quantum materials as ground-truth physical data and show how deep under-sampling with a corresponding reduction of acquisition time by 1 order of magnitude or more retains the core s-SNOM image information. We demonstrate that a sampling rate of up to 6× smaller than the Nyquist criterion can be applied, which would provide a 30-fold reduction in the data required under typical experimental conditions. Furthermore, our smart s-SNOM approach is generally applicable and provides systematic full spatio-spectral s-SNOM imaging with a large field of view at high spectral resolution and reduced acquisition time.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Correlated purification for restoring 𝑁-representability in quantum simulation

Experimentally measured reduced density matrices (RDMs) often violate constraints that ensure they represent N-electron states—known as N-representability conditions—because of statistical and hardware noise. In this work, we present a correlated purification framework based on semidefinite programming to restore the accuracy of a noisy, unphysical two-electron RDM (2-RDM). The method performs a bi-objective optimization that minimizes both the many-electron energy and the nuclear norm of the correction to the measured 2-RDM. The nuclear norm, often employed in matrix completion, promotes low-rank corrections, while the energy term acts as a regularization term that can improve the purity of the ground state. While the method is particularly effective for ground states, it can also be applied to excited and nonstationary states by decreasing the weight of the energy relative to the error norm. In an application to fermionic shadow tomography of large hydrogen chains, correlated purification yields substantial reductions in both energy and 2-RDM error, achieving chemical accuracy across dissociation curves. This framework provides a robust strategy for tomography in many-body quantum simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Recent Advances toward Efficient Calculation of Higher Nuclear Derivatives in Quantum Chemistry

In this article, we provide an overview of state-of-the-art techniques that are being developed for efficient calculation of second and higher nuclear derivatives of quantum mechanical (QM) energy. Calculations of nuclear Hessians and anharmonic terms incur high costs and memory and scale poorly with system size. Three emerging classes of methods—machine learning (ML), automatic differentiation (AD), and matrix completion (MC)—have demonstrated promise in overcoming these challenges. We illustrate studies that employ unsupervised ML methods to reduce the need for multiple Hessian calculations in dynamics simulations and those that utilize supervised ML to construct approximate potential energy surfaces and estimate Hessians and anharmonic terms at reduced cost. By extension, if electronic structure operations could be written in a manner similar to functions underlying ML methods, rapid differentiation or AD routines can be employed to inexpensively calculate higher arbitrary-order derivatives. While ML approaches are typically black-box, we describe methods such as compressed sensing (CS) and MC, which explicitly leverage problem-specific mathematical properties of higher derivatives such as sparsity and low-rank, to complete higher derivative information using only a small, incomplete sample. The three classes of methods facilitate reliable predictions of observables ranging from infrared spectra to thermal conductivity and constitute a promising way forward in accurately capturing otherwise intractable higher-order responses of QM energy to nuclear perturbations.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2021

Multi Time-scale Imputation aided State Estimation in Distribution System

With the transition to a smart grid, we are witnessing a significant growth in sensor deployments and smart metering infrastructure in the distribution system. However, information from these sensors and meters are typically unevenly sampled at different time-scales and are incomplete. It is critical to effectively aggregate these information sources for situational awareness. In order to reconcile the heterogeneous multi-scale time-series data, we present a multi-task Gaussian process framework. This framework exploits the spatio-temporal correlation across the time-series data to impute data at any desired timescale while providing confidence bounds on the imputations. The value of the imputed data for distribution system operation is illustrated via a matrix completion based state estimation strategy. Results on the IEEE 37 bus distribution system reveals the superior performance of the proposed approach relative to linear interpolation approaches.

Dahale, Shweta↗
DOE OSTI2022

Voltage Violation Prediction in Unobservable Distribution Systems

Recently, distributed energy resources (DERs) such as photovoltaic (PV) systems have garnered significant attention due to their economic and environmental benefits. However, DERs can also pose new technical challenges to distribution system operation including under/over voltage issues. In this regard, voltage violation prediction (VVP) becomes an essential component of system operation as it enables proactive control strategies. Unfortunately, classical voltage monitoring techniques assume full availability of state measurements across all nodes in the system. In real-world scenarios, distribution systems are limited with few measurement devices, rendering the system unobservable. Therefore, this paper proposes a new Bayesian matrix completion (BMC) based VVP technique that accurately predicts the probability of nodal voltage violations in unobservable (and unbalanced) distribution systems. The proposed approach is tested via simulations on the IEEE 37 test system. Results show that the proposed method offers over 90% violation prediction accuracy with as low as 50% fraction of available data.

Abujubbeh, Mohammad↗
DOE OSTI2022

Recursive Gaussian Process over graphs for Integrating Multi-timescale Measurements in Low-Observable Distribution Systems

The transition to a smarter grid is empowered by enhanced sensor deployments and smart metering infrastructure in the distribution system. Measurements from these sensors and meters can be used for many applications, including distribution system state estimation (DSSE). However, these measurements are typically sampled at different rates and could be intermittent due to losses during the aggregation process. These multi timescale measurements should be reconciled in real-time to perform accurate grid monitoring. This paper tackles this problem by formulating a recursive multi-task Gaussian process (RGP-G) approach that sequentially aggregates sensor measurements. Specifically, we formulate a recursive multi-task GP with and without network connectivity information to reconcile the multi time-scale measurements in distribution systems. Here, the proposed framework is capable of aggregating the multi-time scale measurements batch-wise or in real-time. Following the aggregation of the multi time-scale measurements, the spatial states of the consistent time-series are estimated using matrix completion based DSSE approach. Simulation results on IEEE 37 and IEEE 123 bus test systems illustrate the efficiency of the proposed methods from the standpoint of both multi time-scale data aggregation and DSSE.

42 ENGINEERING↗
DOE OSTI2026

Transplatformer: translating toxicogenomic profiles between generations of platforms

Background Transcriptomic profiling technologies have advanced the analysis of biological and toxicological responses. However, substantial differences in probe design, dynamic range, gene coverage, and preprocessing pipelines across platforms introduce artifacts that limit cross-study integration and hinder the reuse of historical datasets. We aim to develop computational methods for accurate cross-platform translation to maximize the value of legacy resources. Results We present TransPlatformer a deep learning framework for translating gene expression profiles across heterogeneous toxicogenomics platforms. TransPlatformer employs a novel attention-based architecture to map high-dimensional fold-change vectors from legacy microarray technologies to current platforms. Models are trained and evaluated using DrugMatrix, spanning three technological generations. We investigate mixed-tissue, single-tissue, and cross-tissue training paradigms and benchmark performance against multilayer perceptron and matrix-completion baselines. In mixed-tissue training, TransPlatformer achieves a greater than 50% reduction in mean absolute error (0.043 vs. 0.09) and nearly doubles Pearson correlation ( ≈ 0.71 vs. 0.37) relative to baseline methods. Importantly, TransPlatformer preserves rare but biologically meaningful over- and under-expressed signals, with mean absolute error below 0.22. Single-tissue models yield further improvements for well-represented organs, such as a 10% reduction in liver mean absolute error, while underscoring the need for data augmentation strategies in low-sample tissues.ra Conclusions TransPlatformer provides an effective and scalable computational solution for cross-platform transcriptomic translation. By enabling biologically faithful harmonization of gene expression data, the proposed approach facilitates the reuse of legacy toxicogenomics datasets, enhances downstream biomarker discovery, and supports more reproducible predictive modeling in toxicology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Cognition at the Point of Sensing

Over the last 15 years, compressive sensing techniques have been developed which have the potential to greatly reduce the amount of data collected by systems while preserving the amount of information obtained. A cost of this efficiency is that a computationally-intensive optimization routine must be used to put the sensed data into a form that a person can interpret. At the same time, machine learning techniques have experienced tremendous growth as well. Machines have demonstrated the ability learn how to effectively perform tasks such as detection and classification at speeds much faster than humanly possible. Our goal in this project was to study the feasibility of using compressive sensing systems "at the edge." That is, how can compressive sensing sensors be deployed such that information is created at the remote sensor rather than sending raw data to a central processing location? Studies were performed to analyze whether machine learning could be done on the compressively sensed data in its raw form. If a machine is performing the task, is it possible to do so without putting the data into a human interpretable form? We show that this is possible for some systems, in particular a compressive sensing snapshot imaging spectrometer. Machine learning tasks were demonstrated to be more effective and more robust to noise when the machine learning algorithm worked on data in its raw form. This system is shown to outperform a traditional spectrometer. Techniques for reducing the complexity of the reconstruction routine were also analyzed. Techniques for such as data regularization, deep neural networks, and matrix completion were studied and shown to have benefits over traditional reconstruction techniques. In this project we showed that compressive sensing sensors are indeed feasible at the edge. As always, sensors and algorithms must be carefully tuned to work in the constrained environment. In this project we developed tools and techniques to enable those analyses.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Mild chemical recycling of carbon fiber-reinforced epoxy composites in aqueous buffers and development of hydrothermally recyclable vitrimer composites from recyclates

Carbon fiber-reinforced polymer (CFRP) composites have gained widespread adoption across diverse industries. However, their inherent stability, stemming from the crosslinked structure of the matrix resin, poses a significant challenge in managing the growing volume of CFRP waste. Consequently, there is a pressing need for an eco-friendly upcycling method that effectively recovers and reuses both the valuable carbon fibers and the polymer matrix from CFRP waste. This study presents a novel approach for upcycling CFRP waste under environmentally friendly conditions. Firstly, CFRP waste with an amine-cured epoxy matrix was completely decomposed in aqueous buffer solutions under mild conditions (≤ 220 °C, pH = 4.8). Further, decomposed matrix polymer (DMP) was employed to transform conventional epoxy resins into recyclable vitrimers. Lastly, new composites that can be hydrothermally recycled were fabricated using both DMP and recovered carbon fibers. As a result, this work proposes a novel strategy for achieving a circular economy within the CFRP industry.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Perturbative readout-error mitigation for near-term quantum computers

Readout errors on near-term quantum computers can introduce significant error to the empirical probability distribution sampled from the output of a quantum circuit. These errors can be mitigated by classical postprocessing given the access of an experimental response matrix that describes the error associated with the measurement of each computational basis state. However, the resources required to characterize a complete response matrix and to compute the corrected probability distribution scale exponentially with the number of qubits, n . In this work, we modify standard matrix inversion techniques using perturbative approximations with significantly reduced complexity and bounded error when the likelihood of high-order bit-flip events is strongly suppressed. Given a characteristic error rate q , we discuss a method to recover the probability of the all-zeros bit string p 0 by sampling only a small subspace of the response matrix before inverting readout error, resulting in a relative speedup of poly [ 2 n / ( n w ) ] , which we motivate using a simplified error model for which the approximation incurs only O ( q w ) error for some integer w . We then provide a generalized technique to efficiently recover full output distributions with O ( q w ) error in the perturbative limit. These approximate techniques for readout-error correction may greatly accelerate near-term quantum computing applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Measurement of the CMB temperature power spectrum and constraints on cosmology from the SPT-3G 2018 T T , T E , and E E dataset

We present a sample-variance-limited measurement of the temperature power spectrum ( T T ) of the cosmic microwave background using observations of a ∼ 1500 deg 2 field made by the SPT-3G in 2018. We report multifrequency power spectrum measurements at 95, 150, and 220 GHz covering the angular multipole range 750 ≤ ℓ < 3000 . We combine this T T measurement with the published polarization power spectrum measurements from the 2018 observing season and update their associated covariance matrix to complete the SPT-3G 2018 T T / T E / E E dataset. This is the first analysis to present cosmological constraints from SPT T T , T E , and E E power spectrum measurements jointly. We blind the cosmological results and subject the dataset to a series of consistency tests at the power spectrum and parameter level. We find excellent agreement between frequencies and spectrum types and our results are robust to the modeling of astrophysical foregrounds. We report results for Λ CDM and a series of extensions, drawing on the following parameters: the amplitude of the gravitational lensing effect on primary power spectra A L , the effective number of neutrino species N eff , the primordial helium abundance Y P , and the baryon clumping factor due to primordial magnetic fields b . We find that the SPT-3G 2018 T T / T E / E E data are well fit by Λ CDM with a probability to exceed of 15%. For Λ CDM , we constrain the expansion rate today to H 0 = 68.3 ± 1.5 km s - 1 Mpc - 1 and the combined structure growth parameter to S 8 = 0.797 ± 0.042 . The SPT-based results are effectively independent of Planck, and the cosmological parameter constraints from either dataset are within < 1 σ of each other. The addition of temperature data to the SPT-3G T E / E E power spectra improves constraints by 8–27% for each of the Λ CDM cosmological parameters. When additionally fitting A L , N eff , or N eff + Y P , the posteriors of these parameters tighten by 5–24%. In the case of primordial magnetic fields, complete T T / T E / E E power spectrum measurements are necessary to break the degeneracy between b and n s , the spectral index of primordial density perturbations. We report a 95% confidence upper limit from SPT-3G data of b < 1.0 . The cosmological constraints in this work are the tightest from SPT primary power spectrum measurements to date and the analysis forms a new framework for future SPT analyses.

79 ASTRONOMY AND ASTROPHYSICS↗