Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Preprocessing Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

421 records · Page 24
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Analysis of ETMS Data Quality for Traffic Flow Management Decisions

The data needed for air traffic flow management decision support tools is provided by the Enhanced Traffic Management System (ETMS). This includes both the tools that are in current use and the ones being developed for future deployment. Since the quality of decision support provided by all these tools will be influenced by the quality of the input ETMS data, an assessment of ETMS data quality is needed. Motivated by this desire, ETMS data quality is examined in this paper in terms of the unavailability of flight plans, deviation from the filed flight plans, departure delays, altitude errors and track data drops. Although many of these data quality issues are not new, little is known about their extent. A goal of this paper is to document the magnitude of data quality issues supported by numerical analysis of ETMS data. Guided by this goal, ETMS data for a 24-hour period were processed to determine the number of aircraft with missing flight plan messages at any given instant of time. Results are presented for aircraft above 18,000 feet altitude and also at all altitudes. Since deviation from filed flight plan is also a major cause of trajectory-modeling errors, statistics of deviations are presented. Errors in proposed departure times and ETMS-generated vertical profiles are also shown. A method for conditioning the vertical profiles for improving demand prediction accuracy is described. Graphs of actual sector counts obtained using these vertical profiles are compared with those obtained using the Host data for sectors in the Fort Worth Center to demonstrate the benefit of preprocessing. Finally, results are presented to quantify the extent of data drops. A method for propagating track positions during ETMS data drops is also described.

Chatterji, Gano B.↗
DOE OSTI2021

Fault Detection Utilizing Convolution Neural Network on Timeseries Synchrophasor Data From Phasor Measurement Units

An end-to-end supervised learning method is proposed for fault detection in the electric grid using Big Data from multiple Phasor Measurement Units (PMUs). The approach consists of preprocessing steps aimed at reducing data noise and dimensionality, followed by utilization of six classification models considered for detecting faults. Three of the models were variants of Convolutional Neural Network (CNN) architectures that consider a single type of measurement (voltage, current or frequency) at all PMUs or all types together also at all PMUs. CNN based models were compared to traditional methods of Logistic Regression (LR), Multi-layer Perceptron (MLP) and Support Vector Machine (SVM). Evaluation was conducted on two-year data measured by PMUs at 37 locations in a large electric grid. Here, the response variable for classification were extracted from the grid-wide outage event log. Experiments show that CNN-based models outperformed traditional methods on one year out-of-sample outage detection over the entire grid.

42 ENGINEERING↗
DOE OSTI2022

Data and scripts associated with the manuscript "Encoding Diel Hysteresis and the Birch Effect in Dryland Soil Respiration Models through Knowledge-Guided Deep Learning"

This package contains the data and scripts used in "Encoding Diel Hysteresis and the Birch Effect in Dryland Soil Respiration Models through Knowledge-Guided Deep Learning" (Jiang et al., 2022). The data.zip file contains the flux tower and automated chamber observations used for developing the deep learning model for modeling soil respiration. The scripts.zip file contains the Jupyter notebooks and python scripts for preprocessing the data, training the deep learning models, and postprocessing the results. The src.zip contains the source code for training the deep learning model, performing mutual information analysis, and plotting functions. The trained_models.zip contains multiple folders used for hosting the trained deep-learning models and the associated soil respiration predictions. The whole process is performed using python. We include the REAMD.md to document the python package requirements.Soil respiration in dryland ecosystems is challenging to model due to its complex interactions with environmental drivers. Knowledge-guided deep learning provides a much more effective means of accurately representing these complex interactions than traditional Q10-based models. Mutual information analysis revealed that future soil temperature shares more information with soil respiration than past soil temperature, consistent with their clockwise diel hysteresis. We explicitly encoded diel hysteresis, soil drying, and soil rewetting effects on soil respiration dynamics in a newly designed Long Short Term Memory (LSTM) model. The model takes both past and future environmental drivers as inputs to predict soil respiration. The new LSTM model substantially outperformed three Q10-based models and the Community Land Model when reproducing the observed soil respiration dynamics in a semi-arid ecosystem. The new LSTM model clearly demonstrated its superiority for temporally extrapolating soil respiration dynamics, such that the resulting correlation with observational data is up to 0.7 while the correlations of both Q10-based models and the Community Land Model (CLM) are less than 0.4. Our results underscore the high potential for knowledge-guided deep learning to replace Q10-based soil respiration modules in Earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Models, data, and scripts associated with “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning”

This data package is associated with the publication “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning’’ submitted to the Journal of Geophysical Research: Machine Learning and Computation (Scheibe et al. 2024). River sediment respiration observations are expensive and labor intensive to obtain and there is no physical model for predicting this quantity. The Worldwide Hydrobiogeochemisty Observation Network for Dynamic River Systems (WHONDRS) observational data set (Goldman et al.; 2020) is used to train machine learning (ML) models to predict respiration rates at unsampled sites. This repository archives training data, ML models, predictions, and model evaluation results for the purposes of reproducibility of the results in the associated manuscript and community reuse of the ML models trained in this project. One of the key challenges in this work was to find an optimum configuration for machine learning models to work with this feature-rich (i.e. 100+ possible input variables) data set. Here, we used a two-tiered approach to managing the analysis of this complex data set: 1) a stacked ensemble of ML models that can automatically optimize hyperparameters to accelerate the process of model selection and tuning and 2) feature permutation importance to iteratively select the most important features (i.e. inputs) to the ML models. The major elements of this ML workflow are modular, portable, open, and cloud-based, thus making this implementation a potential template for other applications. This data package is associated with the GitHub repository found at Please see the file level metadata (flmd; “sl-archive-whondrs_flmd.csv”) for a list of all files contained in this data package and descriptions for each. Please see the data dictionary (dd; “sl-archive-whondrs_dd.csv”) for a list of all column headers contained within comma separated value (csv) files in this data package and descriptions for each. The GitHub repository is organized into five top-level directories: (1) “input_data” holds the training data for the ML models; (2) “ml_models” holds machine learning models trained on the data in “input_data”; (3) “scripts” contains data preprocessing and postprocessing scripts and intermediate results specific to this data set that bookend the ML workflow; (4) “examples” contains the visualization of the results in this repository including plotting scripts for the manuscript (e.g., model evaluation, FPI results) and scripts for running predictions with the ML models (i.e., reusing the trained ML models); (5) “output_data” holds the overall results of the ML model on that branch. Each trained ML model resides on its own branch in the repository; this means that inputs and outputs can be different branch-to-branch. Furthermore, depending on the number of features used to train the ML models, the preprocessing and postprocessing scripts, and their intermediate results, can also be different branch-to-branch. The “main-*” branches are meant to be starting points (i.e. trunks) for each model branch (i.e. sprouts). Please see the Branch Navigation section in the top-level README.md in the GitHub repository for more details. There is also one hidden directory “.github/workflows”. This hidden directory contains information for how to run the ML workflow as an end-to-end automated GitHub Action but it is not needed for reusing the ML models archived here. Please the top-level README.md in the GitHub repository for more details on the automation.

13C↗
NASA NTRS2023

Commercial Smallsat Data Acquisition Program On-ramp #2 Airbus U.S. Synthetic Aperture Radar (SAR) Evaluation Report

In 2017, NASA’s Earth Science Division (ESD) launched the Private-Sector Small Constellation Satellite Data Product Pilot, now referred to as the Commercial Smallsat Data Acquisition (CSDA) program. The objective of CSDA is to identify, evaluate, and acquire commercial remote sensing data that support NASA’s Earth science research and application activities. The Pilot successfully concluded in early 2020, when CSDA transitioned into a sustained program with on-ramping opportunities for new vendors as the industry emerges with new candidates and capabilities. In October 2019, a Request for Information (RFI) seeking capability statements from parties interested in providing data from spaceborne platforms was released for the CSDA on-ramp #2 evaluations. To be responsive to the RFI, the commercial satellite constellations had to consist of three or more operating spacecraft actively collecting data in a non-geostationary orbit with full latitudinal coverage and be U.S. companies. Two vendors responded to the RFI and were evaluated by a committee composed of NASA ESD leadership, program managers, and scientists. Both vendors satisfied the RFI requirements and were asked to respond to a Request for Proposal (RFP). After review of the proposals, NASA entered into a Blanket Purchase Agreement (BPA) with Airbus Defense and Space GEO, Inc. (Airbus) U.S. in September 2021 and with BlackSky Geospatial Solutions, Inc. (BlackSky) in November 2021. In this report, CSDA provides an evaluation of the usefulness of data provided by the Airbus U.S. Synthetic Aperture Radar (SAR) satellite constellation, consisting of TerraSAR-X (launched in 2007), TanDEM-X (launched in 2010), and PAZ (launched in 2018), for advancing NASA’s Earth system science research and applications. The evaluation of the BlackSky commercial data will be provided in a separate report. To conduct the Airbus evaluation, NASA’s ESD augmented 13 existing research projects that could potentially benefit from, and had the expertise to evaluate, the commercial data being considered for longer-term purchase. Investigators from NASA’s Research and Analysis Program science focus areas and from NASA’s Applied Sciences Program elements participated in the evaluation. A summary of the research areas evaluated by the Principal Investigator (PI) teams is presented in Figure 3. CSDA also funded a dedicated activity to evaluate the satellite data quality (calibration and geolocation) independently by assessing the accuracy of data from Airbus. Evaluation activities were carried out by the selected PIs from December 7, 2022, to December 7, 2023. Delivery of datasets requested by the researchers began in January 2023. The vendors were evaluated on the accessibility of data, accuracy and completeness of metadata, and promptness and quality of user support services. Datasets purchased during the evaluation have been archived by NASA and will be made available to current and future government-funded researchers in accordance with the End User License Agreement (EULA). This synthesis report distills and integrates the findings of research reports commissioned by NASA for the Airbus evaluation. This report also includes recommendations that inform the way ahead for the program. The scientific results from the evaluations demonstrated that the commercial data from Airbus were able to advance NASA research and applications. However, the PIs encountered limitations that diminished the usefulness of the data due to the amount of effort that was required to access, preprocess, and analyze these data. One significant issue encountered was the limited spatial and temporal coverage of the data in the Airbus archive that could be used to conduct time series analyses or assessments over large spatial scales. Overall, however, the utility and the quality of the evaluated data outweighed the difficulties encountered, and NASA has concluded that the Airbus SAR data would complement NASA’s existing Earth observation capabilities and Airbus U.S. would qualify to participate in the sustained phase of the program.

Batuhan Osmanoglu↗
DOE OSTI2026

Short‐Term Hourly Weather Forecasting Using PredRNN With Image Preprocessing

Global weather forecast models are vital tools with numerous applications, including public safety, agriculture, and transportation. Recent advancements in artificial intelligence (AI) and deep learning (DL) have shown the potential to enhance weather forecasting accuracy and speed. In this study, we developed a short-term hourly weather forecast framework with a wavelet transform function for data preprocessing and a spatiotemporal DL model, PredRNN, for predicting five surface atmospheric variables, including wind speed and direction, mean sea level pressure (MSLP), temperature, and precipitation. The framework demonstrated promising results. It produces global forecasts at 0.25° (∼25 km) with a 1-day lead time RMSE of 1.8 m/s for wind components, 180 Pa for MSLP, and 1.8 K for temperature. Although our model does not surpass state-of-the-art AI weather forecast models across all metrics, it outperforms these models in precipitation forecasting and wind prediction at short lead times and achieves comparable accuracy for MSLP. Its native hourly forecasting capability, together with training on widely accessible GPU hardware, contributes meaningfully to the advancement of accessible DL weather forecasting methods. Our work highlights the importance of integrating temporal components and data transformation techniques to improve the predictability and accuracy of weather forecasts.

Tran, Hoang [Pacific Northwest National Laboratory↗
DOE OSTI2023

Laboratory-Scale Coal-Derived Graphene Process (Final Report)

The Energy & Environmental Research Center (EERC) conducted a laboratory-scale coal-derived graphene (CDG) project focused on developing a technological process for making graphene from four U.S. domestic coal or coal wastes, including lignite from North Dakota, subbituminous coal from Wyoming, bituminous coal from Utah, and anthracite from Pennsylvania. The project was divided into two performance or budget periods (BPs), with BP1 comprising the up-front laboratory experiments to make graphene materials from coal beginning on May 1, 2020, to April 30, 2022. BP2 was conducted from May 1, 2022, to April 30, 2023, and was focused on analyzing the CDG process economic feasibility and the technical gaps for technological scale-up and commercialization. During this project, a few different coal-derived high-value products have been demonstrated, including graphite, graphene oxide (GO), reduced graphene oxide (rGO), and graphene quantum dots (GQDs). A new graphite microstructure was discovered and named “croissant graphite” because of the exterior morphological and textural resemblance to croissant food items sold in commercial groceries stores. The new graphite structure and the associated preparation from coal or coal waste feedstocks has been the subject of a U.S. patent application. The systematic experimental processes involving coal cleaning, upgrading, and conversion to high-value carbon products culminated into a developed upgraded coal-to-products (UCP) technology that is being pursued for potential fast-track commercialization, if funding is available. It is envisioned that commercialization of the UCP technology would increase consumption of U.S. domestic coals or coal wastes to make environmentally sustainable high-value products for the electronics industry, high-energy-storage applications, and clean energy technologies such as electric vehicle (EV) lithium-ion batteries (LIBs), for which graphite has become a critical mineral commodity. Croissant graphite microstructures, when observed by field emission scanning electron microscopy (FESEM), display wavy surface morphology and often grow from a base that is made of graphitized particles with honeycomb-like layers, which are believed to be graphene layers. While more studies are needed to fully ascertain the mechanisms of the croissant graphite microstructure formation, it is postulated that their growth may begin from curling of the graphene sheets into ribbon-like structures, and continuous growth and densification of the ribbon-like structures forms croissant microstructures. Additional studies are ongoing to evaluate the electrochemical performance of croissant graphite for LIB applications and to determine the experimental conditions necessary to tune on/off croissant formation so that it can be either optimized or suppressed depending on performance evaluation results. In addition to the discovery of croissant graphite, the graphitization process from the four coal ranks in general was successful. X-ray diffraction (XRD) analysis showed that the degree of graphitization (DoG) ranged from 12% to 80% in an early sample set, and further optimization on lignite coal produces a DoG of about 92%, which was spectacular to see as lignite is the lowest-rank coal. Thus, it is expected that the graphitization performance for higher-rank coals will be similar or better when optimized as well. The coal-derived graphite was used to make GO and rGO. Analytical characterization, e.g., by methods such as Raman spectroscopy, XRD, Fourier transform infrared (FTIR) spectroscopy and FESEM, showed that the sequence of converting the coal to graphite, exfoliating it to GO, and then chemically reducing the GO to rGO was successful. Although coal naturally contains aromatic compounds and some relatively small-sized condensed aromatic units, it does not contain graphene sheets. In the UCP process, the aromatic domains in the coals, particularly low-rank coals, are concentrated and condensed further into graphene sheets, which are ordered into a 3D stack during graphitization. The synthesized graphite is then unpacked by methods such as exfoliation to various graphene products. GQDs were synthesized from all four coal types, and their optical properties were demonstrated to be tunable by the coal precursor preprocessing treatments. In all four coal types, enhanced optical properties were observed for the produced GDQs with incremental improvements made to the coal precursors. GQDs produced from raw coal samples displayed lower ultraviolet–visible (UV–Vis) spectroscopy absorbance intensity compared to those obtained from cleaned and upgraded coal residues. The photoluminescence (PL) intensities also varied with pretreatment conditions and with the concentration of GQDs in aqueous solutions. GQDs obtained from anthracite show longer emission wavelengths and can be excited by visible light as opposed to GQDs derived from the other coal ranks. UV fluorescence 3D maps and spectra revealed that the emission wavelength at which the GQDs solutions display the highest intensity was slightly redshifted based on the coal precursor pretreatments. In low-rank coal (lignite and subbituminous) samples, two clusters were observed in the maps for GQDs, which may suggest that there are potentially two types of fluorophores in solution or two main size populations. The ability to tune the properties of GQDs based on processing methods can be exploited to make GQDs for various optical display or optoelectronics applications. The results also highlight the importance of removing coal-borne impurities to improve the quality of the coal precursor for preparation of graphene products. Coal and/or coal wastes preprocessing methods were developed and applied to clean and upgrade the coal precursors prior to graphitization and subsequent conversion to graphene products. The preprocessing methods involve high specific-gravity separations, mineral acid cleaning (no hydrofluoric acid), and subsequent upgrading by reducing the coal-borne heteroatom (nitrogen, sulfur, and oxygen) content using proprietary chemical agents. Analytical characterization revealed that the preprocessing steps were successful, with ash reductions that range from 38% to 80% and residual ash content that was below the 5 wt% initial target. Based on proximate and ultimate analysis, the heteroatom reduction reactions produced upgraded coal residues with the oxygen content reduced by 8% to 24%, with additional reductions in the nitrogen and sulfur contents. An initial assessment of the waste streams from the UCP process shows very small to negligible environmental impact due to CO 2 , NO x , and SO x because most process steps are performed under inert atmosphere with argon. Consequently, reactive oxygen environments that tend to create these species are avoided. The inorganic and potentially hazardous species are released into aqueous waste streams that are easy to handle for proper disposal. The liquid waste streams were found to contain low-level concentrations of rare-earth elements (REEs), which could be concentrated and recovered as value-added by-products. Additionally, the volatile and gaseous fractions from carbonization and heat treatment contain useful organic compounds that can also be recovered as potential valuable by-products. Thus, the UCP technology is considered an environmentally sustainable and promising emerging technology for making high-value products from coal and coal wastes, with potential additional value-added by-products. Analysis of potential markets for the coal-derived carbon products shows a strong demand in both niche market sectors and across a wide variety of other industrial sectors. Graphite is currently considered a critical mineral commodity that has a large and growing demand in the LIB industry for EV applications. Based on data from Fortune Business Insights (2022) and Marketwatch (2023) reports, the average global graphite market is projected to reach about 33 billion by 2028, growing at a compound annual growth rate (CAGR) of about 7%, with much of this growth expected to be in the LIB industry. GO and rGO have strong market potentials in various application areas, such as coatings for anticorrosion, anti-icing, and antimicrobial protection, thermal barriers, wear resistance, sensors, additive manufacturing such as 3D inks, and others. GQDs are the emerging key player in the bioimaging, photovoltaics, and light-emitting diodes (LEDs) applications, with the potential to replace traditional semiconductor quantum dots (SQDs), which are based on metallic systems that are more toxic and more expensive. Biomedical applications of GQDs are becoming more attractive because of low to no toxicity and extremely low cost compared to SQDs. The major challenges for scale-up and commercialization of coal-derived carbon products such as graphene vary from the inherent attributes of graphene itself to reluctance to accept graphene in new manufacturing processes because of the uncertainty of the unknown. The 2D nature of graphene materials with a thickness of one atom presents significant challenges to proper handling/processing, and process scale-up becomes difficult because it requires high-end, expensive equipment, even for routine handling and analysis for quality assurance and control. Pristine graphene can also be extremely difficult to work into other matrices, thus hindering downstream processibility, especially at large scale. Currently, the cost of graphene and graphene products is still high and presents an economic risk that tends to slow down investment in scaling up emerging technologies. The lack of a standard for graphene materials for quality assurance and quality control poses a great challenge not only for the markets but also for commercialization efforts. A first-look economic feasibility analysis of the UCP technology provided valuable information that suggests the UCP process would be feasible, especially when it is scaled to a pilot scale and could be more competitive at the full scale. Graphitization was found to be the most energy-consuming and most capital-intensive step in the overall process. In small laboratory- and bench-scale experiments, labor is a significant contributor to the total process costs. Although these energy, capital, and labor constraints contribute to a higher selling price for the product, a preliminary economic model suggests that the process would be feasible at large scale when the process is fully integrated, optimized, and automated.

01 COAL, LIGNITE, AND PEAT↗