Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “temporal learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

A deep learning approach to anomaly detection in geological carbon sequestration sites using pressure measurements

Carbon capture and storage (CCS) has been extensively investigated as a potential engineering measure to reduce anthropogenic carbon emission to the atmosphere. Real-time monitoring of the safety and integrity of carbon storage reservoirs is a critical aspect of any commercial-scale CCS deployment. Pressure-based sensing is cost effective, suitable for real-time monitoring, and scalable to large monitoring networks. However, questions remain on how to best harness intelligent information from the high-frequency pressure monitoring sensors to support real-time decisions. Here this work presents a deep-learning-based framework for analyzing and detecting anomalies in pressure data streams by using a convolutional long short-term memory (ConvLSTM) neural network model, which allows for the fusion of both static and dynamic reservoir data. In ConvLSTM, the convolutional neural network (CNN) is used for spatial pattern mining and the LSTM is used for temporal pattern recognition. The performance of the ConvLSTM model for real-time anomaly detection is demonstrated using a set of pressure monitoring data collected from Cranfield, Mississippi, an active enhanced-oil-recovery field. The anomaly detection model is trained using bottom-hole pressure data acquired from the base experiment (without leak event) and then tested on pressure data collected during a series of controlled CO 2 release experiments (with artificially created leak events). Results show that the ConvLSTM neural network model successfully detected anomalies in the pressure time series obtained from the controlled release experiments. Inclusion of static information into the model further improves the robustness of ConvLSTM.

13 HYDRO ENERGY↗

Space-Based Sensor Web for Earth Science Applications: An Integrated Architecture for Providing Societal Benefits

There is a significant interest in the Earth Science research and user remote sensing community to substantially increase the number of useful observations relative to the current frequency of collection. The obvious reason for such a push is to improve the temporal, spectral, and spatial coverage of the area(s) under investigation. However, there is little analysis available in terms of the benefits, costs and the optimal set of sensors needed to make the necessary observations. Classic observing system solutions may no longer be applicable because of their point design philosophy. Instead, a new intelligent data collection system paradigm employing both reactive and proactive measurement strategies with adaptability to the dynamics of the phenomena should be developed. This is a complex problem that should be carefully studied and balanced across various boundaries including: science, modeling, applications, and technology. Modeling plays a crucial role in making useful predictions about naturally occurring or human-induced phenomena In particular, modeling can serve to mitigate the potentially deleterious impacts a phenomenon may have on human life, property, and the economy. This is especially significant when one is interested in learning about the dynamics of, for example, the spread of forest fires, regional to large-scale air quality issues, the spread of the harmful invasive species, or the atmospheric transport of volcanic plumes and ash. This paper identifies and examines these challenging issues and presents architectural alternatives for an integrated sensor web to provide observing scenarios driving the requisite dynamic spatial, spectral, and temporal characteristics to address these key application areas. A special emphasis is placed on the observing systems and its operational aspects in serving the multiple users and stakeholders in providing societal benefits. We also address how such systems will take advantage of technological advancement in small spacecraft and emerging information technologies, and how sensor web options may be realized and made affordable. Specialized detector subsystems and precision flying techniques may still require substantial innovation, development time and cost: we have presented the considerations for these issues. Finally, data and information gathering and compression techniques are also briefly described.

Habib, Shahid↗

Predictive Modeling for Differential Diagnosis and Mortality Risk Assessment

The prevalence of electronic health record (EHR) systems has brought prodigious biomedical informatics opportunity. Automated machine learning methods can effectively utilize such data and have become common tools for healthcare predictive modeling. Researches in medical informatics have explored the potential of deep learning and classical models in emergent care scenarios. In particular, predicting differential diagnoses for admissions have proven useful in decreasing unnecessary lab tests and improving inpatient triage decision-making. Moreover, identification of high-risk patients for in-hospital mortality is vitally important to maximize allocation of medical resources.The Medical Information Mart for Intensive Care (MIMIC-III) database, containing de-identified critical care inpatient was used in our study. This data set captures hospital patient laboratory measurements, pharmacologic prescriptions, diagnostic data and procedure event recordings. When considering adult patients and discounting admissions with ICU length of stay less than 24 hours, there were 37,787 unique admissions and 30,414 total patients. We examined the top 25 most prevalent ICD-9 group-level disease specificities in MIMIC-III using a multi-label classification model. In-hospital mortality was modeled as binary classification with 4,155 (13%) adult patients that expired, of which 3,138 (75.5%) were in the ICU setting. The metrics AUC, F1 score, sensitivity and specificity values calculated for each disease label measured prediction performance.The usage of ICD-9 group codes reduced feature dimension from 14,567 to 942 and greatly improved distribution of patient diagnostic categories. Disease temporal patterns were captured by considering the most frequently sampled 6 vital signs and 13 laboratory values. Missing data were imputed at each time-stamp. Time-series raw hourly average values were converted into 5 summary features (mean, standard deviation, number of observations, min & max values). Patient demographic variables such as age, gender, marital status and ethnicity were also factored into the modeling. Choi et al showed that contextual embedding of medical data, diagnostic and procedural codes alone can predict future diagnoses with sensitivity as high as 0.79. We utilized an embedding technique called word2vec which allowed sparse representations of medical history to be transformed into dense word vectors. The mappings captured contextual information by treating each admission as a sentence and learning the most likely neighboring words in a sliding window fashion. Binary and multi-label classification was achieved via collapse models, which do not consider temporal information, as well as recurrent neural networks with regularization, Softmax output layer activation together with categorical cross-entropy as the loss function.

US Army collaboration↗

Evaluation of Machine Learning and Deep Learning Algorithms for Fire Prediction in Southeast Asia

Vegetation fires are prevalent in South/Southeast Asian countries, making fire prediction crucial due to their potential environmental, economic, and social impacts. Accurate predictions of fires facilitate timely interventions, helping to mitigate uncontrolled fires that can lead to biodiversity loss and air quality issues. In this study, we utilize VIIRS satellite-derived fire data alongside six machine learning and deep learning models—Simple Persistence, Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), CNN-LSTM, and ConvLSTM—to determine the most effective fire prediction model, using Root Mean Square Error (RMSE) as the metric. Our results indicate that the CNN model is the most reliable in regions with spatial dependencies, such as Brunei, Indonesia, Malaysia, the Philippines, Timor-Leste, and Thailand. Conversely, the ConvLSTM model excels in countries with complex spatiotemporal dynamics like Laos, Myanmar, and Vietnam. The CNN-LSTM hybrid model also performed well in Cambodia, suggesting a need for a balanced approach in areas requiring both spatial and temporal feature extraction. Furthermore, simpler models like Persistence and MLP showed limitations in capturing dynamic patterns and temporal dependencies. Our findings highlight the importance of evaluating models before implementing any decision support systems (DSS) in fire management. By tailoring models to specific regional fire data, we can enhance prediction accuracy and responsiveness, ultimately improving fire risk management in Southeast Asia and beyond.

Deep learning↗

A multi-scale time-series dataset with benchmark for machine learning in decarbonized energy grids

The electric grid is a key enabling infrastructure for the ambitious transition towards carbon neutrality as we grapple with climate change. With deepening penetration of renewable resources, the reliable operation of the electric grid becomes increasingly challenging. In this paper, we present PSML, a first-of-its-kind open-access multi-scale time-series dataset, to aid in the development of data-driven machine learning (ML)-based approaches towards reliable operation of future electric grids. The dataset is synthesized from a joint transmission and distribution electric grid to capture the increasingly important interactions and uncertainties of the grid dynamics, containing power, voltage and current measurements over multiple spatio-temporal scales. Using PSML, we provide state-of-the-art ML benchmarks on three challenging use cases of critical importance to achieve: (i) early detection, accurate classification and localization of dynamic disturbances; (ii) robust hierarchical forecasting of load and renewable energy; and (iii) realistic synthetic generation of physical-law-constrained measurements. We envision that this dataset will provide use-inspired ML research in safety-critical systems, while simultaneously enabling ML researchers to contribute towards decarbonization of energy sectors.

54 ENVIRONMENTAL SCIENCES↗

Astronomical image time series classification using CONVolutional attENTION (ConvEntion)

Aims: The treatment of astronomical image time series has won increasing attention in recent years. Indeed, numerous surveys following up on transient objects are in progress or under construction, such as the Vera C. Rubin Observatory Legacy Survey for Space and Time (LSST), which is poised to produce huge amounts of these time series. The associated scientific topics are extensive, ranging from the study of objects in our galaxy to the observation of the most distant supernovae for measuring the expansion of the universe. With such a large amount of data available, the need for robust automatic tools to detect and classify celestial objects is growing steadily. Methods: This study is based on the assumption that astronomical images contain more information than light curves. In this paper, we propose a novel approach based on deep learning for classifying different types of space objects directly using images. We named our approach ConvEntion, which stands for CONVolutional attENTION. It is based on convolutions and transformers, which are new approaches for the treatment of astronomical image time series. Our solution integrates spatio-temporal features and can be applied to various types of image datasets with any number of bands. Results: In this work, we solved various problems the datasets tend to suffer from and we present new results for classifications using astronomical image time series with an increase in accuracy of 13%, compared to state-of-the-art approaches that use image time series, and a 12% increase, compared to approaches that use light curves.

79 ASTRONOMY AND ASTROPHYSICS↗

Long-Term Fish Monitoring in Large Rivers: Utility of “Benchmarking” across Basins

Abstract In business, benchmarking is a widely used practice of comparing your own business processes to those of other comparable companies and incorporating identified best practices to improve performance. Biologists and resource managers designing and conducting monitoring programs for fish in large river systems tend to focus on single river basins or segments of large rivers, missing opportunities to learn from those conducting fish monitoring in other rivers. We briefly examine five long-term fish monitoring programs in large rivers in the United States (Colorado, Columbia, Mississippi, Illinois, and Tallapoosa rivers) and identify opportunities for learning across programs by detailing best monitoring practices and why these practices were chosen. Although monitoring objectives, methods, and program maturity differ between each river system, examples from these five case studies illustrate the important role that long-term monitoring programs play in interpreting temporal and spatial shifts in fish populations for both established objectives and newly emerging questions. We suggest that deliberate efforts to develop a broader collaborative network through benchmarking will facilitate sharing of ideas and development of more effective monitoring programs.

Ward, David L.↗

Tokamak edge localized mode onset prediction with deep neural network and pedestal turbulence

Abstract A neural network, BES-ELMnet, predicting a quasi-periodic disruptive eruption of the plasma energy and particles known as edge localized mode (ELM) onset is developed with observed pedestal turbulence from the beam emission spectroscopy system in DIII-D. BES-ELMnet has convolutional and fully-connected layers, taking two-dimensional plasma fluctuations with a temporal window of size 128 µ s and generating a scalar output which can be interpreted as a probability of the upcoming ELM onset. As approximately labeled inter-ELM broadband ( 15 kHz ⩽ f ⩽ 150 kHz ) fluctuations are given to the network, BES-ELMnet learns by itself ELM-related precursors arising before the onsets through supervised learning scheme. BES-ELMnet achieves the gradually increasing ELM onset probabilities between two consecutive ELMs during the inter-ELM phases and can forecast the first ELM onsets which occur after the high confinement mode transition. We further investigate the network generality in terms of the selected frequency band to ensure the use of BES-ELMnet for various operation regimes without changing the trained architecture. Therefore, our novel prediction method will enhance a proactive high confinement mode control of fusion-grade plasmas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Comparative Analysis of ML Techniques for Data-Driven Anomaly Detection, Classification and Localization in Distribution System

High penetration of Distributed Energy Resources (DERs), fundamental load behavior changes, controllable loads, and significant increase in Electrical Vehicles (EVs) lead to complex dynamic behavior of the electric distribution system. Increasing number of components also means more measurements, more data and more data anomalies. Detecting, classifying and localizing these anomalies are important for situational awareness, and at the same time, very challenging given increasing complexity of the system. Highly accurate and high-resolution analytical techniques are needed to support anomaly detection, classification and localization (AD-C-L) for monitoring, root cause analysis and decision making. This paper provides comprehensive review and analysis of the existing spatio-temporal AD-C-L techniques within the distribution system. Challenges for specific problems in AD-C- L have been also discussed in this paper. Existing AD-C- L techniques have been categorized and synthesized for specific merits and limitations of multiple Machine Learning (ML) methodologies using common developed metrics of performance. The comparative analysis is summarized and presented with the open research challenges and path forward for future research needs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Arctic Sea Ice Classification Using Microwave Scatterometer and Radiometer Data During 2002-2017

Temporal and spatial variation of sea ice type in the Arctic is an indicator of regional and global change. Arctic sea ice can be classified into two major categories: multiyear ice (MYI) and first-year ice. In this paper, classification method based on machine learning is established and applied to produce daily sea ice classification data set during the winter (November–April) from 2002 to 2017 using active microwave data from QuikSCAT and Advanced Scatterometer as well as passive microwave data from Advanced Microwave Scanning Radiometer for EOS, Special Sensor Microwave Imager/Sounder, and Advanced Microwave Scanning Radiometer 2 radiometer. First, the open water area is flagged out using brightness temperature (Tb) from the passive microwave sensor. Then, K-means algorithm is applied to identify the clusters of the two ice types in the Tb/backscatter parameter space and finally assign pixels to each class. Two optimization methods based on the movement of MYI and marginal ice zone are used to correct the misclassification of MYI. The results have shown a decrease of MYI in winter from 2002 to 2017, especially in 2008 and 2013 with a remarkable recovery in 2014. The classifications are consistent with results by visual interpretation from synthetic aperture radar images in the Canadian Arctic Archipelago with overall classification accuracy over 93%. Comparison with classifications from previous studies and products shows that our method could reflect more differences in MYI declining trend interannually and less anomalous fluctuations in certain years.

58 GEOSCIENCES↗

Gravity receptors and responses

The overall process of gravity sensing and response processes in plants may be divided conveniently into at least four components or stages: Stimulus susception (a physical event, characteristically the input to the G receptor system of environmental information about the G force magnitude, its vector direction, or both); information perception (an influence of susception on some biological structure or process that can be described as the transformation of environmental information into a biologicallly meaningful change); information transport (the export, if required, of an influence (often chemical) to cells and organs other than those at the sensor location); and biological response (almost always (in plants) a growth change of some kind). Some analysts of the process identify, between information perception and information transport, an additional stage, transduction, which would emphasize the importance of a transformation from one form of information to another, for example from mechanical statolith displacement to an electric, chemical, or other alteration that was its indirect result. These four (or five) stages are temporally sequential. Even if all that occurs at each stage can not be confidently identified, it seems evident that during transduction and transport, matters dealt with are found relatively late in the information flow rather than at the perception stage. As more and more is learned about the roles played by plant hormones which condition the G responses, the mechanism(s) of perception which should be are not necessarily better understood. However, if by asking the right questions and being lucky with experiments perhaps the discovery of how some process (such as sedimentation of protoplasmic organelles) dictates what happens down stream in the information flow sequence may be made.

Brown, Allan H.↗

Learning electric vehicle driver range anxiety with an initial state of charge-oriented gradient boosting approach

This manuscript focuses on the modeling of electric vehicle (EV) driver’s range anxiety, a fear that a vehicle does not have sufficient range, or state of charge (SOC) of the battery pack, to reach its destination and would strand its occupants. Despite numerous research studies on the modeling of charging behaviors, modeling efforts to understand at what battery percentages do EV drivers charge their vehicles, and what are the associated contributing factors, are rather limited. To this end, an ensemble learning model based on gradient boosting is developed. The model sequentially fits new predictors to new residuals of the previous prediction and, then, minimizes the loss when adding the latest prediction. A total of 18 features are defined and extracted from the multisource data, which cover information on driver, vehicles, stations, traffic conditions, as well as spatial-temporal context information of the charging events. The analyzed dataset includes 4.5-year’s charging event log data from 3,096 users and 468 public charging stations in Kansas City Missouri, and the macroscopic travel demand model maintained by the metropolitan planning organization. Here, the result shows the proposed model achieved a satisfactory result with a R square value of 0.54 and root mean square error of 0.14, both better than multiple linear regression model and random forest model. To reduce range anxiety, it is suggested that the priorities of deploying new charging facilities should be given to the areas with higher daily traffic prediction, with more conservative EV users or that are further from residential areas.

33 ADVANCED PROPULSION SYSTEMS↗

A hybrid CNN-LSTM surrogate model for hyper-resolution spatiotemporal flood forecasting in Norfolk, Virginia

Study region: Norfolk, Virginia, United States Study focus: Accurate and timely flood forecasting is essential for enhancing resilience in coastal urban areas in the context of increasing frequency and intensity of rainfall, sea level rise and urbanization. This study presents a hybrid deep learning-based surrogate model that integrates Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks to enable real-time spatiotemporal flood forecasting. The model leverages CNN to capture spatial features from inputs such as elevation and Topographic Wetness Index (TWI), while LSTM processes time-series inputs of rainfall and tide data to capture temporal features. New hydrologic insights for the region: The hybrid CNN-LSTM model was trained using the physics-based hydrodynamic model simulations obtained from the Two-dimensional Unsteady FLOW (TUFLOW) model for Norfolk, Virginia, and achieved high predictive accuracy across diverse flood-prone areas. The reduced computational time from four to six hours using TUFLOW to 3.2 min per event using CNN-LSTM enables rapid flood inundation mapping and early warning applications. The model effectively captured both spatial flood extents and their temporal evolution across different flooding scenarios, providing forecasts at a 2.5-m spatial resolution and 15-min temporal resolution and a one-hour-ahead prediction horizon. While challenges remain in terms of transferability to new regions and real-time data assimilation, this approach demonstrates strong potential for supporting operational flood risk management in coastal urban environments.

Coastal urban flooding↗

Real-Time Movement-Based Traffic Volume Prediction at Signalized Intersections

The traffic volume of each movement at signalized intersections can provide valuable information on real-time traffic conditions that enable traffic control systems to dynamically respond to the fluctuated traffic demands. Real-time movement-based traffic volume prediction is challenging due to various nonlinear spatial relationships at different locations/approaches and the complicated underlying temporal dependencies. In this study, a novel deep intersection spatial-temporal network (DISTN) is developed for real-time movement-based traffic volume prediction at signalized intersections, which considers both spatial and temporal features by the convolutional neural network (CNN) and long short-term memory (LSTM), respectively. In addition, the within-day, daily, and weekly periodic trends of traffic volume are also considered in the proposed model. This is the first time that a deep-learning method has been applied for movement-based traffic volume prediction at signalized intersections. In the numerical experiment, the proposed model is evaluated using real-world data and simulation data to demonstrate its effectiveness. The impacts of various structures of traffic networks on the proposed model are also discussed. Results show that the proposed model outperforms some of the state-of-the-art volume prediction methods currently in the literature.

99 GENERAL AND MISCELLANEOUS↗

Deep-learning-based canopy height model generation from sub-meter resolution panchromatic satellite imagery

Canopy height models (CHMs) with sufficient resolution to distinguish individual trees are useful for a variety of applications. However, standard techniques to acquire such data, such as airborne lidar surveying, are often prohibitively expensive. Deep learning techniques for generating CHMs from high-resolution imagery are an attractive option to reduce costs. To date, success with these methods has been demonstrated using multichannel aerial photography and specialized satellite data products derived from multiple sensors, neither of which is commonly available at temporal resolutions finer than one year. Here we demonstrate a method to generate sub-meter resolution CHMs in three forests in California using a more abundant data source: sub-meter resolution, panchromatic satellite imagery from a single sensor. We show that phenology and species composition play important roles in model transferability; when trained using imagery from a single conifer forest in autumn, the model performs well on autumn imagery from a second conifer forest several hundred kilometers distant with no re-training. With modest additions to the training dataset, the same model generates minimally biased estimates of canopy height in both conifer and deciduous forests during multiple seasons. Because the model operates on satellite data with global coverage and a relatively short return interval, we propose its suitability to extrapolate tree-level canopy height data to remote regions and conduct high-temporal resolution monitoring of forest structure. We furthermore demonstrate the workflow’s applicability to fire modeling by conducting simulations in forests populated by trees measured using both this approach and airborne lidar surveying. We find minimal differences in fire behavior relative to a baseline case in which only statistical distributions of tree height and crown area are known. This result underscores the value of forest structural information derived from our workflow for improving the fidelity of wildland fire simulations, among other ecological applications.

54 ENVIRONMENTAL SCIENCES↗

Improving net ecosystem CO2 flux prediction using memory-based interpretable machine learning

Terrestrial ecosystems play a central role in the global carbon cycle and affect climate change. However, our predictive understanding of these systems is still limited due to their complexity and uncertainty about how key drivers and their legacy effects influence carbon fluxes. Here, we propose an interpretable Long Short-Term Memory (iLSTM) network for predicting net ecosystem CO 2 exchange (NEE) and interpreting the influence on the NEE prediction from environmental drivers and their memory effects. We consider five drivers and apply the method to three forest sites in the United States. Besides performing the prediction in each site, we also conduct transfer learning by using the iLSTM model trained in one site to predict at other sites. Results show that the iLSTM model produces good NEE predictions for all three sites and, more importantly, it provides reasonable interpretations on the input driver's importance as well as their temporal importance on the NEE prediction. Additionally, the iLSTM model demonstrates good across-site transferability in terms of both prediction accuracy and interpretability. The transferability can improve the NEE prediction in unobserved forest sites, and the interpretability advances our predictive understanding and guides process-based model development.

Liu, Siyan↗

Design, tuning, and blackbox optimization of laser systems

Chirped pulse amplification (CPA) and subsequent nonlinear optical (NLO) systems constitute the backbone of myriad advancements in semiconductor manufacturing, communications, biology, defense, and beyond. Accurately and efficiently modeling CPA+NLO-based laser systems is challenging because of the complex coupled processes and diverse simulation frameworks. Our modular start-to-end model unlocks the potential for exciting new optimization and inverse design approaches reliant on data-driven machine learning methods, providing a means to create tailored CPA+NLO systems unattainable with current models. To demonstrate this new, to our knowledge, technical capability, we present a study on the LCLS-II photo-injector laser, representative of a high-power and spectro-temporally non-trivial CPA+NLO system.

47 OTHER INSTRUMENTATION↗

High Performance Geocomputation for Assessing Human Dynamics at Planet Scale

Settlements are key indicators of human presence on the landscape. Large scale mapping of human settlements and their morphology from very high-resolution satellite images is a critical step towards developing an interpretative understanding of population distribution and the sociocultural attributes of the built environment they live in. Convolutional neural network (CNN) based Deep Learning experiments indicates that such computations can be scaled to some of the largest high-performance computing (HPC) architectures. While early results are encouraging for developing settlement and corresponding population maps at unprecedented speed and spatial resolutions, characterizing human dynamics at planet scale with high temporal resolution will require the community to develop novel geocomputational infrastructures and ecosystems.

Bhaduri, Budhu↗