Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “temporal learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Rethinking skip connections in Spiking Neural Networks with Time-To-First-Spike coding

Time-To-First-Spike (TTFS) coding in Spiking Neural Networks (SNNs) offers significant advantages in terms of energy efficiency, closely mimicking the behavior of biological neurons. In this work, we delve into the role of skip connections, a widely used concept in Artificial Neural Networks (ANNs), within the domain of SNNs with TTFS coding. Our focus is on two distinct types of skip connection architectures: (1) addition-based skip connections, and (2) concatenation-based skip connections. We find that addition-based skip connections introduce an additional delay in terms of spike timing. On the other hand, concatenation-based skip connections circumvent this delay but produce time gaps between after-convolution and skip connection paths, thereby restricting the effective mixing of information from these two paths. To mitigate these issues, we propose a novel approach involving a learnable delay for skip connections in the concatenation-based skip connection architecture. This approach successfully bridges the time gap between the convolutional and skip branches, facilitating improved information mixing. We conduct experiments on public datasets including MNIST and Fashion-MNIST, illustrating the advantage of the skip connection in TTFS coding architectures. Additionally, we demonstrate the applicability of TTFS coding on beyond image recognition tasks and extend it to scientific machine-learning tasks, broadening the potential uses of SNNs.

97 MATHEMATICS AND COMPUTING↗

Discovering multi-scale co-occurrence patterns of asthma and influenza with the Oak Ridge bio-surveillance toolkit

Here, we describe a data-driven unsupervised machine learning approach to extract geo-temporal co-occurrence patterns of asthma and the flu from large-scale electronic healthcare reimbursement claims (eHRC) datasets. Specifically, we examine the eHRC data from 2009 to 2010 pandemic H1N1 influenza season and analyze whether different geographic regions within the United States (US) showed an increase in co-occurrence patterns of the flu and asthma. Our analyses reveal that the temporal patterns extracted from the eHRC data show a distinct lag time between the peak incidence of the asthma and the flu. While the increased occurrence of asthma contributed to increased flu incidence during the pandemic, this co-occurrence is predominant for female patients. The geo-temporal patterns reveal that the co-occurrence of the flu and asthma are typically concentrated within the south-east US. Further, in agreement with previous studies, large urban areas (such as New York, Miami, and Los Angeles) exhibit co-occurrence patterns that suggest a peak incidence of asthma and flu significantly early in the spring and winter seasons. Together, our data-analytic approach, integrated within the Oak Ridge Bio-surveillance Toolkit platform, demonstrates how eHRC data can provide novel insights into co-occurring disease patterns.

60 APPLIED LIFE SCIENCES↗

Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2

Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.

Sergii Skakun↗

Characterizing mammography reports for health analytics

As massive collections of digital health data are becoming available, the opportunities for large scale automated analysis increase. In particular, the widespread collection of detailed health information is expected to help realize a vision of evidence-based public health and patient-centric health care. Within such a framework for large scale health analytics we describe several methods to characterize and analyze free-text mammography reports, including their temporal dimension, using information retrieval, supervised learning, and classical statistical techniques. We present experimental results with a large collection of mostly unlabeled reports that demonstrate the validity and usefulness of the approach, since these results are consistent with the known features of the data and provide novel insights about it.

60 APPLIED LIFE SCIENCES↗

Real-time monitoring and prediction of water quality parameters and algae concentrations using microbial potentiometric sensor signals and machine learning tools

We report the overarching hypothesis of this study was that temporal microbial potentiometric sensor (MPS) signal patterns could be used to predict changes in commonly monitored water quality parameters by using artificial intelligence/machine learning tools. To test this hypothesis, the study first examines a proof of concept by correlating between MPS's signals and high algae concentrations in an algal cultivation pond. Then, the study expanded upon these findings and examined if multiple water quality parameters could be predicted in real surface waters, like irrigation canals. Signals generated between the MPS sensors and other water quality sensors maintained by an Arizona utility company, including algae and chlorophyll, were collected in real time at time intervals of 30 min over a period of 9 months. Data from the MPS system and data collected by the utility company were used to train the ML/AI algorithms and compare the predicted with actual water quality parameters and algae concentrations. Based on the composite signal obtained from the MPS, the ML/AI was used to predict the canal surface water's turbidity, conductivity, chlorophyll, and blue-green algae (BGA), dissolved oxygen (DO), and pH, and predicted values were compared to the measured values. Initial testing in the algal cultivation pond revealed a strong linear correlation (R 2 = 0.87) between mixed liquor suspended solids (MLSS) and the MPSs' composite signals. The Normalized Root Mean Square Error (NRMSE) between the predicted values and measured values were <6.5%, except for the DO, which was 10.45%. The results demonstrate the usefulness of MPSs to predict key surface water quality parameters through a single composite signal, when the ML/AI tools are used conjunctively to disaggregate these signal components. The maintenance-free MPS offers a novel and cost-effective approach to monitor numerous water quality parameters at once with relatively high accuracy.

54 ENVIRONMENTAL SCIENCES↗

NeuralCubes: Deep Representations for Visual Data Exploration

Visual exploration of large multi-dimensional datasets has seen tremendous progress in recent years, allowing users to express rich data queries that produce informative visual summaries, all in real time. Techniques based on data cubes are some of the most promising approaches. However, these techniques usually require a large memory footprint for large datasets. To tackle this problem, we present NeuralCubes: neural networks that predict results for aggregate queries, similar to data cubes. NeuralCubes learns a function that takes as input a given query, for instance, a geographic region and temporal interval, and outputs the result of the query. The learned function serves as a real-time, low-memory approximator for aggregation queries. Our models are small enough to be sent to the client side (e.g. the web browser for a web-based application) for evaluation, enabling data exploration of large datasets without database/network connection. Here, we demonstrate the effectiveness of NeuralCubes through extensive experiments on a variety of datasets and discuss how NeuralCubes opens up opportunities for new types of visualization and interaction.

97 MATHEMATICS AND COMPUTING↗

A cross-dimensional analysis of data-driven short-term load forecasting methods with large-scale smart meter data

Electricity load forecasting is essential to utility operation and power grid stability. A wide spectrum of data-driven methods, ranging from linear regression models to more recent deep learning models have been adopted to forecast electric load over the years. However, there still lacks a holistic evaluation of the applicability of conventional statistical and machine learning based algorithms with respect to different temporal and spatial scopes, computational requirements, and sensitivity of model-tuning. Enabled by a large-scale electricity load profile dataset of over 40,000 residential customers in a utility region, we conducted a cross-dimensional analysis of data-driven load forecasting methods. Three regression-based and seven deep learning algorithms with different model configurations were evaluated in terms of their overall and peak load prediction accuracy, and training burdens, across spatial aggregation levels ranging from the transformer, feeder, substation, to neighborhood. We found, first, the load forecasting accuracy is constrained by a predictability boundary, influenced by the forecasting horizon and spatial aggregation level. Specifically, RandomForest, XGBoost, TFT, TSMixer, and TiDE models achieved less than 10 % prediction error for up to 96-h ahead forecasting for district, substation, and feeder levels, while other models struggle at long-horizon predictions; Second, for winter and summer peak load dates, most models were able to predict the peak demand timing within ± 1 h, but the prediction percentage error varied by models, with TFT and TiDE models being the top performers; Third, models with similar prediction accuracy can differ in training burden by an order of magnitude. Therefore, choosing model configurations that balance prediction performance and computational resource is an important practical consideration for large-scale deployment of the machine learning based load forecasting. The outcome of this study can guide researchers and practitioners to choose the proper load forecasting algorithms based on their problem scope, required accuracy, and available resources. The predictability boundary can serve as a benchmark for electricity load forecasting problems with new algorithms and datasets.

Li, Han↗

QUICR-learning for Multi-Agent Coordination

Coordinating multiple agents that need to perform a sequence of actions to maximize a system level reward requires solving two distinct credit assignment problems. First, credit must be assigned for an action taken at time step t that results in a reward at time step t > t. Second, credit must be assigned for the contribution of agent i to the overall system performance. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning. The second credit assignment problem is typically addressed by creating custom reward functions. To address both credit assignment problems simultaneously, we propose the "Q Updates with Immediate Counterfactual Rewards-learning" (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. QUICR-learning is based on previous work on single-time-step counterfactual rewards described by the collectives framework. Results on a traffic congestion problem shows that QUICR-learning is significantly better than a Q-learner using collectives-based (single-time-step counterfactual) rewards. In addition QUICR-learning provides significant gains over conventional and local Q-learning. Additional results on a multi-agent grid-world problem show that the improvements due to QUICR-learning are not domain specific and can provide up to a ten fold increase in performance over existing methods.

Agogino, Adrian K.↗

Quicker Q-Learning in Multi-Agent Systems

Multi-agent learning in Markov Decisions Problems is challenging because of the presence ot two credit assignment problems: 1) How to credit an action taken at time step t for rewards received at t' greater than t; and 2) How to credit an action taken by agent i considering the system reward is a function of the actions of all the agents. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning OK TD(lambda) The second credit assi,onment problem is typically addressed either by hand-crafting reward functions that assign proper credit to an agent, or by making certain independence assumptions about an agent's state-space and reward function. To address both credit assignment problems simultaneously, we propose the Q Updates with Immediate Counterfactual Rewards-learning (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. Instead of assuming that an agent s value function can be made independent of other agents, this method suppresses the impact of other agents using counterfactual rewards. Results on multi-agent grid-world problems over multiple topologies show that QUICR-learning can achieve up to thirty fold improvements in performance over both conventional and local Q-learning in the largest tested systems.

Agogino, Adrian K.↗

A Novel Machine Learning Method for Surface PM2.5 Estimations from Geostationary Satellites

Particulate matter (PM) with a diameter of less or equal to 2.5 μm, known as PM , affects human health as it penetrates the respiratory system. The Environmental Protection Agency (EPA) measures the atmospheric concentration of PM using air quality monitors stationed throughout the Continental United States (CONUS). Such measurements are points on a spatial domain and therefore, might not be representative of the air quality at nearby areas considering that the composition of the atmosphere is highly variable from place to place. Satellite based AOD permits a spatially uniform means of estimating PM and new geostationary satellites provide high temporal and spatial resolution estimation of AOD. However, the concentration of PM is non-linearly dependent on other atmospheric parameters that include relative humidity, temperature, and height of the planetary boundary layer. This information may be estimated at similar spatial and temporal resolutions as AOD from numerical modeling such as from the National Oceanic and Atmospheric Administration’s (NOAA) High Resolution Rapid Refresh (HRRR) model which resolves near real-time atmospheric conditions over the CONUS. The estimation of PM concentration is a multi-parametric problem that considers the effect of temporal dependencies among the different parameters. Deep learning approaches are appropriate for such complex estimation problems as they intrinsically capture relations among multiple non-linear parameters. This study compares deep-learning methods to traditional regression analysis to demonstrate the capabilities of these methods in predicting PM2.5 concentrations. Additionally, a novel ensemble learning approach is employed to identify scientific processes that could further improve the estimation of PM concentration. Utilizing Long Short-Term Memory (LSTM) neural networks, which are suitable for multivariate time series estimation problems as they are capable of learning long-term dependencies, individual models are created for each EPA station and trained on the aforementioned dataset collocated over each station. Individual station models are merged if the model's performance is improved by reducing the root mean squared error (RMSE) metric. This ensemble training method ultimately reduces the RMSE value. Evaluation of these results provide insights into physical processes and related observable parameters that may contribute to PM concentrations. Identified parameters evaluated to be statistically different between the merged and unmerged models are expected to improve overall performance. These new parameters are then utilized for reevaluation of the deep learning methods with an extreme gradient boosting model with an RMSE of 5.5 providing the best results.

George Priftis↗

Towards Misregistration-Tolerant Change Detection using Deep Learning Techniques with Object-Based Image Analysis

Co-registrating is a common pre-processing step for existing change detection algorithms, but registering bi-temporal images is nontrivial. The use of image patch as input for deep learning techniques provides a natural avenue to apply them in the OBIA framework, and have shown successful performance in the object-based land cover mapping and change detection applications. Even though attempts of applying deep learning techniques for change detection applications have been made with varying success, its application under OBIA framework for change detection have not been conducted and its tolerance for misregistration among temporal images are neither known. This study performed change detection under OBIA framework using deep learning techniques for the first time, and evaluated its performance regarding their tolerance of image misregistration on training and testing dataset. Our results demonstrate the proposed change detection scheme is surprisingly robust to image misregistration on the testing dataset, while classifiers trained with the training dataset containing image misregistration errors suffer from slight decrease of overall accuracy.

Liu, Tao↗

Light storage in a doped solid enhanced by feedback-controlled pulse shaping

We report on experiments dealing with feedback-controlled pulse shaping to optimize the efficiency of light storage by electromagnetically induced transparency (EIT) in a Pr{sup 3+}:Y{sub 2}SiO{sub 5} crystal. A learning loop in combination with an evolutionary algorithm permits the automatic determination of optimal temporal profiles of intensities and frequencies in the driving laser pulses (i.e., the probe and coupling pulses). As a main advantage, the technique finds optimal solutions even in the complicated multilevel excitation scheme of a doped solid, involving large inhomogeneous broadening. The learning loop experimentally determines optimal temporal intensity profiles of the coupling pulses for a given probe pulse. The optimized intensity pulse shapes enhance the light-storage efficiency in the doped solid by a factor of 2. The learning loop also determines a fast and efficient preparation pulse sequence, which serves to optically prepare the crystal prior to light-storage experiments. The optimized preparation sequence is 5 times faster than standard preparation sequences. Moreover, the optimized preparation sequence enhances the optical depth in the medium by a factor of 5. As a consequence, the efficiency of light storage also increases by another factor of 3. Our experimental data clearly demonstrate the considerable potential of feedback-controlled pulse shaping, applied to EIT-driven light storage in solid media.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Multisensor Machine Learning to Retrieve High Spatiotemporal Resolution Land Surface Temperature

Climate change is making heat waves more frequent, long-lasting, and severe. While multiple satellite types provide data to monitor surface temperature, geostationary (GEO) sensors provide near-continuous, continental-scale observations which can better capture the diurnal variability of land surface temperature (LST) than intermittent observations from low-earth orbit (LEO) sensors. However, standard products from GEO satellites are available at coarsened spatial and temporal resolutions compared to the native sensor resolution. Using datasets from the NASA Earth Exchange, we leveraged co-located, co-temporal observations from LEO and GEO satellites to learn a data-driven mapping using a convolutional neural network. The resulting NASA Earth eXchange Artificial Intelligence LST (NEXAI-LST) achieved a mean absolute error of 1.73 K relative to the target LEO product and improves on both spatial and temporal resolution [2 km, 10 minute] compared to the GEO full disk standard product [10 km, hourly]. In validation against measurements from a ground-based sensor network, NEXAI-LST achieves similar or better fit than both LEO and GEO standard products, while depending none of the prior knowledge of land surface and atmospheric states required by physical-statistical models. Further, application of the model to unseen LEO and GEO satellites demonstrates robust generalization of the model across spatial region, time of day, and sensor. In support of NASA’s open-source science initiative, we make our NEXAI-LST product, model, and codes available to facilitate data exploration and further studies.

Kate Marie Duffy↗

Frameworks, Algorithms, and Scalable Technologies for Mathematics (FASTMath) SciDAC Institute

As computational models scale to larger computers, the rate at which they produce data has far outstripped the same computers ability to write that data and further the file systems ability to store that data. Almost all of the SciDAC applications, but especially those related to fusion solve very large scale PDEs whose scientific output his impacted by this problem. To gain access to dynamics in an exascale simulation that are not identifiable a priori and to make that dynamical data available to machine learning requires fundamental research in the area of in situ data data analytics. Here data analytics includes compression, visualization, uncertainty quantification, and machine learning. This in situ data analytics will enable on-the-fly spatial and temporal compression of solution dynamics, expose that space-time compressed field to machine learning algorithms that have been specialized to work with dynamically evolving data (existing machine learning algorithms treat data sets as static), greatly improving the opportunity for machine learning to provide feedback to the compression, all within an ongoing simulation, without the need to write data to files. The same concepts are also being applied to uncertainty quantification and multi-fidelity modeling which have similar needs for spatial and temporal compression of the ongoing exascale simulation to perform either without the typical, unacceptable writing of data to files.

97 MATHEMATICS AND COMPUTING↗

AttentionFire_v1.0: interpretable machine learning fire model for burned-area predictions over tropics

Abstract. African and South American (ASA) wildfires account for more than 70 % of global burned areas and have strong connection to local climate for sub-seasonal to seasonal wildfire dynamics. However, representation of the wildfire–climate relationship remains challenging due to spatiotemporally heterogenous responses of wildfires to climate variability and human influences. Here, we developed an interpretable machine learning (ML) fire model (AttentionFire_v1.0) to resolve the complex controls of climate and human activities on burned areas and to better predict burned areas over ASA regions. Our ML fire model substantially improved predictability of burned areas for both spatial and temporal dynamics compared with five commonly used machine learning models. More importantly, the model revealed strong time-lagged control from climate wetness on the burned areas. The model also predicted that, under a high-emission future climate scenario, the recently observed declines in burned area will reverse in South America in the near future due to climate changes. Our study provides a reliable and interpretable fire model and highlights the importance of lagged wildfire–climate relationships in historical and future predictions.

58 GEOSCIENCES↗

Panoramic Mapping of Phonon Transport from Ultrafast Electron Diffraction and Scientific Machine Learning

Abstract One central challenge in understanding phonon thermal transport is a lack of experimental tools to investigate frequency‐resolved phonon transport. Although recent advances in computation lead to frequency‐resolved information, it is hindered by unknown defects in bulk regions and at interfaces. Here, a framework that can uncover microscopic phonon transport information in heterostructures is presented, integrating state‐of‐the‐art ultrafast electron diffraction (UED) with advanced scientific machine learning (SciML). Taking advantage of the dual temporal and reciprocal‐space resolution in UED, and the ability of SciML to solve inverse problems involving coupled Boltzmann transport equations, the frequency‐dependent interfacial transmittance and frequency‐dependent relaxation times of the heterostructure from the diffraction patterns are reliably recovered. The framework is applied to experimental Au/Si UED data, and a transport pattern beyond the diffuse mismatch model is revealed, which further enables a direct reconstruction of real‐space, real‐time, frequency‐resolved phonon dynamics across the interface. The work provides a new pathway to probe interfacial phonon transport mechanisms with unprecedented details.

36 MATERIALS SCIENCE↗

Deep learning models map rapid plant species changes from citizen science and remote sensing data

Anthropogenic habitat destruction and climate change are reshaping the geographic distribution of plants worldwide. However, we are still unable to map species shifts at high spatial, temporal, and taxonomic resolution. Here, we develop a deep learning model trained using remote sensing images from California paired with half a million citizen science observations that can map the distribution of over 2,000 plant species. Our model— Deepbiosphere— not only outperforms many common species distribution modeling approaches (AUC 0.95 vs. 0.88) but can map species at up to a few meters resolution and finely delineate plant communities with high accuracy, including the pristine and clear-cut forests of Redwood National Park. These fine-scale predictions can further be used to map the intensity of habitat fragmentation and sharp ecosystem transitions across human-altered landscapes. In addition, from frequent collections of remote sensing data, Deepbiosphere can detect the rapid effects of severe wildfire on plant community composition across a 2-y time period. These findings demonstrate that integrating public earth observations and citizen science with deep learning can pave the way toward automated systems for monitoring biodiversity change in real-time worldwide.

Gillespie, Lauren E.↗