Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “temporal learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

Predicting CO 2 Plume Migration in Heterogeneous Formations Using Conditional Deep Convolutional Generative Adversarial Network

Numerical simulation of flow and transport in heterogeneous formations has long been studied, especially for uncertainty quantification and risk assessment. The high computational cost associated with running large-scale numerical simulations in a Monte Carlo sense has motivated the development of surrogate models, which aim to capture the important input-output relations of physics-based models but require only a fraction of the cost of full model runs. In this work, we formulate a conditional deep convolutional generative adversarial network (cDC-GAN) surrogate model to learn the dynamic functional mappings in multiphase models. The cDC-GAN belongs to a class of semisupervised learning methods that can be used to learn the data generation processes. Like the original GAN, a main strength of the cDC-GAN is that it includes a self-training scheme for improving the quality of generative modeling in a game theoretic framework, without requiring extensive statistical knowledge and assumptions on input data distributions. In particular, our cDC-GAN model is designed to learn cross-domain mappings between high-dimensional input (e.g., permeability) and output (e.g., phase saturations) pairs, with the ability to incorporate conditioning information (e.g., prediction time). As a use case, we demonstrate the performance of cDC-GAN for predicting the migration of carbon dioxide (CO 2 ) plume in heterogeneous carbon storage reservoirs, which has both numerical and practical significance because of the safe storage requirements now mandated in many countries. Results show that cDC-GAN achieves high accuracy in predicting the spatial and temporal evolution patterns of the injected CO 2 plume, as compared to the original results obtained using a compositional reservoir simulator. The performance of cDC-GAN models, trained using the same number of training samples, stays relatively robust when the level of spatial heterogeneity is increased. Our cDC-GAN is pattern based and is not limited by the underlying physics. Thus, it provides a general framework for developing surrogate models, and for conducting uncertainty analyses for a wide range of physics-based models used in both groundwater and subsurface energy exploration applications.

54 ENVIRONMENTAL SCIENCES↗

Modeling and observations of North Atlantic cyclones: Implications for U.S. Offshore wind energy

To meet the Biden-Harris administration's goal of deploying 30 GW of offshore wind power by 2030 and 110 GW by 2050, expansion of wind energy into U.S. territorial waters prone to tropical cyclones (TCs) and extratropical cyclones (ETCs) is essential. This requires a deeper understanding of cyclone-related risks and the development of robust, resilient offshore wind energy systems. Here, this paper provides a comprehensive review of state-of-the-science measurement and modeling capabilities for studying TCs and ETCs, and their impacts across various spatial and temporal scales. We explore measurement capabilities for environments influenced by TCs and ETCs, including near-surface and vertical profiles of critical variables that characterize these cyclones. The capabilities and limitations of Earth system and mesoscale models are assessed for their effectiveness in capturing atmosphere–ocean–wave interactions that influence TC/ETC-induced risks under a changing climate. Additionally, we discuss microscale modeling capabilities designed to bridge scale gaps from the weather scale (a few kilometers) to the turbine scale (dozens to a few meters). We also review machine learning (ML)-based, data-driven models for simulating TC/ETC events at both weather and wind turbine scales. Special attention is given to extreme metocean conditions like extreme wind gusts, rapid wind direction changes, and high waves, which pose threats to offshore wind energy infrastructure. Finally, the paper outlines the research challenges and future directions needed to enhance the resilience and design of next-generation offshore wind turbines against extreme weather conditions.

17 WIND ENERGY↗

Recurrent convolutional neural networks for modeling nonadiabatic dynamics of quantum-classical systems

Recurrent neural networks (RNNs) have recently been extensively applied to model the time evolution in fluid dynamics, weather predictions, and even chaotic systems due to their ability to capture temporal dependencies and sequential patterns in data. Here we present an RNN model based on convolutional neural networks for modeling the nonlinear nonadiabatic dynamics of hybrid quantum-classical systems. The dynamical evolution of the hybrid systems is governed by equations of motion for classical degrees of freedom and von Neumann equation for electrons. The Physics-Aware Recurrent Convolution (PARC) neural network structure incorporates a differentiator-integrator architecture that inductively models the spatiotemporal dynamics of generic physical systems. Here, we apply our RNN approach to learn the space-time evolution of a one-dimensional semiclassical Holstein model after an interaction quench. For shallow quenches (small changes in electron-lattice coupling), the deterministic dynamics can be accurately captured using a single-CNN-based recurrent network. In contrast, deep quenches induce chaotic evolution, making long-term trajectory prediction significantly more challenging. Nonetheless, we demonstrate that the PARC-CNN architecture can effectively learn the statistical climate of the Holstein model under deep-quench conditions.

Holstein model↗

Quantifying the Known Unknown: Including Marine Sources of Greenhouse Gases in Climate Modeling

Researchers have recently estimated that Arctic submarine permafrost currently traps 60 billion tons of methane and contains 560 billion tons of organic carbon in seafloor sediments and soil, a giant pool of carbon with potentially large feedbacks on the climate system. Unlike terrestrial permafrost, the submarine permafrost system has remained a “known unknown” because of the difficulty in acquiring samples and measurements. Consequently, this potentially large carbon stock never yet considered in global climate models or policy discussions, represents a real wildcard in our understanding of Earth’s climate. This report summarizes our group’s effort at developing a numerical modeling framework designed to produce a first-of-its-kind estimate of Arctic methane gas releases from the marine sediments to the water column, and potentially to the atmosphere, where positive climate feedback may occur. Newly developed modeling capability supported by the Laboratory Directed Research and Development (LDRD) program at Sandia National Laboratories now gives us the ability to probabilistically map gas distribution and quantity in the seabed by using a hybrid approach of geospatial machine learning, and predictive numerical thermodynamic ensemble modeling. The novelty in this approach is its ability to produce maps of useful data in regions that are only sparsely sampled, a common challenge in the Arctic, and a major obstacle to progress in the past. By applying this model to the circum-Arctic continental shelves and integrating the flux of free gas from in situ methanogenesis and dissociating gas hydrates from the sediment column under climate forcing, we can provide the most reliable estimate of a spatially and temporally varying source term for greenhouse gas flux that can be used by global oceanographic circulation and Earth system models (such as DOE’s E3SM). The result will allow us to finally tackle the wildcard of the submarine permafrost carbon system, and better inform us about the severity of future national security threats that sustained climate change poses.

54 ENVIRONMENTAL SCIENCES↗

Unsupervised Clustering of Microseismic Events and Focal Mechanism Analysis at the CO 2 Injection Site in Decatur, Illinois

Characterization of induced microseismicity at a carbon dioxide (CO 2 ) storage site is critical for preserving reservoir integrity and mitigating seismic hazards. We apply a multilevel machine learning (ML) approach that combines the nonnegative matrix factorization and hidden Markov model to extract spectral representations of microseismic events and cluster them to identify seismic patterns at the Illinois Basin-Decatur Project. Unlike traditional waveform correlation methods, this approach leverages spectral characteristics of first arrivals to improve event classification and detect previously undetected planes of weakness. By integrating ML-based clustering with focal mechanism analysis, we resolve small-scale fault structures that are below the detection limits of conventional seismic imaging. Our findings reveal temporal bursts of microseismicity associated with brittle failure, providing insights into the spatio-temporal evolution of fault reactivation during CO 2 injection. This approach enhances seismic monitoring capabilities at CO 2 injection sites by improving fault characterization beyond the resolution of standard geophysical surveys.

Willis, Rachel Marie [Sandia National Laboratories↗

Unifying Temporal and Structural Credit Assignment Problems

Single-agent reinforcement learners in time-extended domains and multi-agent systems share a common dilemma known as the credit assignment problem. Multi-agent systems have the structural credit assignment problem of determining the contributions of a particular agent to a common task. Instead, time-extended single-agent systems have the temporal credit assignment problem of determining the contribution of a particular action to the quality of the full sequence of actions. Traditionally these two problems are considered different and are handled in separate ways. In this article we show how these two forms of the credit assignment problem are equivalent. In this unified frame-work, a single-agent Markov decision process can be broken down into a single-time-step multi-agent process. Furthermore we show that Monte-Carlo estimation or Q-learning (depending on whether the values of resulting actions in the episode are known at the time of learning) are equivalent to different agent utility functions in a multi-agent system. This equivalence shows how an often neglected issue in multi-agent systems is equivalent to a well-known deficiency in multi-time-step learning and lays the basis for solving time-extended multi-agent problems, where both credit assignment problems are present.

Agogino, Adrian K.↗

The application of artificial intelligence to astronomical scheduling problems

Efficient utilization of expensive space- and ground-based observatories is an important goal for the astronomical community; the cost of modern observing facilities is enormous, and the available observing time is much less than the demand from astronomers around the world. The complexity and variety of scheduling constraints and goals has led several groups to investigate how artificial intelligence (AI) techniques might help solve these kinds of problems. The earliest and most successful of these projects was started at Space Telescope Science Institute in 1987 and has led to the development of the Spike scheduling system to support the scheduling of Hubble Space Telescope (HST). The aim of Spike at STScI is to allocate observations to timescales of days to a week observing all scheduling constraints and maximizing preferences that help ensure that observations are made at optimal times. Spike has been in use operationally for HST since shortly after the observatory was launched in Apr. 1990. Although developed specifically for HST scheduling, Spike was carefully designed to provide a general framework for similar (activity-based) scheduling problems. In particular, the tasks to be scheduled are defined in the system in general terms, and no assumptions about the scheduling timescale are built in. The mechanisms for describing, combining, and propagating temporal and other constraints and preferences are quite general. The success of this approach has been demonstrated by the application of Spike to the scheduling of other satellite observatories: changes to the system are required only in the specific constraints that apply, and not in the framework itself. In particular, the Spike framework is sufficiently flexible to handle both long-term and short-term scheduling, on timescales of years down to minutes or less. This talk will discuss recent progress made in scheduling search techniques, the lessons learned from early HST operations, the application of Spike to other problem domains, and plans for the future evolution of the system.

Johnston, Mark D.↗

Hidden Markov models for fault detection in dynamic systems

The invention is a system failure monitoring method and apparatus which learns the symptom-fault mapping directly from training data. The invention first estimates the state of the system at discrete intervals in time. A feature vector x of dimension k is estimated from sets of successive windows of sensor data. A pattern recognition component then models the instantaneous estimate of the posterior class probability given the features, p(w(sub i) (vertical bar)/x), 1 less than or equal to i isless than or equal to m. Finally, a hidden Markov model is used to take advantage of temporal context and estimate class probabilities conditioned on recent past history. In this hierarchical pattern of information flow, the time series data is transformed and mapped into a categorical representation (the fault classes) and integrated over time to enable robust decision-making.

Smyth, Padhraic J.↗

Linking OH Variability to Observable Variables, Meteorology and Transport

The hydroxyl radical (OH) plays a vital role in tropospheric chemistry, as it provides the dominant sink for a multitude of pollutants and climate-relevant gases such as methane. Observational constraints on the global distribution and temporal variability of OH are limited, and models simulate a wide range of OH distributions. While OH itself has a short atmospheric lifetime, OH is photochemically coupled to longer-lived species that undergo atmospheric transport. Here, we investigate how much of the OH variability within and between models can be explained by differences in observable species to develop diagnostics for OH differences. We find that NO 2 and water vapor together explain much of the spatial and temporal variability in simulated OH, and we use satellite observations to identify biases in these variables. The OH response to ENSO also differs between models, and we investigate potential causes of these differences such as differences in convection or lightning NOx. We also explore the potential of idealized tracers to represent the OH distribution. Within a single model, meteorological variables such as humidity and idealized tracers of transport can explain a significant portion of the OH spatial variability. We use a Gradient Boosted Regression Trees, a type of machine learning, to account for non-linear relationships between OH and the input variables.

Meteorology↗

Foundation Models of Scientific Knowledge for Chemistry: Opportunities, Challenges and Lessons Learned

Foundation models pre-trained on large corpora demonstrate significant gains across many natural language processing tasks and domains e.g., law, healthcare, education, etc. However, only limited efforts have investigated the opportunities and limitations of applying these powerful models to science and security applications. In this work we develop foundation models of scientific knowledge for chemistry to augment scientists with the advanced ability to perceive and reason at scale previously unimagined. Specifically, we build large-scale (1.47B parameter) general-purpose models for chemistry that can be effectively used to perform a wide range of in-domain and out-of-domain tasks. Evaluating these models in a zero-shot setting, we analyze the effect of model and data scaling, knowledge depth, and temporality on model performance in context of model training efficiency. Our novel findings demonstrate that (1) model size significantly contributes to the task performance when evaluated in a zero-shot setting; (2) data quality (aka diversity) affects model performance more than data quantity; (3) similarly, unlike previous work (Luu et al., 2021) temporal order of the documents in the corpus boosts model performance only for specific tasks, e.g., SciQ; and (4) models pre-trained from scratch perform better on in-domain tasks than those tuned from general-purpose models like Open AI’s GPT-2.

Foundation Models, Chemistry↗

Hidden Markov models for fault detection in dynamic systems

The invention is a system failure monitoring method and apparatus which learns the symptom-fault mapping directly from training data. The invention first estimates the state of the system at discrete intervals in time. A feature vector x of dimension k is estimated from sets of successive windows of sensor data. A pattern recognition component then models the instantaneous estimate of the posterior class probability given the features, p(w(sub i) perpendicular to x), 1 less than or equal to i is less than or equal to m. Finally, a hidden Markov model is used to take advantage of temporal context and estimate class probabilities conditioned on recent past history. In this hierarchical pattern of information flow, the time series data is transformed and mapped into a categorical representation (the fault classes) and integrated over time to enable robust decision-making.

Smyth, Padhraic J.↗

Global Landslide Hazard Assessment for Situational Awareness (LHASA) Version 2: New Activities and Future Plans

A remote sensing-based system has been developed to characterize the potential for rainfall-triggered landslides across the globe in near real-time. The Landslide Hazard Assessment for Situational Awareness (LHASA) model uses a decision tree framework to combine a static susceptibility map derived from information on slope, rock characteristics, forest loss, distance to fault zones and distance to road networks with satellite precipitation estimates from the Global Precipitation Measurement (GPM) mission. Since 2016, the LHASA model has been providing near real-time and retrospective estimates of potential landslide activity. Results of this work are available at https://landslides.nasa.gov. In order to advance LHASA’s capabilities to characterize landslide hazards and impacts dynamically, we have implemented a new approach that leverages machine learning, new parameters, and new inventories. LHASA 2.0 uses the XGBoost machine learning model to bring in dynamic variables as well as additional static variables to better represent landslide hazard globally. Global rainfall forecasts are also being evaluated to provide a 1-3 day forecast of potential landslide activity. Additional factors such as recent seismicity and burned areas are also being considered to represent the preconditioning or changing interactions with subsequent rainfall over affected areas. A series of parameters are being tested within this structure using NASA’s Global Landslide Catalog as well as many other event-based and multi-temporal inventories mapped by the project team or provided by project partners. In addition to estimates of landslide hazard, LHASA Version 2 will incorporate dynamic estimates of exposure including population, roads and infrastructure to highlight the potential impacts that rainfall-triggered landslides. The ultimate goal of LHASA Version 2.0 is to approximate the relative probabilities of landslide hazard and exposure across different space and time scales to inform hazard assessment retrospectively over the past 20 years, in near real-time, and in the future. In addition to the hazard. This presentation will outline the new activities for LHASA Version 2.0 and present some next steps for this system.

Dalia Kirschbaum↗

Plenario: An Open Data Discovery and Exploration Platform for Urban Science

The past decade has seen the widespread release of open data concerning city services, conditions, and activities by government bodies and public institutions of all sizes. Hundreds of open data portals now host thousands of datasets of many different types. These new data sources represent enormous po- tential for improved understanding of urban dynamics and processes—and, ultimately, for more livable, efficient, and prosperous communities. However, those who seek to realize this potential quickly discover that discovering and applying those data relevant to any particular question can be extraordinarily dif- ficult, due to decentralized storage, heterogeneous formats, and poor documentation. In this context, we introduce Plenario, a platform designed to automating time-consuming tasks associated with the discovery, exploration, and application of open city data—and, in so doing, reduce barriers to data use for researchers, policymakers, service providers, journalists, and members of the general public. Key innovations include a geospatial data warehouse that allows data from many sources to be registered into a common spatial and temporal frame; simple and intuitive interfaces that permit rapid discovery and exploration of data subsets pertaining to a particular area and time, regardless of type and source; easy export of such data subsets for further analysis; a user-configurable data ingest framework for automated importing and periodic updating of new datasets into the data warehouse; cloud hosting for elastic scaling and rapid creation of new Plenario instances; and an open source implementation to enable community contributions. We describe here the architecture and implementation of the Plenario platform, discuss lessons learned from its use by several communities, and outline plans for future work.

Catlett, Charlie↗

DETECTING FIRE WITH MACHINE LEARNING-ENABLED VISUAL MONITORING FOR NUCLEAR POWER PLANT ENVIRONMENTS

Nuclear power plants are experiencing significant cost challenges to remain competitive with other energy-generation utilities. Unlike other industries, the cost of operation and maintenance activities is mostly attributed to workforce costs. To mitigate this, nuclear power plant stakeholders are increasingly interested in the development and deployment of machine learning methods to potentially automate or augment manually intensive tasks to reduce costs, especially for monitoring activities. One monitoring function that is visually demanding and that can occur frequently to meet the requirements of a fire protection program is visually monitoring an area for fire occurrence. Currently, fire watch activities consist of a worker physically stationed at a given location with the sole responsibility of observing a given area to ensure a fire is detected and mitigated promptly. This effort focused on the development and evaluation of a suitable deep convolutional neural network to classify individual video frames at a sub-second frequency for the occurrence of “fire” and “no fire” in varying industrial environments similar to nuclear power plants. It is believed that a trained neural network model could be integrated with existing facility video surveillance camera feeds to generate alerts when fire inferences occur in individual frames captured at sub-second temporal resolutions. Extensive effort was dedicated to identifying and curating suitable imagery training data representing varying environments and scene settings with and without flame features to maximize generalization in nuclear power plant environments. The data collection effort resulted in the aggregation of a large, labeled image library exceeding 12,000 images to support model training for diverse industrial environments. A deep neural network model incorporating parallel multi-scale capabilities was developed and trained to support accurate image-based detection of flame incidents of varying sizes and spectral feature properties within heterogeneous scenes. Analysis results show that the trained model can achieve high inference accuracy despite heterogeneous scene environments and components. Testing accuracy exceeded 95.0 percent with very low false positive and false negative inferences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Crash Risks Evaluation of Urban Expressways: A Case Study in Shanghai

We report that proactive traffic safety management systems can reduce crashes by identifying crash precursors, evaluating real-time crash risks, and implementing suitable interventions. The basic prerequisite for developing such a system is to propose a reliable crash risk evaluation model that takes real-time traffic flow data as input. Previous studies have primarily focused on real-time crash prediction using some statistical or machine-learning methods. However, further quantitative evaluation and classification of crash risks have been ignored. In this study, we conduct a systematic crash risk evaluation workflow, including crash risk prediction, crash risk quantification, and crash risk classification. Specifically, the crash risk prediction using an extended logit model is proposed, from which CAS, CSD, UAS, DAS, DTV are identified to be contributing factors of crash risks. Then a crash risk quantification model based on the parameter evaluation of the extended logit model is developed. The crash risks of urban expressways and their spatial-temporal evolution trends are quantified. Finally, the crash risks are classified into high crash risk level, moderate crash risk level, and low crash risk level by the k-means cluster algorithm. Then the threshold boundaries of different crash risk levels are determined. The research results provide a proactive guidance for traffic safety management of urban expressways.

33 ADVANCED PROPULSION SYSTEMS↗

The Role of Dimensionality in Understanding Granuloma Formation

Within the first 2–3 months of a Mycobacterium tuberculosis (Mtb) infection, 2–4 mm spherical structures called granulomas develop in the lungs of the infected hosts. These are the hallmark of tuberculosis (TB) infection in humans and non-human primates. A cascade of immunological events occurs in the first 3 months of granuloma formation that likely shapes the outcome of the infection. Understanding the main mechanisms driving granuloma development and function is key to generating treatments and vaccines. In vitro, in vivo, and in silico studies have been performed in the past decades to address the complexity of granuloma dynamics. This study builds on our previous 2D spatio-temporal hybrid computational model of granuloma formation in TB (GranSim) and presents for the first time a more realistic 3D implementation. We use uncertainty and sensitivity analysis techniques to calibrate the new 3D resolution to non-human primate (NHP) experimental data on bacterial levels per granuloma during the first 100 days post infection. Due to the large computational cost associated with running a 3D agent-based model, our major goal is to assess to what extent 2D and 3D simulations differ in predictions for TB granulomas and what can be learned in the context of 3D that is missed in 2D. Our findings suggest that in terms of major mechanisms driving bacterial burden, 2D and 3D models return very similar results. For example, Mtb growth rates and molecular regulation mechanisms are very important both in 2D and 3D, as are cellular movement and modulation of cell recruitment. The main difference we found was that the 3D model is less affected by crowding when cellular recruitment and movement of cells are increased. Overall, we conclude that the use of a 2D resolution in GranSim is warranted when large scale pilot runs are to be performed and if the goal is to determine major mechanisms driving infection outcome (e.g., bacterial load). To comprehensively compare the roles of model dimensionality, further tests and experimental data will be needed to expand our conclusions to molecular scale dynamics and multi-scale resolutions.

2D vs. 3D↗

Using remotely sensed information to support landslide hazard and exposure assessment throughout the disaster lifecycle

The global coverage and temporal frequency that satellites provide offers a unique opportunity to estimate landslide hazard and exposure throughout the disaster lifecycle, from pre-event planning and forecasting to post-event mapping and impact assessment, and finally to recovery and mitigation. The relevance of satellite-derived data and model products is largely contingent on the spatiotemporal sampling, the hazard characteristics, and the needs from the research or applications community. This work presents an advanced Landslide Hazard Assessment for Situational Awareness (LHASA) framework that brings together satellite and model products with new machine learning techniques and global inventory data to better model landslide hazard and exposure. We present several new ways to map, model, and assess landslide hazard using a range of satellite data. Two new thrusts of this work are to better account for the exacerbating impacts of fires and to provide a multi-day forecast of potential hazard. Together, these additional components blend information from a suite of satellite and model sources to improve early warning of potentially hazardous areas, identify landslide occurrence and impacts in near real-time, and better characterize the spatiotemporal patterns of landslide hazard and exposure more broadly for future awareness and planning. This suite of tools and products is open to the public and provides information to better assess the potential occurrence and impacts of landslides within different regions of the world. This presentation explores both the architecture behind this framework and examples of how the model components and products have been used by different stakeholders around the world.

Thomas Stanley↗

Meeting Global Health Needs via Infectious Disease Forecasting: Development of a Reliable Data-Driven Framework

Infectious diseases (IDs) have a significant detrimental impact on global health. Timely and accurate ID forecasting can result in more informed implementation of control measures and prevention policies. To meet the operational decision-making needs of real-world circumstances, we aimed to build a standardized, reliable, and trustworthy ID forecasting pipeline and visualization dashboard that is generalizable across a wide range of modeling techniques, IDs, and global locations. We forecasted 6 diverse, zoonotic diseases (brucellosis, campylobacteriosis, Middle East respiratory syndrome, Q fever, tick-borne encephalitis, and tularemia) across 4 continents and 8 countries. We included a wide range of statistical, machine learning, and deep learning models (n=9) and trained them on a multitude of features (average n=2326) within the One Health landscape, including demography, landscape, climate, and socioeconomic factors. The pipeline and dashboard were created in consideration of crucial operational metrics—prediction accuracy, computational efficiency, spatiotemporal generalizability, uncertainty quantification, and interpretability—which are essential to strategic data-driven decisions. While no single best model was suitable for all disease, region, and country combinations, our ensemble technique selects the best-performing model for each given scenario to achieve the closest prediction. For new or emerging diseases in a region, the ensemble model can predict how the disease may behave in the new region using a pretrained model from a similar region with a history of that disease. The data visualization dashboard provides a clean interface of important analytical metrics, such as ID temporal patterns, forecasts, prediction uncertainties, and model feature importance across all geographic locations and disease combinations. As the need for real-time, operational ID forecasting capabilities increases, this standardized and automated platform for data collection, analysis, and reporting is a major step forward in enabling evidence-based public health decisions and policies for the prevention and mitigation of future ID outbreaks.

60 APPLIED LIFE SCIENCES↗