Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,261 records · Page 70

Designing resilient IoT and Edge Computing with federated tinyML

The rapid growth of the Internet of Things (IoT) and Edge Computing (EC) has brought significant conveniences to modern society but has also greatly expanded the cyber attack surfaces, particularly as these technologies are being increasingly integrated into critical systems such as power grids, healthcare, and smart homes. Here, to improve IoT/EC’s cybersecurity posture, we leveraged Artificial Intelligence (AI) and Machine Learning (ML) by employing tinyML to monitor voluminous IoT data for cyber threats while addressing devices’ resource constraints, and utilizing Federated Learning (FL) to share local detection knowledge across the system while preserving privacy. Building on our three-layer architecture combining tinyML and FL to enhance autonomous cyber attack detection, this paper demonstrated that the architecture improves detection accuracy, reduces resource consumption, and enables lightweight, secure IoT device monitoring. These results were validated using the public N-BaIoT dataset as well as real IoT network traffic data collected under multiple attack scenarios from our testbeds. Additionally, we introduced an enhanced FL methodology with a novel preprocessing stage, including federated feature selection and global preprocessor construction, to address IoT/EC data heterogeneity. We developed a physical IoT testbed for attack simulations and data collection, implemented a tinyML-powered detector for realistic model validation, and also built a virtual testbed for scalable evaluations of FL models across diverse network environments.

Cognitive cyber↗

Digital Twin for Hydropower System Object Modeling: Alder Dam (FY2023)

Hydropower is the world's largest source of renewable electricity, and hydropower plants are distributed all over the world. Typical major components of a hydropower plant are the governor, excitation, generator, thrust bearing, hydraulic turbine, transformer, the main lead, metering and control, tailwater depression, and dissolved oxygen. For each component, various measures are taken. The measurements are acquired by various heterogeneous systems, including standalone sensors, programmable logic controllers (PLC), Supervisory control and data acquisition (SCADA), Internet of Things (IoT), and data acquisition and integration platforms such as OSI/PI. The measured data are often archived within the plant by a data management platform, and many institutions have cloud-based archive systems, such as Hydropower Research Institution (HRI), U.S. Army Corps of Engineers (USACE), and Columbia River Data Access in Real Time (DART). Object Modeling is a general framework for designing information systems. It focuses on objects, the actions they perform, and the messages they send to one another to cause those actions to be taken. The major differences among object modeling, network modeling, data modeling, and process modeling are that in the first we focus on the actions in response to information, objects which form the system, the actions they perform, and how they pass information to one another, while in the second we concentrate on where, when and how much information is moved, while in the third we focus on what information is moved and where it is moved, while in the last we focus on how it is moved and when it is moved. Object modeling was developed basically as a method to develop object-oriented systems and to support object-oriented programming. It describes the static structure of the system. The object Modeling Technique is easy to draw and use. That is why we choose object modeling to connect physical hydropower plants to Digital Twin. It recognizes the objects and the relationship between them. It identifies the attributes and functions of each class. Dynamic Modeling: It explains how objects respond to events. Functional Modeling indicates the processes executed in an object and how data changes when it moves to objects. It has been used in many applications like telecommunication, transportation, etc.

13 HYDRO ENERGY↗

In Silico Human Mobility Data Science: Leveraging Massive Simulated Mobility Data (Vision Paper)

Human mobility data science using trajectories or check-ins of individuals has many applications. Recently, we have seen a plethora of research efforts that tackle these applications. However, research progress in this field is limited by a lack of large and representative datasets. The largest and most commonly used dataset of individual human trajectories captures fewer than 200 individuals, while datasets of individual human check-ins capture fewer than 100 check-ins per city per day. Thus, it is not clear if findings from the human mobility data science community would generalize to large populations. Since obtaining massive, representative, and individual-level human mobility data is hard to come by due to privacy considerations, the vision of this work is to embrace the use of data generated by large-scale socially realistic microsimulations. Informed by both real data and leveraging social and behavioral theories, massive spatially explicit microsimulations may allow us to simulate entire megacities at the person level. The simulated worlds, which do not capture any identifiable personal information, allow us to perform “in silico” experiments using the simulated world as a sandbox in which we have perfect information and perfect control without jeopardizing the privacy of any actual individual. In silico experiments have become commonplace in other scientific domains such as chemistry and biology, permitting experiments that foster the understanding of concepts without any harm to individuals. This work describes challenges and opportunities for leveraging massive and realistic simulated alternate worlds for in silico human mobility data science.

97 MATHEMATICS AND COMPUTING↗

Convolutional Neural Networks Applied to Neutrino Events in a Liquid Argon Time Projection Chamber

We present several studies of convolutional neural networks applied to data coming from the MicroBooNE detector, a liquid argon time projection chamber (LArTPC). The algorithms studied include the classification of single particle images, the localization of single particle and neutrino interactions in an image, and the detection of a simulated neutrino event overlaid with cosmic ray backgrounds taken from real detector data. These studies demonstrate the potential of convolutional neural networks for particle identifi-cation or event detection on simulated neutrino interactions. We also address technical issues that arise when applying this technique to data from a large LArTPC at or near ground level.

Acciarri, R↗

Identifying Critical Infrastructure in Imagery Data Using Explainable Convolutional Neural Networks

To date, no method utilizing satellite imagery exists for detailing the locations and functions of critical infrastructure across the United States, making response to natural disasters and other events challenging due to complex infrastructural interdependencies. This paper presents a repeatable, transferable, and explainable method for critical infrastructure analysis and implementation of a robust model for critical infrastructure detection in satellite imagery. This model consists of a DenseNet-161 convolutional neural network, pretrained with the ImageNet database. The model was provided additional training with a custom dataset, containing nine infrastructure classes. The resultant analysis achieved an overall accuracy of 90%, with the highest accuracy for airports (97%), hydroelectric dams (96%), solar farms (94%), substations (91%), potable water tanks (93%), and hospitals (93%). Critical infrastructure types with relatively low accuracy are likely influenced by data commonality between similar infrastructure components for petroleum terminals (86%), water treatment plants (78%), and natural gas generation (78%). Local interpretable model-agnostic explanations (LIME) was integrated into the overall modeling pipeline to establish trust for users in critical infrastructure applications. The results demonstrate the effectiveness of a convolutional neural network approach for critical infrastructure identification, with higher than 90% accuracy in identifying six of the critical infrastructure facility types.

97 MATHEMATICS AND COMPUTING↗

LSTM-Based Data Integration to Improve Snow Water Equivalent Prediction and Diagnose Error Sources

Accurate prediction of snow water equivalent (SWE) can be valuable for water resource managers. Recently, deep learning methods such as long short-term memory (LSTM) have exhibited high accuracy in simulating hydrologic variables and can integrate lagged observations to improve prediction, but their benefits were not clear for SWE simulations. Here we tested an LSTM network with data integration (DI) for SWE in the western United States to integrate 30-day-lagged or 7-day-lagged observations of either SWE or satellite-observed snow cover fraction (SCF) to improve future predictions. SCF proved beneficial only for shallow-snow sites during snowmelt, while lagged SWE integration significantly improved prediction accuracy for both shallow- and deep-snow sites. The median Nash–Sutcliffe model efficiency coefficient (NSE) in temporal testing improved from 0.92 to 0.97 with 30-day-lagged SWE integration, and root-mean-square error (RMSE) and the difference between estimated and observed peak SWE values d max were reduced by 41% and 57%, respectively. DI effectively mitigated accumulated model and forcing errors that would otherwise be persistent. Moreover, by applying DI to different observations (30-day-lagged, 7-day-lagged), we revealed the spatial distribution of errors with different persistent lengths. For example, integrating 30-day-lagged SWE was ineffective for ephemeral snow sites in the southwestern United States, but significantly reduced monthly-scale biases for regions with stable seasonal snowpack such as high-elevation sites in California. These biases are likely attributable to large interannual variability in snowfall or site-specific snow redistribution patterns that can accumulate to impactful levels over time for nonephemeral sites. These results set up benchmark levels and provide guidance for future model improvement strategies.

54 ENVIRONMENTAL SCIENCES↗

ConnectX-2 InfiniBand Management Queues: First Investigation of the New Support for Network Offloaded Collective Operations

This paper introduces the newly developed Infini-Band (IB) Management Queue capability, used by the Host Channel Adapter (HCA) to manage network task data flow dependancies, and progress the communications associated with such flows. These tasks include sends, receives, and the newly supported wait task, and are scheduled by the HCA based on a data dependency description provided by the user. This functionality is supported by the ConnectX-2 HCA, and provides the means for delegating collective communication management and progress to the HCA, also known as collective communication offload. This provides a means for overlapping collective communications managed by the HCA and computation on the Central Processing Unit (CPU), thus making it possible to reduce the impact of system noise on parallel applications using collective operations. This paper further describes how this new capability can be used to implement scalable Message Passing Interface (MPI) collective operations, describing the high level details of how this new capability is used to implement the MPI Barrier collective operation, focusing on the latency sensitive performance aspects of this new capability. This paper concludes with small scale benchmark experiments comparing implementations of the barrier collective operation, using the new network offload capabilities, with established point-to-point based implementations of these same algorithms, which manage the data flow using the central processing unit. These early results demonstrate the promise this new capability provides to improve the scalability of high performance applications using collective communications. The latency of the HCA based implementation of the barrier is similar to that of the best performing point-to-point based implementation managed by the central processing unit, starting to outperform these as the number of processes involved in the collective operation increases.

Graham, Richard L↗

ConnectX2 In niBand Management Queues: New support for Network Of oaded

This paper introduces the newly developed InfiniBand (IB) Management Queue capability, used by the Host Channel Adapter (HCA) to manage network task data flow dependancies, and progress the communications associated with such flows. These tasks include sends, receives, and the newly supported wait task, and are scheduled by the HCA based on a data dependency description provided by the user. This functionality is supported by the ConnectX-2 HCA, and provides the means for delegating collective communication management and progress to the HCA, also known as collective communication offload. This provides a means for overlapping collective communications managed by the HCA and computation on the Central Processing Unit (CPU), thus making it possible to reduce the impact of system noise on parallel applications using collective operations. This paper further describes how this new capability can be used to implement scalable Message Passing Interface (MPI) collective operations, describing the high level details of how this new capability is used to implement the MPI Barrier collective operation, focusing on the latency sensitive performance aspects of this new capability. This paper concludes with small scale benchmark experiments comparing implementations of the barrier collective operation, using the new network offload capabilities, with established point-to-point based implementations of these same algorithms, which manage the data flow using the central processing unit. These early results demonstrate the promise this new capability provides to improve the scalability of high-performance applications using collective communications. The latency of the HCA based implementation of the barrier is similar to that of the best performing point-to-point based implementation managed by the central processing unit, starting to outperform these as the number of processes involved in the collective operation increases.

99 GENERAL AND MISCELLANEOUS↗

Prior knowledge driven Granger causality analysis on gene regulatory network discovery

Our study focuses on discovering gene regulatory networks from time series gene expression data using the Granger causality (GC) model. However, the number of available time points (T) usually is much smaller than the number of target genes (n) in biological datasets. The widely applied pairwise GC model (PGC) and other regularization strategies can lead to a significant number of false identifications when n>>T. In this study, we proposed a new method, viz., CGC-2SPR (CGC using two-step prior Ridge regularization) to resolve the problem by incorporating prior biological knowledge about a target gene data set. In our simulation experiments, the propose new methodology CGC-2SPR showed significant performance improvement in terms of accuracy over other widely used GC modeling (PGC, Ridge and Lasso) and MI-based (MRNET and ARACNE) methods. In addition, we applied CGC-2SPR to a real biological dataset, i.e., the yeast metabolic cycle, and discovered more true positive edges with CGC-2SPR than with the other existing methods. In our research, we noticed a “ 1+1>2” effect when we combined prior knowledge and gene expression data to discover regulatory networks. Based on causality networks, we made a functional prediction that the Abm1 gene (its functions previously were unknown) might be related to the yeast’s responses to different levels of glucose. In conclusion, our research improves causality modeling by combining heterogeneous knowledge, which is well aligned with the future direction in system biology. Furthermore, we proposed a method of Monte Carlo significance estimation (MCSE) to calculate the edge significances which provide statistical meanings to the discovered causality networks. All of our data and source codes will be available under the link https://bitbucket.org/dtyu/granger-causality/wiki/Home.

60 APPLIED LIFE SCIENCES↗

Message communications of particular message types between compute nodes using DMA shadow buffers

Message communications of particular message types between compute nodes using DMA shadow buffers includes: receiving a buffer identifier specifying an application buffer having a message of a particular type for transmission to a target compute node through a network; selecting one of a plurality of shadow buffers for a DMA engine on the compute node for storing the message, each shadow buffer corresponding to a slot of an injection FIFO buffer maintained by the DMA engine; storing the message in the selected shadow buffer; creating a data descriptor for the message stored in the selected shadow buffer; injecting the data descriptor into the slot of the injection FIFO buffer corresponding to the selected shadow buffer; selecting the data descriptor from the injection FIFO buffer; and transmitting the message specified by the selected data descriptor through the data communications network to the target compute node.

Blocksome, Michael A↗

The NASA Soil Moisture Active Passive (SMAP) Mission - Algorithm and Cal/Val Activities and Synergies with SMOS and Other L-Band Missions

NASA's Soil Moisture Active Passive (SMAP) mission, planned for launch in late 2014, has as its key measurement objective the frequent, global mapping of near-surface soil moisture and its freeze-thaw state. SMAP soil moisture and freeze/thaw measurements at 10 km and 3 km resolutions respectively, would enable significantly improved estimates of water, energy and carbon transfers between the land and atmosphere. Soil moisture control of these fluxes is a key factor in the performance of atmospheric models used for weather forecasts and climate projections Soil moisture measurements are also of great importance in assessing floods and for monitoring drought. In addition, observations of soil moisture and freeze/thaw timing over the boreal latitudes can help reduce uncertainties in quantifying the global carbon balance. The SMAP measurement concept utilizes an L-band radar and radiometer sharing a rotating 6-meter mesh reflector antenna. The SMAP radiometer and radar flight hardware and ground processing designs are incorporating approaches to identify and mitigate potential terrestrial radio frequency interference (RFI). The radar and radiometer instruments are planned to operate in a 680 km polar orbit, viewing the surface at a constant 40-degree incidence angle with a 1000-km swath width, providing 3-day global coverage. Data from the instruments would yield global maps of soil moisture and freeze/thaw state to be provided at 10 km and 3 km resolutions respectively, every two to three days. Plans are to provide also a radiometer-only soil moisture product at 40-km spatial resolution. This product and the underlying brightness temperatures have characteristics similar to those provided by the Soil Moisture and Ocean Salinity (SMOS) mission. As a result, there are unique opportunities for common data product development and continuity between the two missions. SMAP also has commonalities with other satellite missions having L-band radiometer and/or radar sensors applicable to soil moisture measurement, such as Aquarius, SAO COM, and ALOS-2. The algorithms and data products for SMAP are being developed in the SMAP Science Data System (SDS) Testbed. The algorithms are developed and evaluated in the SDS Testbed using simulated SMAP observations as well as observational data from current airborne and spaceborne L-band sensors including SMOS. The SMAP project is developing a Calibration and Validation (Cal/Val) Plan that is designed to support algorithm development (pre-launch) and data product validation (post-launch). A key component of the Cal/Val Plan is the identification, characterization, and instrumentation of sites that can be used to calibrate and validate the sensor data (Level I) and derived geophysical products (Level 2 and higher). In this presentation we report on the development status of the SMAP data product algorithms, and the planning and implementation of the SMAP Cal/Val program. Several components of the SMAP algorithm development and Cal/Val plans have commonality with those of SMOS, and for this reason there are shared activities and resources that can be utilized between the missions, including in situ networks, ancillary data sets, and long-term monitoring sites.

Njoku, Eni↗

An operational all-weather Great Lakes ice information system

A description is given of the NASA developed all-weather ice information system for the Great Lakes winter navigation program. The system utilizes an X-band side looking airborne radar (SLAR) for determining type, location, and areal distribution of the ice cover in the Great Lakes and an airborne, S band, down looking short pulse radar for obtaining ice thickness. Digitized SLAR data are relayed in real time via the NOAA-GOES satellite in geosynchronous orbit. The SLAR images along with hand drawn interpretative ice charts for various Great Lakes winter shipping areas are broadcast to facsimile recorders aboard vessles is the area via the MARAD marine VHF-FM radio network. These data assist such vessels in navigating both through and around the ice.

Gedney, R. T.↗

Task 28: Web Accessible APIs in the Cloud Trade Study

This study explored three candidate architectures for serving NASA Earth Science Hierarchical Data Format Version 5 (HDF5) data via Hyrax running on Amazon Web Services (AWS). We studied the cost and performance for each architecture using several representative Use-Cases. The objectives of the project are: Conduct a trade study to identify one or more high performance integrated solutions for storing and retrieving NASA HDF5 and Network Common Data Format Version 4 (netCDF4) data in a cloud (web object store) environment. The target environment is Amazon Web Services (AWS) Simple Storage Service (S3).Conduct needed level of software development to properly evaluate solutions in the trade study and to obtain required benchmarking metrics for input into government decision of potential follow-on prototyping. Develop a cloud cost model for the preferred data storage solution (or solutions) that accounts for different granulation and aggregation schemes as well as cost and performance trades.

cost model↗

Antarctic Meteorite Location and Mapping Project (AMLAMP): Antarctic meteorite location map series explanatory text and user's guide to AMLAMP data

This technical report is an update to LPI Technical Report 89-02, which contained data and information that was current to May 1987. Since that time approximately 4000 new meteorites have been collected, mapped, and characterized, mainly from the numerous ice fields in the Allan Hills-David Glacier region, from the Pecora Escarpment and Moulton Escarpment in the Thiel Mountains-Patuxent region, the Wisconsin Range region, and from the Beardmore region. Meteorite location maps for ice fields from these regions have been produced and are available. This report includes explanatory texts for the maps of new areas and provides information on updates of maps of the areas covered in LPI Technical Report 89-02. Sketch maps and description of locales that have been searched and have yielded single or few meteorites are also included. The meteorite listings for all the ice fields have been updated to include any classification changes and new meteorites recovered from ice fields in the Allan Hills-David Glacier region since 1987. The text has been reorganized and minor errors in the original report have been corrected. Computing capabilities have improved immensely since the early days of this project. Current software and hardware allow easy access to data over computer networks. With various commercial software packages, the data can be used many different ways, including database creation, statistics, and mapping. The databases, explanatory texts, and the plotter files used to produce the meteorite location maps are available through a computer network. Information on how to access AMLAMP data, its formats, and ways it can be used are given in the User's Guide to AMLAMP Data section. Meteorite location maps and thematic maps may be ordered from the Lunar and Planetary Institute. Ordering information is given in Appendix A.

Schutt, J.↗

On the Impact of Granularity of Space-Based Urban CO2 Emissions in Urban Atmospheric Inversions: A Case Study for Indianapolis, IN

Quantifying greenhouse gas (GHG) emissions from cities is a key challenge towards effective emissions management. An inversion analysis from the INdianapolis FLUX experiment (INFLUX) project, as the first of its kind, has achieved a top-down emission estimate for a single city using CO2 data collected by the dense tower network deployed across the city. However, city-level emission data, used as a priori emissions, are also a key component in the atmospheric inversion framework. Currently, fine-grained emission inventories (EIs) able to resolve GHG city emissions at high spatial resolution, are only available for few major cities across the globe. Following the INFLUX inversion case with a global 1x1 km ODIAC fossil fuel CO2 emission dataset, we further improved the ODIAC emission field and examined its utility as a prior for the city scale inversion. We disaggregated the 1x1 km ODIAC non-point source emissions using geospatial datasets such as the global road network data and satellite-data driven surface imperviousness data to a 3030 m resolution. We assessed the impact of the improved emission field on the inversion result, relative to priors in previous studies (Hestia and ODIAC). The posterior total emission estimate (5.1 MtC/yr) remains statistically similar to the previous estimate with ODIAC (5.3 MtC/yr). However, the distribution of the flux corrections was very close to those of Hestia inversion and the model-observation mismatches were significantly reduced both in forward and inverse runs, even without hourly temporal changes in emissions. EIs reported by cities often do not have estimates of spatial extents. Thus, emission disaggregation is a required step when verifying those reported emissions using atmospheric models. Our approach offers gridded emission estimates for global cities that could serves as a prior for inversion, even without locally reported EIs in a systematic way to support city-level Measuring, Reporting and Verification (MRV) practice implementation.

EIs↗

An Adaptive Middleware Framework for Scientific Computing at Extreme Scales

Large computing systems including clusters, clouds, and grids, provide high-performance capabilities that can be utilized for many applications. But as the ubiquity of these systems increases and the scope of analysis being done on them grows, there is a growing need for applications that 1) do not require users to learn the details of high performance systems, and 2) are flexible and adaptive in their usage of these systems to accommodate the best time-to-solution for end users. We introduce a new adaptive interface design and a prototype implementation within the framework of an established middleware framework, MeDICi, for high performance computing systems and describe the applicability of this adaptive design to a real-life scientific workflow. This adaptive framework provides an access model for implementing a processing pipeline using high performance systems that are not local to the data source, making it possible for the compute capabilities at one site to be applied to analysis on data being generated at another site in an automated process. This adaptive design improves overall time-to-solution by moving the data analysis task to the most appropriate resource dynamically, reacting to failures and load fluctuations.

97 MATHEMATICS AND COMPUTING↗

Reconstruction of the regulatory network for Bacillus subtilis and reconciliation with gene expression data

Here, we introduce a manually constructed and curated regulatory network model that describes the current state of knowledge of transcriptional regulation of B. subtilis . The model corresponds to an updated and enlarged version of the regulatory model of central metabolism originally proposed in 2008. We extended the original network to the whole genome by integration of information from DBTBS, a compendium of regulatory data that includes promoters, transcription factors (TFs), binding sites, motifs and regulated operons. Additionally, we consolidated our network with all the information on regulation included in the SporeWeb and Subtiwiki community-curated resources on B. subtilis . Finally, we reconciled our network with data from RegPrecise, which recently released their own less comprehensive reconstruction of the regulatory network for B. subtilis . Our model describes 275 regulators and their target genes, representing 30 different mechanisms of regulation such as TFs, RNA switches, Riboswitches and small regulatory RNAs. Overall, regulatory information is included in the model for approximately 2500 of the ~4200 genes in B. subtilis 168. In an effort to further expand our knowledge of B. subtilis regulation, we reconciled our model with expression data. For this process, we reconstructed the Atomic Regulons (ARs) for B. subtilis , which are the sets of genes that share the same “ON” and “OFF” gene expression profiles across multiple samples of experimental data. We show how atomic regulons for B. subtilis are able to capture many sets of genes corresponding to regulated operons in our manually curated network. Additionally, we demonstrate how atomic regulons can be used to help expand or validate the knowledge of the regulatory networks by looking at highly correlated genes in the ARs for which regulatory information is lacking. During this process, we were also able to infer novel stimuli for hypothetical genes by exploring the genome expression metadata relating to experimental conditions, gaining insights into novel biology.

59 BASIC BIOLOGICAL SCIENCES↗

Photometric Properties of Network and Faculae Derived from HMI Data Compensated for Scattered Light

We report on the photometric properties of faculae and network, as observed in full-disk, scattered-light-corrected images from the Helioseismic Magnetic Imager. We use a Lucy–Richardson deconvolution routine that corrects an image in less than one second. Faculae are distinguished from network through proximity to active regions. This is the first report that full-disk observations, including center-to-limb variations, reproduce the photometric properties of faculae and network observed previously only in sub-arcsecond-resolution; small field-of-view studies, i.e. that network, as defined by distance from active regions, exhibit higher photometric contrasts. Specifically, for magnetic flux values larger than approximately 300 G, the network is brighter than faculae and the contrast differences increase toward the limb, where the network contrast is about twice the facular one. For lower magnetic flux values, network appear darker than faculae. Contrary to reports from previous full-disk observations, we also found that network exhibits a higher center-to-limb variation. Our results are in agreement with reports from simulations that indicate magnetic flux alone is a poor proxy of the photometric properties of magnetic features. We estimate that the contribution of faculae and network to Total Solar Irradiance variability of the current Cycle 24 is overestimated by at least 11%, due to the photometric properties of network and faculae not being recognized as different. This estimate is specific to the method employed in this study to reconstruct irradiance variations, so caution should be paid when extending it to other techniques.

79 ASTRONOMY AND ASTROPHYSICS↗