SEARCH · Engineering Papers

Results for “Cloud computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

NASA NTRS2019

Analysis Ready Data in Analytics Optimized Data Stores for Analysis of Big Earth Data in the Cloud

Cloud computing offers the possibility of making the analysis of Big Data approachable for a wider community due to affordable access to computing power, an ecosystem of usable tools for parallel processing, and migration of many large datasets to archives in the cloud, allowing data-proximal computing. Generally, data analysis acceleration in the cloud comes from running multiple nodes in a split-combine-apply strategy. Data systems such as the Earth Observing System Data and Information System are in a position to "pre-split" the data by storing them in a data store that is optimized for data parallel computing, i.e., an Analytics-Optimized Data Store (AODS). A variety of approaches to AODS are possible, from highly scalable databases to scalable filesystems to data formats optimized for cloud access (e.g., zarr and cloud-optimized datasets), with the optimal choice dependent on both the types of analysis and the geospatial structure of the data. A key question is how much preprocessing of the data to do, both before splitting and as the first part of the apply step. Again, the geospatial structure of the data and the analysis type influence the decision, with the added complexity of the user type. Trans-disciplinary users who are not well-versed in the nuances of quality-filtering and georeferencing of remote sensing orbit/swath/scene data tend to ask for more highly processed data, relying on the data provider to make sensible decisions on preprocessing parameters. (This accounts for the popularity of "Level 3" gridded data, despite the lower spatial resolution it provides.) In this case, data can be preprocessed before the split, resulting in higher performance in the rest of the "apply" step, which can be transformative for use cases such as interactive data exploration at scale. Discipline researchers who are experienced with remote sensing data often prefer more flexibility in customizing the preprocessing data into Analysis Ready Data, resulting in more need for on-the-fly preprocessing.

Lynnes, Christopher↗

DOE OSTI2023

Measurement and applications: Exploring the challenges and opportunities of hierarchical federated learning in sensor applications

Sensor applications have become ubiquitous in modern society as the digital age continues to advance. AI-based techniques (e.g., machine learning) are effective at extracting actionable information from large amounts of data. An example would be an automated water irrigation system that uses AI-based techniques on soil quality data to decide how to best distribute water. However, these AI-based techniques are costly in terms of hardware resources, and Internet-of-Things (IoT) sensors are resource-constrained with respect to processing power, energy, and storage capacity. These limitations can compromise the security, performance, and reliability of sensor-driven applications. To address these concerns, cloud computing services can be used by sensor applications for data storage and processing. Unfortunately, cloud-based sensor applications that require real-time processing, such as medical applications (e.g., fall detection and stroke prediction), are vulnerable to issues such as network latency due to the sparse and unreliable networks between the sensor nodes and the cloud server [1]. As users approach the edge of the communications network, latency issues become more severe and frequent. A promising alternative is edge computing, which provides cloud-like capabilities at the edge of the network by pushing storage and processing capabilities from centralized nodes to edge devices that are closer to where the data are gathered, resulting in reduced network delays [2], [3].

Po-Leen Ooi, Melanie↗

DOE OSTI2024

The ATLAS Workflow Management System Evolution in the LHC Run3 and towards the High-Luminosity LHC era

The ATLAS experiment has 18+ years of experience using workload management systems to deploy and develop workflows to process and to simulate data on the distributed computing infrastructure. Simulation, processing and analysis of LHC experiment data require the coordinated work of heterogeneous computing resources. In particular, the ATLAS experiment utilizes the resources of 250 computing centers worldwide, the power of supercomputing centres, and national, academic and commercial cloud computing resources. In this contribution, we present new techniques for cost-effectively improving efficiency introduced in workflow management system software. The evolution from a mesh framework to new types of computing facilities such as cloud and HPCs is described, as well as new types of production and analysis workflows.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS2017

NASA Wrangler: Automated Cloud-Based Data Assembly in the RECOVER Wildfire Decision Support System

NASA Wrangler is a loosely-coupled, event driven, highly parallel data aggregation service designed to take advantageof the elastic resource capabilities of cloud computing. Wrangler automatically collects Earth observational data, climate model outputs, derived remote sensing data products, and historic biophysical data for pre-, active-, and post-wildfire decision making. It is a core service of the RECOVER decision support system, which is providing rapid-response GIS analytic capabilities to state and local government agencies. Wrangler reduces to minutes the time needed to assemble and deliver crucial wildfire-related data.

decision support↗

NASA NTRS2014

RECOVER: An Automated Cloud-Based Decision Support System for Post-fire Rehabilitation Planning

RECOVER is a site-specific decision support system that automatically brings together in a single analysis environment the information necessary for post-fire rehabilitation decision-making. After a major wildfire, law requires that the federal land management agencies certify a comprehensive plan for public safety, burned area stabilization, resource protection, and site recovery. These burned area emergency response (BAER) plans are a crucial part of our national response to wildfire disasters and depend heavily on data acquired from a variety of sources. Final plans are due within 21 days of control of a major wildfire and become the guiding document for managing the activities and budgets for all subsequent remediation efforts. There are few instances in the federal government where plans of such wide-ranging scope and importance are assembled on such short notice and translated into action more quickly. RECOVER has been designed in close collaboration with our agency partners and directly addresses their high-priority decision-making requirements. In response to a fire detection event, RECOVER uses the rapid resource allocation capabilities of cloud computing to automatically collect Earth observational data, derived decision products, and historic biophysical data so that when the fire is contained, BAER teams will have a complete and ready-to-use RECOVER dataset and GIS analysis environment customized for the target wildfire. Initial studies suggest that RECOVER can transform this information-intensive process by reducing from days to a matter of minutes the time required to assemble and deliver crucial wildfire-related data.

cloud computing↗

DOE OSTI2024

Cybersecurity Considerations for Emerging Energy Technologies

AI, cloud computing, post-quantum cryptography, zero-trust architectures, microgrids, and virtual power plants. What do these things have in common? They are all emerging technologies in the clean energy space that will be a critical part of grid modernization efforts. As we work towards clean energy and decarbonization targets, these technologies, developed to solve real-world problems, will help us reach goals and achieve new efficiencies as the paradigm of grid operation shifts. However, there are growing concerns about the cybersecurity risks associated with these trending topics as they are used in critical infrastructure applications. This talk will cover gaps, challenges, and opportunities for the secure implementation of grid modernization solutions and novel energy applications of state-of-the-art networking and communications. Proactive risk mitigation strategies, including the application of cyber-informed engineering, will be discussed. Practical applications of these techniques will help provide countermeasures to the impact of cyberattacks on critical infrastructure technologies in a new, digitized grid landscape.

14 SOLAR ENERGY↗

NASA NTRS1990

ER-2 lidar measurements of stratocumulus cloud top structure on July 14, 1987

On July 14, 1987 NASA's ER-2 high altitude aircraft flew a mission to measure the structure of stratocumulus clouds off the coast of California. A flight pattern was executed so that the two-dimensional variability of the clouds could be detected. The technique of analysis of the lidar data to measure cloud tops follows. First each signal is searched for its maximum in return strength. This maximum is caused by scattering of the laser light off cloud particles or from the ocean surface. Next the variance of the signal return above the level of maximum backscatter is determined. Cloud top is assigned to a level (above the level of maximum backscatter) where the backscatter exceeds the average variance. This two-step process is necessary because the level of maximum backscatter does not correspond to the cloud top. Ocean surface returns are easily separated from cloud returns in this process, described in detail by Boers, Spinhirne, and Hart (1988). Analysis of the data so far has shown that there were very few breaks in the clouds. Furthermore the layer top was very flat with local oscillations not exceeding 30 m. Such small cloud top variations are still well within the range of detectability, because the precision of this technique of cloud top detection has previously been established to be 13 to 15 m. Data are presently being analyzed to compute cloud top distributions and fractional cloudiness. The aim of this research is to relate the fractional cloudiness to the mean thermodynamic structure of the boundary layer. Researchers plan to compute spectral scales of the cloud top variability in two dimensions to determine the orientation of the clouds with respect to the mean wind. Furthermore the lidar derived cloud top distribution will be used in the computation of the thermodynamic and radiation budget of the boundary layer.

Boers, Reinout↗

DOE OSTI2020

Softwarized Federations of Science Instruments with Edge-Continuum Containers

Significant expansion of capabilities of DOE science complex of supercomputers, instruments and networks is expected as powerful experimental facilities, exascale computers and terabit networks are added. Combined with the advances in edge and cloud computing technologies, DOE science users now have the promise of unprecedented execution of complex, continuum workflows, namely, small and latency-sensitive computations at the edge and site, massive computations at remote HPC systems, and everything in between on the cloud. But, bringing this capability to the science user requires overcoming the overwhelming complexity of forming the federations of systems, and efficiently and effectively orchestrating the workflows while ensuring high utilization of the expensive facilities. Current manual configuration of the federated systems simply will not scale, since the coordination across sites may take weeks to months, often leading to under-utilized and hard-to-diagnose compositions. A powerful, composable software stack will be developed to (i) wrap the systems so that federations can be composed fast in software, and (ii) containerize computations to be orchestrated across the edge, site, cloud and HPC resources.

Rao, Nageswara S.↗

DOE OSTI2022

Bringing chemical structures to life with augmented reality, machine learning, and quantum chemistry

Visualizing 3D molecular structures is crucial to understanding and predicting their chemical behavior. However, static 2D hand-drawn skeletal structures remain the preferred method of chemical communication. Here, we combine cutting-edge technologies in augmented reality (AR), machine learning, and computational chemistry to develop MolAR, an open-source mobile application for visualizing molecules in AR directly from their hand-drawn chemical structures. Users can also visualize any molecule or protein directly from its name or protein data bank ID and compute chemical properties in real time via quantum chemistry cloud computing. MolAR provides an easily accessible platform for the scientific community to visualize and interact with 3D molecular structures in an immersive and engaging way.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Feedback-based quantum algorithm inspired by counterdiabatic driving

In recent quantum algorithmic developments, a feedback-based approach has shown promise for preparing quantum many-body system ground states and solving combinatorial optimization problems. This method utilizes quantum Lyapunov control to iteratively construct quantum circuits. Here, we propose a substantial enhancement by implementing a protocol that uses ideas from quantum Lyapunov control and the counterdiabatic driving protocol, a key concept from quantum adiabaticity. Our approach introduces an additional control field inspired by counterdiabatic driving. We apply our algorithm to prepare ground states in one-dimensional quantum Ising spin chains. Comprehensive simulations demonstrate a remarkable acceleration in population transfer to low-energy states within a significantly reduced time frame compared to conventional feedback-based quantum algorithms. This acceleration translates to a reduced quantum circuit depth, a critical metric for potential quantum computer implementation. We validate our algorithm on the IBM cloud computer, highlighting its efficacy in expediting quantum computations for many-body systems and combinatorial optimization problems.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

NASA NTRS2021

Applying the Cognitive Space Gateway to Swarm Topologies

NASA's future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize routing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, CubeSat swarm topologies, and cloud services. To test the algorithm in a realistic scenario, the emulated network topology is based on a CubeSat swarm. The swarm may function as a mesh of nodes or as a hub-and-spoke network. An emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. The cloud environment may enable a flexible, lower maintenance approach versus a multi-hop network based in a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent↗

NASA NTRSDate not supplied

Applying the Cognitive Space Gateway to Swarm Topologies

NASA’s future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize outing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, SmallSat swarm topologies, and cloud services. The CSG algorithm is tested in a realistic scenario in which the emulated network topology is based on a SmallSat swarm. The emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. This work investigates the ability of such a platform to enable a flexible, lower maintenance approach to creating a multihop network outside of a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent↗

NASA NTRS2011

Detecting Abnormal Machine Characteristics in Cloud Infrastructures

In the cloud computing environment resources are accessed as services rather than as a product. Monitoring this system for performance is crucial because of typical pay-peruse packages bought by the users for their jobs. With the huge number of machines currently in the cloud system, it is often extremely difficult for system administrators to keep track of all machines using distributed monitoring programs such as Ganglia1 which lacks system health assessment and summarization capabilities. To overcome this problem, we propose a technique for automated anomaly detection using machine performance data in the cloud. Our algorithm is entirely distributed and runs locally on each computing machine on the cloud in order to rank the machines in order of their anomalous behavior for given jobs. There is no need to centralize any of the performance data for the analysis and at the end of the analysis, our algorithm generates error reports, thereby allowing the system administrators to take corrective actions. Experiments performed on real data sets collected for different jobs validate the fact that our algorithm has a low overhead for tracking anomalous machines in a cloud infrastructure.

Bhaduri, Kanishka↗

NASA NTRS2019

A Web of Data Analytics Services

Cloud Computing has become the ubiquitous approach to our Big Data challenge. However, one will quickly discover that moving (a.k.a. forklifting) existing on-premise data analytics solutions to the Cloud doesn’t always translate to costing saving and performance boost. The Cloud’s elasticity, its availability, and its wide selection of computing options and selections of costing models making Cloud an attractive environment to tackle our Big Data challenge. The fact is Cloud, on its own, is not the silver bullet to our daunting challenge need for analyze and derive scientific inferences through vast collections of multi-sensor measurements. We would like to have all scientific data in one easy to access environment, but getting the world of scientific data in one analytic system is immensely difficult to achieve. This paper describes the data analytics web architecture NASA is developing by infusing instances of Integrated Data Analytics systems next to the data. The goal is to minimize unnecessary data movement through collection of data access and analytics webservices for researchers to interact with and analyze measurements without have to download data to their local computer. These services are RESTful and provisioned by the data centers with the help from subject matter and science experts. These services encapsulate the physical computing infrastructure, which could local computing cluster, on-premise or public Cloud environment.

Huang, Thomas↗

NASA NTRS2014

Climate Analytics as a Service

Climate science is a big data domain that is experiencing unprecedented growth. In our efforts to address the big data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS). CAaaS combines high-performance computing and data-proximal analytics with scalable data management, cloud computing virtualization, the notion of adaptive analytics, and a domain-harmonized API to improve the accessibility and usability of large collections of climate data. MERRA Analytic Services (MERRA/AS) provides an example of CAaaS. MERRA/AS enables MapReduce analytics over NASA's Modern-Era Retrospective Analysis for Research and Applications (MERRA) data collection. The MERRA reanalysis integrates observational data with numerical models to produce a global temporally and spatially consistent synthesis of key climate variables. The effectiveness of MERRA/AS has been demonstrated in several applications. In our experience, CAaaS is providing the agility required to meet our customers' increasing and changing data management and data analysis needs.

big data↗

NASA NTRS2023

The 50-Year Landsat Collection 2 Archive

The Landsat global consolidated data archive now exceeds 50 years. In recognition of the need for consistently processed data across the Landsat satellite series, the U.S. Geological Survey (USGS) initiated collection-based processing of the entire archive that was processed as Collection 1 in 2016. In preparation for the data from the now successfully launched Landsat 9, the USGS reprocessed the Landsat archive as Collection 2 in 2020. This paper describes the rationale for, and the contents and advancements provided by Collection 2, and highlights the differences between the Collection 1 and Collection 2 products. Notably, the Collection 2 products have improved geolocation and, for the first time, the USGS provides a global inventory of Level 2 surface reflectance and surface temperature products. Also for the first time, the USGS used a commercial cloud computing architecture to efficiently process the archive and enable direct cloud access of the Landsat products. The paper concludes with discussion of likely improvements expected in Collection 3 in preparation for the Landsat Next mission that is planned for launch in the early 2030s.

Christopher J Crawford↗

NASA NTRS2019

"Sensor Web Evolution - Webs of Webs for NASA Science - Focus on small Uninhabited Aerial Systems (sUAS)"

This paper will describe the evolution of information collection, derivation and delivery mechanisms in webs of NASA sensor webs, with a focus on recent advancements in small Uninhabited Aerial Systems (sUAS). I will discuss the movement to "Fog Computing", also known as Edge Computing. Fog Computing facilitates the distribution of common operations and networking between edge devices and cloud computing facilities, optimizing the production of actionable intelligence. Initially, sUASs utilized onboard data collection as standard, with minimal data downloaded directly. Information products were derived in conventional computational environments, generally desk top computers, and information products made available to the Science Community in weeks or months. With the increased availability, and increasingly lower costs, of beyond line of sight (BLOS) satellite based communication, transmission rates and data volumes increased, and processing migrated to Cloud based services. Contemporary sUASs are moving some of that information product derivation to on vehicle services, and are creating a distributed Cloud/Fog environment. I will describe the technological advances that have made this possible, including low power multi-core Central Processing Units (CPU), and, more recently, the availability of high end Graphical Processing Units (GPU) that consume only a few watts. Intelligent system software, leveraging these hardware advances, finally allows for information product generation on-board, rather than simple data collection. Additionally, intelligent flight control systems now support mutual vehicle to vehicle collaboration, allowing sUASs to create ad-hoc sensor webs on demand, as required. Also discussed will be the lessons learned by the Authors' development of data systems for NASA's large High Altitude Long Endurance (HALE) UASs like Predator and Global Hawk, and how those lessons are being applied to sUAS development. This paper will focus on application, rather a deep dive into the technology, and will highlight improving data management through these new technologies.

Sensor Web↗

NASA NTRSDate not supplied

UASs in the VOG/Edge/FOG Sensor Web Environment

This paper will describe the evolution of information collection, derivation and delivery mechanisms in sensor webs utilizing Uninhabited Aerial Systems (UAS).We will discuss the movement to "Fog Computing", also known as Edge Computing. Fog Computing facilitates the distribution of common operations and networking between edge devices and cloud computing facilities, optimizing the production of actionable intelligence. Initially, UASs utilized onboard data collection as standard, with minimal data downloaded directly. Information products were derived in conventional computational environments, generally desk top computers, and information products made available to the Science Community in weeks or months. With the increased availability, and increasingly lower costs, of beyond line of sight (BLOS) satellite based communication, transmission rates and data volumes increased, and processing migrated to Cloud based services. Contemporary UASs are moving some of that information product derivation to on vehicle services, and are creating a distributed Cloud/Fog environment. The Author will describe the technological advances that have made this possible, including low power multi-core Central Processing Units (CPU), and, more recently, the availability of high end Graphical Processing Units (GPU) that consume only a few watts. Intelligent system software, leveraging these hardware advances, finally allows for information product generation on-board, rather than simple data collection. Additionally, intelligent flight control systems now support mutual vehicle to vehicle collaboration, allowing UASs to create ad-hoc sensor webs on demand, as required. Also discussed will be the lessons learned by the Authors' development of data systems for NASA's large High Altitude Long Endurance (HALE) UASs like Predator and Global Hawk, and how those lessons are being applied to other UAS development This paper will focus on applications, rather a deep dive into the technology, and will highlight improving data management through these new technologies.

UAS↗