Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Cloud Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly. Reviewed by Mark McInerney ESDIS Deputy Project Manager.

Lynnes, Christopher↗
DOE OSTI2017

HEPCloud, a New Paradigm for HEP Facilities: CMS Amazon Web Services Investigation

Historically, high energy physics computing has been performed on large purpose-built computing systems. These began as single-site compute facilities, but have evolved into the distributed computing grids used today. Recently, there has been an exponential increase in the capacity and capability of commercial clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing interest among the cloud providers to demonstrate the capability to perform large-scale scientific computing. In this paper, we discuss results from the CMS experiment using the Fermilab HEPCloud facility, which utilized both local Fermilab resources and virtual machines in the Amazon Web Services Elastic Compute Cloud. We discuss the planning, technical challenges, and lessons learned involved in performing physics workflows on a large-scale set of virtualized resources. Additionally, we will discuss the economics and operational efficiencies when executing workflows both in the cloud and on dedicated resources.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Nanosystems, Edge Computing, and the Next Generation Computing Systems

It is widely recognized that nanoscience and nanotechnology and their subfields, such as nanophotonics, nanoelectronics, and nanomechanics, have had a tremendous impact on recent advances in sensing, imaging, and communication, with notable developments, including novel transistors and processor architectures. For example, in addition to being supremely fast, optical and photonic components and devices are capable of operating across multiple orders of magnitude length, power, and spectral scales, encompassing the range from macroscopic device sizes and kW energies to atomic domains and single-photon energies. The extreme versatility of the associated electromagnetic phenomena and applications, both classical and quantum, are therefore highly appealing to the rapidly evolving computing and communication realms, where innovations in both hardware and software are necessary to meet the growing speed and memory requirements. Development of all-optical components, photonic chips, interconnects, and processors will bring the speed of light, photon coherence properties, field confinement and enhancement, information-carrying capacity, and the broad spectrum of light into the high-performance computing, the internet of things, and industries related to cloud, fog, and recently edge computing. Conversely, owing to their extraordinary properties, 0D, 1D, and 2D materials are being explored as a physical basis for the next generation of logic components and processors. Carbon nanotubes, for example, have been recently used to create a new processor beyond proof of principle. These developments, in conjunction with neuromorphic and quantum computing, are envisioned to maintain the growth of computing power beyond the projected plateau for silicon technology. We survey the qualitative figures of merit of technologies of current interest for the next generation computing with an emphasis on edge computing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Women in Science: Fermilab Scientific Computing Specialist Krista Majewski

With February 11 marking the International Day of Women and Girls in Science, female physicists, engineers and computer scientists from CERN and Fermilab share their experiences of building careers in science. Krista Majewski came to Fermilab nine years ago as a software developer and now works in an operations role. She is a computing services specialist and part of the grid and cloud operations group, which manages computing clusters for the CMS experiment and other intensity frontier experiments at the lab. "My typical day would be maintaining the services that our group is responsible for and the machines that they run on. So if there are problems with a computing cluster where a user can't run their jobs, or they're trying to run on resources on the grid and they're unable to, then our group would be responsible for helping debug some of those issues," Krista says. Her path to programming at Fermilab has been winding. After receiving a degree in biomedical engineering, she went to work doing web development and programming for a large consulting firm, then moved to the financial industry. But programming called her back, and she earned a master's degree in computer science to start her new career. For Krista, programming is similar to something she enjoyed as child: solving puzzles. "I think it's a skill that you use in a lot of different ways," she says. Credits: Video production - Fermilab Creative Services Music Copyright: Universal Production Music Astral Light Chevalier Idea originally conceived by Tanya Levshina, Fermilab and Maria Dimou, CERN.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS1993

Inference of cirrus cloud properties using satellite-observed visible and infrared radiances. I - Parameterization of radiance fields

The impact of using phase functions for spherical droplets and hexagonal ice crystals to analyze radiances from cirrus is examined. Adding-doubling radiative transfer calculations are employed to compute radiances for different cloud thicknesses and heights over various backgrounds. These radiances are used to develop parameterizations of top-of-the-atmosphere visible reflectance and IR emittance using tables of reflectances as a function of cloud optical depth, viewing and illumination angles, and microphysics. This parameterization, which includes Rayleigh scattering, ozone absorption, variable cloud height, and an anisotropic surface reflectance, reproduces the computed top-of-the-atmosphere reflectances with an accruacy of +/- 6 percent for four microphysical models: 10-micron water droplet, small symmetric crystal, cirrostratus, and cirrus uncinus. The accuracy is twice that of previous models.

Minnis, Patrick↗
DOE OSTI2016

Stochastic Simulation Service: Bridging the Gap between the Computational Expert and the Biologist

We present StochSS: Stochastic Simulation as a Service, an integrated development environment for modeling and simulation of both deterministic and discrete stochastic biochemical systems in up to three dimensions. An easy to use graphical user interface enables researchers to quickly develop and simulate a biological model on a desktop or laptop, which can then be expanded to incorporate increasing levels of complexity. StochSS features state-of-the-art simulation engines. As the demand for computational power increases, StochSS can seamlessly scale computing resources in the cloud. In addition, StochSS can be deployed as a multi-user software environment where collaborators share computational resources and exchange models via a public model repository. We also demonstrate the capabilities and ease of use of StochSS with an example of model development and simulation at increasing levels of complexity.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

HEPCloud, an Elastic Hybrid HEP Facility using an Intelligent Decision Support System

HEPCloud is rapidly becoming the primary system for provisioning compute resources for all Fermilab-affiliated experiments. In order to reliably meet the peak demands of the next generation of High Energy Physics experiments, Fermilab must plan to elastically expand its computational capabilities to cover the forecasted need. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and at which scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources spanning multiple cloud providers, multiple HPC centers, and grid computing federations. In this paper, we discuss the goals and architecture of the HEPCloud Facility, the architecture of the IDSS, and our early experience in using the IDSS for automated facility expansion both at Fermi and Brookhaven National Laboratory.

Mhashilkar, Parag↗
NASA NTRS2002

Parameterization of Shortwave Cloud Optical Properties for a Mixture of Ice Particle Habits for use in Atmospheric Models

Based on the single-scattering optical properties pre-computed with an improved geometric optics method, the bulk absorption coefficient, single-scattering albedo, and asymmetry factor of ice particles have been parameterized as a function of the effective particle size of a mixture of ice habits, the ice water amount, and spectral band. The parameterization has been applied to computing fluxes for sample clouds with various particle size distributions and assumed mixtures of particle habits. It is found that flux calculations are not overly sensitive to the assumed particle habits if the definition of the effective particle size is consistent with the particle habits that the parameterization is based. Otherwise, the error in the flux calculations could reach a magnitude unacceptable for climate studies. Different from many previous studies, the parameterization requires only an effective particle size representing all ice habits in a cloud layer, but not the effective size of individual ice habits.

Chou, Ming-Dah↗
DOE OSTI2025

torc (Torc Workflow Management System) [SWR-24-127]

This software package orchestrates execution of a workflow of jobs on distributed computing resources. It is optimized for use on HPCs with Slurm, but also can be used in the cloud and on local computers. Please refer to the documentation at https://nrel.github.io/torc

Thom, Daniel [National Renewable Energy Laboratory↗
NASA NTRSDate not supplied

Integrating Multi-agency Data Products in a Cloud-based Platform for Streamlined Discovery, Visualization, and Use

Earth science data users almost always have an interest in utilizing geospatial data from multiple agencies. As computing capability and cloud-based infrastructures accelerate the pace at which scientific research can be done, there is a growing need to enable search, discovery, and use of multi-agency geospatial observations relevant for a common use case - without undergoing the search and discovery process in a less efficient, disparate path with each agency. NASA’s Earth Observing System Data and Information System (EOSDIS) and NOAA’s National Environmental Satellite, Data and Information Service (NESDIS) both support a wide range of Earth science disciplines’ research, operations, and applications activities. Presently, however, there are few examples of data discovery frameworks supporting an inquiry of both NASA’s and NOAA’s extensive archives of Earth observations that are equally suitable for a particular science scenario, regardless of the agency that “owns” the data. NASA and NOAA are collaborating on a data expedition platform for exploring fire weather using data products from both agencies. Users will be able to search, discover, and visualize NASA and NOAA products in one interface. Each agency will curate metadata for its respective datasets, providing for a rich search experience. The collaboration will pilot a shared search interface into these metadata datastores. Data products will be stored in the cloud in cloud-optimized format(s). These formats will allow for optimized data access and visualization to support the “data expedition”. Avenues for further development and application of this cloud-based, multi-agency data provisioning platform will also be discussed.

cloud-based technology↗
DOE OSTI2015

High Performance Data Analytics on a Commodity Private Cloud

Sandia is researching novel methods of analysis for large datasets. The term “Big Data” has been used frequently in recent years to describe a discipline of computer science that attempts to distil an increasing quantity of data into a result or meaningful subset that can be easily visualized and acted upon by human consumers. These data are frequently characterized by one or more of the following: high volume, velocity, variety, and veracity. Sandia scientists and researchers are no strangers to large datasets, which often typify the result of high-performance computing workloads. The challenge for Sandia data scientists, however, comes at the arrival of exascale computing, which has become mandated by executive order for national computing centers. Because of power and networking limitations, exascale computing precludes the movement of datasets at exascale from data production to data analysis. Such analysis must be done in-situ, to mitigate the time and power costs of network transport. Currently, many of Sandia’s data analytics environments exist as independent entities, often specialized for individual use cases, interconnected to other data analytics, high-performance computing, and storage systems. Sandia is procuring, installing, and researching future platforms for such data analytics. A memory footprint of 256 GB per node, fourteen data rate (FDR) InfiniBand networking, PCIe connected non-volatile memory (NVMe) data storage, and an object data store will characterize the coming institutional data analytics platforms. The combined systems will contain 300 compute nodes. A cloud-inspired software stack resulting from collaboration between the Sandia centers will allow the systems to support a flexible mix of user and system-facing workloads and use cases including code development and testing in R and Python, Apache Hadoop and Spark, distributed in-memory databases, graph analytics, emulytics, system log and predictive analytics, and data visualization for institutional and mission customers. These systems will arrive at the end of FY 2015 and production is planned by the spring of FY 2016.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

Jumping the Queue: From NASA to the Commercial Cloud

NASA's High-End Computing Capability (HECC) Project has made it possible for its users to run on commercial cloud resources in a seamless way. In the first of three phases, we implemented a pilot project for a few users, enabling them to “jump the queue” and burst jobs from the HECC environment to Amazon Web Services (AWS). By using GPU-accelerated nodes at AWS, the users were able to make significant advances in their research. The second phase of the project made AWS access available to all HECC users and added accounting to make users responsible for cloud charges. We are also enabling export-controlled work through the use of AWS GovCloud. In the third phase, we will add web-based mechanisms to permit non-HECC users to access cloud resources for their HPC projects.

Hood, Robert↗
NASA NTRS1989

Reflection of solar radiation by a cylindrical cloud

Potential applications of an analytic method for computing the solar radiation reflected by a cylindrical cloud are discussed, including studies of radiative transfer within finite clouds and evaluations of these effects on other clouds and on remote sensing problems involving finite clouds. The pattern of reflected sunlight from a cylindrical cloud as seen at a large distance has been considered and described by the bidirectional function method for finite cloud analysis, as previously studied theoretically for plane-parallel atmospheres by McKee and Cox (1974); Schmetz and Raschke (1981); and Stuhlmann et al. (1985). However, the lack of three-dimensional radiative transfer solutions for anisotropic scattering media have hampered theoretical investigations of bidirectional functions for finite clouds. The present approach permits expression of the directional variation of the radiation field as a spherical harmonic series to any desired degree and order.

Smith, G. L.↗
NASA NTRS1981

Transport of infrared radiation in cuboidal clouds

The transport of infrared radiation in a single cuboidal cloud is modeled using a variable azimuth two-stream approximation. Computations are made at 10 microns for a Deirmendjian (1969) C-1 water cloud where the single scattering albedo is equal to 0.638 and the asymmetry parameter is 0.865. The results indicate that the emittance of the top face of the model cloud is always less than that for a plane parallel cloud of the same optical depth. The hemispheric flux escaping from the cloud top possesses a gradient from the center to the edges which are warmer when the cloud is over warmer ground. Cooling rate calculations in the 8-13.6 micron region demonstrate that there is cooling out of the sides of the cloud at all levels even when there is heating of the core from the ground below. The radiances exiting from model cuboidal clouds are computed by path integration over the source function obtained with the two-stream approximation. Results indicate that the brightness temperature measured from finite clouds will overestimate the cloud-top temperature.

Harshvardhan, MR.↗
NASA NTRS1982

Applications of stereoscopic height computations from dual geosynchronous satellite data/joint NASA-Japan stereo project

Stereoscopic heights of the top of an Oklahoma thunderstorm were computed, finding that high cloud tops are not always characterized by very cold IR temperature. The identical method was also applied to the computation of stereo heights based on GOES West and GMS stereo pairs obtained under the NASA-JAPAN cooperative program. It was found that stereo techniques are extremely useful in understanding the structure of thunderstorms in the United States, as well as that of hurricanes over the South Pacific.

Fujita, T. T.↗