Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Amazon Web Services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

SQL and NoSQL Databases for Cyber Physical Production Systems in Internet of Things for Manufacturing (IoTfM)

Abstract In this paper, the design and performance differences between Relational Database Management Systems (RDBMS) and NoSQL Database Systems are examined, with attention to their applicability for real-world Internet of Things for manufacturing (IoTfM) data. While previous work has extensively compared SQL and NoSQL for both generalized and IoT uses, this work specifically examines the tradeoffs and performance differences for manufacturing applications by using a high-fidelity data set collected from a large US manufacturing firm. Growing an IoT system beyond the pilot stage requires scalable data storage; this work seeks to determine the impact of selected database systems on data write performance at scale. Payload size and message frequency were used as the primary characteristics to maintain model fidelity in simulated clients. As the number of simulated asset clients grow, the data write latency was calculated to determine how both database systems’ performance were affected. To isolate the RDBMS and NoSQL differences, a cloud environment was created using Amazon Web Services (AWS) with two identical data ingestion pipelines: writing data to an RDMBS (1) using AWS Aurora MySQL, and (2) using AWS DynamoDB NoSQL. The findings may provide guidance for further experimentation in large-scale manufacturing IoT implementations.

Gamero, David↗

SU-F-T-658: Out-Of-Field Dose Comparison for TrueBeam Low Energy Beams for Extended Distances: Measurement Vs Monte Carlo Simulation

Purpose: Patient dose far from the treatment field is comprised of scatter from within the patient, and treatment head leakage. We quantify the treatment head leakage for TrueBeam linear accelerator for 6X and 6X-FFF beams by comparing measurements to Monte Carlo simulations for a variety of jaw sizes and collimator rotations. This work is conceptually similar to that of Kry et al. (Medical Physics 2006; 33: 4405), who considered a Clinac linear accelerator. Methods: Measurements were made using an EXRADIN A101 ion chamber positioned in the patient plane, at distances up to 100 cm from isocenter. Simulations were done using VirtuaLinac, the GEANT4-based Monte Carlo model of the TrueBeam treatment head, and an in-house (U. Virginia) GEANT4-based code. In-house code modelled an ion chamber with build-up, based on a CT scan of the chamber. VirtuaLinac included a detailed model of the treatment head shielding, and was run on the Amazon Web Services cloud to generate spherical phase space files surrounding the treatment head. These phase space files were imported into the in-house code. Results: Initial comparisons between measurements and simulation showed an excess of dose in the in-plane direction, away from the gantry, in the simulations. This was traced to an incomplete model of the shielding—specifically, the component holding the primary collimator was smaller in the model than in the TrueBeam. Modifications were made to VirtuaLinac to more closely match the engineering drawings. In the in-plane direction, the lowest out of field dose was away from gantry (negative abscissa values) at around 60 cm from isocenter, for fields smaller than 10×10 cm2. Out of field dose decreased with decreasing jaw size. Flattening-filter free beam produced out-of-field doses as low as 65% of those with flattened beam. Conclusion: Doses determined from simulation and measurement were in close agreement. Funding support from the Jefferson Trust Foundation.

60 APPLIED LIFE SCIENCES↗

DEMO: Action Recommendation for Cyber Resilience

In this demonstration we show the usefulness of our unifying graph-based model for the representation of infrastructure, behavior, and missions of cyber enterprise in both a software simulation and on an Amazon Web Services (AWS) instance. We show the effectiveness of our recommendation algorithm for preserving various system health metrics in both cases.

cyber security, resilience↗

materialsCloudCompute

materialsCloudCompute (mCC) is a software package for running easily parallelizable materials science theoretical computations on Amazon Web Services (AWS) infrastructure. mCC provides tools to partition out computational problem into smaller subproblems which can be run in parallel, and ensure that each of those subproblems completes successfully. It utilizes the following AWS services: EC2, S3, and RDS. It operates by launch a managing EC2 instance, which can launch worker instances that perform the calculations. The Managing instance tracks the subproblems and ensures that the Worker instances continue to operation, killing and relaunching them if they fail.

Forbes, David↗

PoroTomo Natural Laboratory Horizontal and Vertical Distributed Acoustic Sensing Data

This dataset includes links to the PoroTomo DAS data in both SEG-Y and hdf5 (via h5py and HSDS with h5pyd) formats with tutorial notebooks for use. Data are hosted on Amazon Web Services (AWS) Simple Storage Service (S3) through the Open Energy Data Initiative (OEDI). Also included are links to the documentation for the dataset, Jupyter Notebook tutorials for working with the data as it is stored in AWS S3, and links to data viewers in OEDI for the horizontal (DASH) and vertical (DASV) DAS datasets. Horizontal DAS (DASH) data collection began 3/8/16, paused, and then started again on 3/11/2016 and ended 3/26/2016 using zigzag trenched fiber optic cabels. Vertical DAS (DASV) data collection began 3/17/2016 and ended 3/28/16 using a fiber optic cable through the first 363 m of a vertical well. These are raw data files from the DAS deployment at (DASH) and below (DASV) the surface during testing at the PoroTomo Natural Laboratory at Brady Hot Spring in Nevada. SEG-Y and hdf5 files are stored in 30 second files organized into directories by day. The hdf5 files available via HSDS are stored in daily files. Metadata includes information on the timing of recording gaps and a file count is included that lists the number of files from each day of recording. These data are available for download without login credentials through the free and publicly accessible Open Energy Data Initiative (OEDI) data viewer which allows users to browse and download individual or groups of files.

15 GEOTHERMAL ENERGY↗

Analytical Functions for 200 West Pump-and-Treat SCADA Sensor Data

Historical operations at the U.S. Department of Energy’s Hanford Site included disposal of waste fluids to the subsurface in the 200 West Area on the Hanford Central Plateau. Subsequent infiltration of fluids has resulted in groundwater contamination with carbon tetrachloride, nitrate, uranium, technetium-99, and other contaminants. A pump-and-treat (P&T) system, with an extraction/injection well network and an aboveground treatment plant, was implemented as part of interim and final remedies in the 200 West area. The HYPATIA single-page web application (part of the SOCRATES suite) is being developed to provide access to and analysis of chemistry and treatment facility sensor data for this 200 West P&T system. For the web application, analytical algorithms were developed to perform summing, differencing, smoothing, outlier detection, change-point detection, mass flow rate, and injectivity calculations on the data. Candidate algorithms were identified and tested, with the best-performing algorithms then assembled for implementation in HYPATIA. Because HYPATIA is hosted on the Amazon Web Services (AWS) cloud computing platform, algorithms were implemented in a back-end AWS Lambda function that can be called by the HYPATIA front end. The Lambda function applies the requested data processing to specified data via functions written in R, Python, and JavaScript. Development, testing, and review of the data analysis algorithms was completed under an NQA-1 quality program. This new HYPATIA functionality will provide information to support site decisions regarding P&T system performance and optimization.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Data Fusion to Enhance Quality Control and Analysis with Instruments at the Marine and Coastal Research Laboratory

Deploying environmental monitoring instruments in the marine environment can be challenging, facing challenges around device survivability, biofouling and corrosion, and consistent data collection. This project explores the use of data fusion – the process of integrating multiple data sources to produce more consistent, accurate, and useful information – to build a consistent long-term monitoring system at the Marine and Coastal Research Laboratory (MCRL) in Sequim, Washington. Unused instruments that had been acquired from past projects were inventoried and deployments planned on the MCRL pier and floating dock. A total of 8 instruments were deployed including a tide gauge, hydrophone, acoustic Doppler current profiler (ADCP), photosynthetically active radiation (PAR) sensors, meteorological station, and three water quality sensors. Deployments were planned to be well-protected around the pier structure and a maintenance schedule was created for cleaning and recalibration. An automated data pipeline was created to aggregate data on edge computers that push data to Amazon Web Services (AWS) cloud storage every 15 minutes, performing automated quality control and data transformations using the Time Series Data Analytical Toolkit (TSDAT). Continued efforts are underway to maintain this system into the future, take a data-driven approach to maintenance scheduling, improve the reliability of the system, and share the data with a variety of end-users.

54 ENVIRONMENTAL SCIENCES↗

Solar PV, Wind Generation, and Load Forecasting Dataset for ERCOT 2018: Performance-Based Energy Resource Feedback, Optimization, and Risk Management (P.E.R.F.O.R.M.)

This report describes the Advanced Research Projects Agency-Energy Performance-Based Energy Resource Feedback, Optimization, and Risk Management (PERFORM) Electric Reliability Council of Texas (ERCOT) dataset consisting of load, solar, and wind deterministic and probabilistic forecasts at three timescales. This dataset consists of 1 year of time-coincident load, wind, and solar actuals and probabilistic forecasts for a region similar to ERCOT. All the data are stored in Hierarchical Data Format 5 (HDF5) files and have been uploaded to an Amazon Web Services repository. The ERCOT data set has 2 years (2017, 2018) of actuals and 1 year (2018) of probabilistic forecasts. These data are provided at various spatial (i.e., site-level, zone-level, and system-level) and temporal scales (i.e., day-ahead, intraday, and intra-hour). Specifically, data are provided for 125 existing wind sites, 22 existing solar sites, 139 proposed wind sites, and 204 proposed solar sites.

14 SOLAR ENERGY↗

Evaluating the Cloud for Capability Class Leadership Workloads

Cloud platforms offer a variety of benefits that are very appealing for a large scale HPC facility with a diverse and dynamic user base and workload set. At the same time, there is cause for concern about transitioning to the cloud. Incorporating cloud resources into existing HPC facilities or even fully transitioning to a cloud deployment poses significant challenges at the technical, organizational, and economic levels. Regardless, based on current trends it is highly likely that cloud platforms will become an integral component of many HPC centers in some form. To gain a better understanding of both the limitations and capabilities of current cloud infrastructures we evaluated the public offerings of the three leading cloud platforms (Amazon Web Services, Microsoft Azure, and Google Cloud Platform) using a selection of representative application workloads from our facility. Our findings show that while current HPC offerings are still nascent, significant progress is being made to address the present shortcomings. At the same time, significant challenges and questions remain about whether HPC cloud offerings will be able to deliver the full range of expected benefits.

97 MATHEMATICS AND COMPUTING↗

A cloud computing approach to massively-parallel distributed stochastic optimization with application to geomechanical reservoir characterization

Many search strategies used to optimize parameters during hydrologic model calibration are inherently limited in that simulations must be performed sequentially in order for the algorithm to converge. As a result, the overall time required to reach a solution increases linearly with the number of steps necessary. In contrast, sampling strategies that attempt to approximate the posterior distribution of the parameters do not require the model simulations to be run in sequence and therefore invite the use of a massively parallel approach across heterogeneous computing environments. We apply a cloud computing methodology to this problem where a head node running on Amazon Web Services continuously produces candidate models that are distributed to arbitrary compute environments to run the simulation, in our case Clemson’s Palmetto cluster computer. Thus the time required to obtain a sufficient number of samples to produce an adequate approximation of the posterior distribution of the parameters is limited by the number of available compute sites. Monte Carlo sampling of the posterior space would be duplicative and inefficient, thus we combine Latin hypercube methods with genetic algorithms to ensure that no large regions of the parameter space are left unexplored, and that high probability regions are investigated thoroughly. We independently assess the fit to different types of data sets without the need to construct a single objective function. Sets of Pareto optimal models are particularly valuable when the quality and information content of data is poorly known or conceptual model errors exist in our simulation. We illustrate our approach for a problem where in situ strain data are used to calibrate the parameters of an oil reservoir in the Avant oil field in Oklahoma. We use well injection and shut-in to stress a deep (500m) confined formation, and observe the geomechanical response using borehole strainmeters installed in the shallow subsurface (30m). These data were then used to calibrate the poroelastic properties of the reservoir and caprock along with the geometry of an elliptic channel associated with the formation. Using our approach we are able to identify issues with model error as well as substantial tradeoffs that occur within the parameters.

inversion, stochastic, geomechanics, poroelastic, ↗

Monte Carlo Simulation with CAD Interface for Calculation of 3D Maps of Residual Dose (CRADA)

Objective: To develop an easy-to-use software application to predict and mitigate radiation effects in research environment, space instruments, nuclear plants and medical facilities and help nonproliferation and national security efforts. Tech-X will develop standalone software libraries and command-line tools for ( 1) translating CAD into tessellated surfaces and tetrahedral meshes in GDML (for Geant4 and MARS 15), ROOT (for MARS 15) and HDF5 (for compact representation and for the visualization) formats, (2) healing CAD geometries to make them suitable for Monte Carlo simulations; (3) creating uniform and variable Cartesian and cylindrical meshes for detailed scoring; and ( 4) efficient Monte Carlo navigation in CAD geometries. JLAB will finish automation of simulations of residual dose in CAD geometries and integrate Tech-X software into Geant4 and MARS15. Finally, Tech-X will develop a Graphical User Interface to set up and heal CAD geometries, create input files, run and visualize simulations for residual dose. This application will run on local desktops, local and remote clusters and supercomputers and will be made available through public clouds, such as Amazon Web Services.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DuraMAT Data Hub

The DuraMAT Data Hub has been supporting the consortium for the past six years. The Data Hub has had success in supporting the projects, providing a platform for sharing data within projects and to the public, and learning how to better leverage the existing software platform and the available Amazon Web Services environment. During this new generation of the Data Hub, we are looking at ways to help improve the data hub architecture, user experience, and improve operations by taking advantage of new technology platforms and software that will be more impactful on the consortium researchers and the broader scientific community. In this poster we will look at the current operational capabilities, data dissemination, and development that will improve the system in the near and far future.

14 SOLAR ENERGY↗

Cloud-Based Demonstration of the Eastern Interconnection Situational Awareness Monitoring System (ESAMS)

This report describes a cloud-based implementation and field demonstration of the Eastern Interconnection Situational Awareness and Monitoring System (ESAMS). ESAMS was developed to support the detection and source localization of forced oscillations using synchrophasor measurements from tie-lines connecting areas served by different reliability coordinators (RCs), so that RCs could better coordinate their response to wide-area events. A previous effort had identified deployment barriers associated with hosting shared situational awareness tools at a single RC. To address these barriers, ESAMS was migrated to Amazon Web Services and evaluated in a six-month field demonstration. ISO New England (ISO-NE) and PJM streamed data to the platform using AWS Direct Connect and a site-to-site VPN, respectively. The resulting multi-utility measurement footprint enabled regional source localization across major portions of the U.S. Eastern Interconnection and supported routine identification of oscillation events. During the final three months of the trial, 24 events above 2 MW/MVAR were detected. The largest detected oscillation approached a 25 MW peak-to-peak amplitude, and the longest persisted intermittently for more than 11 hours. The demonstration also assessed operational considerations—including data transfer volumes, end-to-end latency, and cloud computing costs—and found that network and compute requirements were modest relative to typical cloud capabilities while providing performance comparable to prior on-premises deployments. Overall, the results indicate that cloud hosting can provide a practical path to shared interconnection-wide oscillation monitoring. The cloud ESAMS demonstration establishes a foundation for broader utility participation and for building future wide-area analytics that leverage measurements across organizational boundaries.

Follum, James D.↗

The Astronomy Commons Platform: A Deployable Cloud-based Analysis Platform for Astronomy

Abstract We present a scalable, cloud-based science platform solution designed to enable next-to-the-data analyses of terabyte-scale astronomical tabular data sets. The presented platform is built on Amazon Web Services (over Kubernetes and S3 abstraction layers), utilizes Apache Spark and the Astronomy eXtensions for Spark for parallel data analysis and manipulation, and provides the familiar JupyterHub web-accessible front end for user access. We outline the architecture of the analysis platform, provide implementation details and rationale for (and against) technology choices, verify scalability through strong and weak scaling tests, and demonstrate usability through an example science analysis of data from the Zwicky Transient Facility’s 1Bn+ light-curve catalog. Furthermore, we show how this system enables an end user to iteratively build analyses (in Python) that transparently scale processing with no need for end-user interaction. The system is designed to be deployable by astronomers with moderate cloud engineering knowledge, or (ideally) IT groups. Over the past 3 yr, it has been utilized to build science platforms for the DiRAC Institute, the ZTF partnership, the LSST Solar System Science Collaboration, and the LSST Interdisciplinary Network for Collaboration and Computing, as well as for numerous short-term events (with over 100 simultaneous users). In a live demo instance, the deployment scripts, source code, and cost calculators are accessible. 4 4 http://hub.astronomycommons.org/

79 ASTRONOMY AND ASTROPHYSICS↗

Soda Pop: A Time-Series Clustering, Alarming and Disease Forecasting Application

To introduce Soda Pop, an R/Shiny application designed to be a disease agnostic time-series clustering, alarming, and forecasting tool to assist in disease surveillance “triage, analysis and reporting” workflows within the Biosurveillance Ecosystem (BSVE). In this poster, we highlight the new capabilities that are brought to the BSVE by Soda Pop with an emphasis on the impact of methodological decisions. The Biosurveillance Ecosystem (BSVE) is a biological and chemical threat surveillance system sponsored by the Defense Threat Reduction Agency (DTRA). BSVE is intended to be user-friendly, multi-agency, cooperative, modular and threat agnostic platform for biosurveillance. In BSVE, a web-based workbench presents the analyst with applications (apps) developed by various DTRA funded researchers, which are deployed on-demand in the cloud(e.g., Amazon Web Services). These apps aim to address emerging needs and refine capabilities to enable early warning of chemical and biological threats for multiple users across local, state, and federal agencies. Soda Pop is an app developed by Pacific Northwest National Laboratory (PNNL) to meet the current needs of the BSVE for early warning and detection of disease outbreaks. Aimed for use bya diverse set of analysts, the application is agnostic to data source and spatial scale enabling it to be generalizable across many diseases and locations. To achieve this, we placed a particular emphasis on clustering and alerting of disease signals within Soda Pop without strong prior assumptions on the nature of observed diseased counts. Although designed to be agnostic to the data source, Soda Pop was initially developed and tested on data summarizing Influenza-Like Illness in military hospitals from collaboration with the Armed Forces Health Surveillance Branch. Currently, the data incorporated also includes the CDC’s National Notifiable Diseases Surveillance System(NNDSS) tables and the WHO’s Influenza A/B Influenza Data(Flunet). These data sources are now present in BSVE’s Postgres data storage for direct access. Soda Pop is designed to automate time-series tasks of data summarization, exploration, clustering, alarming and forecasting. Built as an R/Shiny application, Soda Pop is founded on the powerful statistical tool R.. Where applicable, Soda Pop facilitates nonparametric seasonal decomposition of time-series; hierarchical agglomerative clustering across reporting areas and between diseases within reporting areas; and a variety of alarming techniques including Exponential Weighted Moving Average alarms and Early Aberration Detection. Soda Pop embeds these techniques within a user-interface designed to enhance an analyst’s understanding of emerging trends in their data and enables the inclusion of its graphical elements into their dossier for further tracking and reporting. The ultimate goal of this software is to facilitate the discovery of unknown disease signals along with increasing the speed of detection of unusual patterns within these signals. Soda Pop organizes common statistical disease surveillance tasks in a manner integrated with BSVE data source inputs and outputs. The app analyzes time-series disease data and supports a robust set of clustering and alarming routines that avoid strong assumptions on the nature of observed disease counts. This attribute allows for flexibility in the data source, spatial scale, and disease types making it useful to a wide range of analysts.

97 MATHEMATICS AND COMPUTING↗

GOES-16 Data for LASSO-CACTI Overview Paper

GOES-16 L1b satellite radiances have been obtained for the LASSO-CACTI case dates. Specifically, the period in the ARM subset is for select days in the period October 26, 2018 through March 15, 2019. These files were downloaded from Amazon Web Services using the GOES-2-Go library, https://blaylockbk.github.io/goes2go/_build/html/.

{"GOES-16 band 13",radiance,LASSO-CACTI}↗

Oyster Fisheries App

This project is creating a cloud‐enabled, HTML 5 web application to help oyster fishermen and state agencies apply Earth science to improve the management of this important natural and economic resource. The Oyster Fisheries app gathers and analyzes environmental and water quality information, and alerts fishermen and resources managers about problems in oyster fishing waters. An intuitive interface based on Google Maps displays the geospatial information and provides familiar interactive controls to the users. Alerts can be tailored to notify users when conditions in specific leases or public fishing areas require attention. The app is hosted on the Amazon Web Services cloud. It is being developed and tested using some of the latest web development tools such as web components and Polymer.

Perez Guerrero, Geraldo A.↗