Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “SERVER”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

TagIt: An Integrated Indexing and Search Service for File Systems

Data services such as search, discovery, and management in scalable distributed environments have traditionally been decoupled from the underlying file systems, and are often deployed using external databases and indexing services. However, modern data production rates, looming data movement costs, and the lack of metadata, entail revisiting the decoupled file system-data services design philosophy.In this paper, we present TagIt, a scalable data management service framework aimed at scientific datasets, which is tightly integrated into a shared-nothing distributed file system. A key feature of TagIt is a scalable, distributed metadata indexing framework, using which we implement a flexible tagging capability to support data discovery. The tags can also be associated with an active operator, for pre-processing, filtering, or automatic metadata extraction, which we seamlessly offload to file servers in a load-aware fashion. Our evaluation shows that TagIt can expedite data search by up to 10X over the extant decoupled approach.

Sim, Hyogi↗
DOE OSTI2017

Diving into Petascale Production File Systems through Large Scale Profiling and Analysis

As leadership computing facilities grow their storage capacity into the multi- petabyte range, the number of files and directories leap into the scale of billions. A complete profiling of such a parallel file system in a production environment presents a unique challenge. On one hand, the time, resources, and negative performance impact on production users can make regular profiling difficult. On the other hand, the result of such profiling can yield much needed understanding of the file system's general characteristics, as well as provide insight to how users write and access their data on a grand scale. This paper presents a lightweight and scalable profiling solution that can efficiently walk, analyze, and profile multi-petabyte parallel file systems. This tool has been deployed and is in regular use on very large-scale production parallel file systems at both Oak Ridge National Lab's Oak Ridge Leadership Facility (OLCF) and Lawrence Livermore National Lab's Livermore Computing (LC) facilities. We present the results of our initial analysis on the data collected from these two large-scale production systems, organized into three use cases: (1) file system snapshot and composition, (2) striping pattern analysis for Lustre, and (3) simulated storage capacity utilization in preparation for future file systems. Our analysis shows that on the OLCF file system, over 96% of user files exhibit the default stripe width, potentially limiting performance on large files by underutilizing storage servers and disks. Our simulated block analysis quantitatively shows the space overhead when doing a forklift system migration. It also reveals that due to the difference in system compositions (OLCF vs. LC), we can achieve better performance and space trade-offs by employing different native file system block sizes.

Wang, Feiyi↗
DOE OSTI2018

WebGIobe - A cloud-based geospatial analysis framework for interacting with climate data

While climate models have evolved over time to produce high fidelity and high resolution climate forecasts, visualization and analysis of the output of the model simulations has been limited, typically constrained to single dimensional charts for visualization and basic aggregate statistics for analytics. Same is true for the large troves of observational data available from meteorological stations all over the world. For richer understanding of climate and the impact of climate change, one needs computational tools that allow researchers, policymakers, and general public, to interact with the climate data. In this paper, we describe, webGlobe, a browser based GIS framework for interacting with climate data, and other datasets available in similar format. webGlobe is a unique resource that allows unprecedented access to climate data through a browser. The framework also allows for deploying machine learning based analytical applications on the climate data without putting computational burden on the client. Instead, webGlobe uses a client-server framework, where the server, deployed on a cloud infrastructure, allows for dynamic allocation of resources for running compute-intensive applications. The capabilities of the framework will be discussed in context of a use case: identifying extreme events from real and simulated climate data using a Gaussian process based change detection algorithm.

Sharma, Arun↗
DOE OSTI2020

Shepherding Metadata Through the Building Lifecycle

Many different digital representations of a building are produced over the course of its lifecycle. These representations contain the metadata required to support different stages of the building, from initial planning and design, to construction and commissioning, through operations, audits, retrofits and maintenance. However, because of differences in the semantics, structure and syntax of these representations, the metadata they contain is not interoperable. We present a novel method for leveraging these representations to create a unified, authoritative Brick metadata model for a building that can be continually maintained over the course of the building lifecycle. A simple synchronization protocol relays inferred Brick metadata from existing metadata sources such as gbXML, BuildingSync, Project Haystack and Modelica to a central integration server, which merges the metadata into a valid Brick model.

Fierro, Gabe↗
DOE OSTI2011

Co-PylotDB - A Python-Based Single-Window User Interface for Transmitting Information to a Database

Co-PylotDB, written completely in Python, provides a user interface (UI) with which to select user and data file(s), directories, and file content, and provide or capture various other information for sending data collected from running any computer program to a pre-formatted database table for persistent storage. The interface allows the user to select input, output, make, source, executable, and qsub files. It also provides fields for specifying the machine name on which the software was run, capturing compile and execution lines, and listing relevant user comments. Data automatically captured by Co-PylotDB and sent to the database are user, current directory, local hostname, current date, and time of send. The UI provides fields for logging into a local or remote database server, specifying a database and a table, and sending the information to the selected database table. If a server is not available, the UI provides for saving the command that would have saved the information to a database table for either later submission or for sending via email to a collaborator who has access to the desired database.

Barnette, DanielW.↗
DOE OSTI2016

Sirepo for Synchrotron Radiation Workshop

Sirepo is an open source framework for cloud computing. The graphical user interface (GUI) for Sirepo, also known as the client, executes in any HTML5 compliant web browser on any computing platform, including tablets. The client is built in JavaScript, making use of the following open source libraries: Bootstrap, which is fundamental for cross-platform web applications; AngularJS, which provides a model–view–controller (MVC) architecture and GUI components; and D3.js, which provides interactive plots and data-driven transformations. The Sirepo server is built on the following Python technologies: Flask, which is a lightweight framework for web development; Jinja, which is a secure and widely used templating language; and Werkzeug, a utility library that is compliant with the WSGI standard. We use Nginx as the HTTP server and proxy, which provides a scalable event-driven architecture. The physics codes supported by Sirepo execute inside a Docker container. One of the codes supported by Sirepo is the Synchrotron Radiation Workshop (SRW). SRW computes synchrotron radiation from relativistic electrons in arbitrary magnetic fields and propagates the radiation wavefronts through optical beamlines. SRW is open source and is primarily supported by Dr. Oleg Chubar of NSLS-II at Brookhaven National Laboratory.

Nagler, Robert↗
DOE OSTI2016

Sirepo - Warp

Sirepo is an open source framework for cloud computing. The graphical user interface (GUI) for Sirepo, also known as the client, executes in any HTML5 compliant web browser on any computing platform, including tablets. The client is built in JavaScript, making use of the following open source libraries: Bootstrap, which is fundamental for cross-platform web applications; AngularJS, which provides a model–view–controller (MVC) architecture and GUI components; and D3.js, which provides interactive plots and data-driven transformations. The Sirepo server is built on the following Python technologies: Flask, which is a lightweight framework for web development; Jin-ja, which is a secure and widely used templating language; and Werkzeug, a utility library that is compliant with the WSGI standard. We use Nginx as the HTTP server and proxy, which provides a scalable event-driven architecture. The physics codes supported by Sirepo execute inside a Docker container. One of the codes supported by Sirepo is Warp. Warp is a particle-in-cell (PIC) code de-signed to simulate high-intensity charged particle beams and plasmas in both the electrostatic and electromagnetic regimes, with a wide variety of integrated physics models and diagnostics. At pre-sent, Sirepo supports a small subset of Warp’s capabilities. Warp is open source and is part of the Berkeley Lab Accelerator Simulation Toolkit.

Nagler, Robert↗
DOE OSTI2019

ldlms_plugins_llnl

ldlms_plugins_llnl contains LLNL-developed plugins for LDMS (Lightweight Distributed Metric Service), which is software for low overhead monitoring, transport and storage of cluster metric data. These plugins add to LDMS the capability to monitor Lustre file system clients, metadata servers, and object storage servers.

Morrone, ChristopherJ↗
DOE OSTI2011

DataMover-Lite

DataMover-Lite (DML) is a client program whose purpose is to automatically manage multi-file transfers to clients. It expects as its input a transfer-instruction file, and a local target directory where the files will be moved to. The input file contains the Logical File Names (LFNs) and the transfer URLs (TURLs) of files that need to be transferred. The TURLs can have various protocols depending on the server that manages the files, including gridftp, ftp, https, and srm. Given a multi-file request for a large number of files, the DML can issue concurrently a number of transfers in order to get the files. The number of transfers it can issue is set as a parameter and is usually less than 10. For example, if the multi-file is for 100 files, the DML may keep active only 5 transfers at any one time. When any of the 5 transfers completes it will initiate another transfer. It is not necessary for a single request to contain files whose TURLs have same protocol. If the request to the DML contains TURLs with various protocols, the DML will contact the various servers on different threads according to each TURL's protocol. The DMK take care if scheduling the transfers, monitoring their progress, releasing the files, and getting the authorization for each file to be transferred.

Natarajan, Vijaya↗
DOE OSTI2021

Sudoku Online Testing Framework v.0.2

The Sudoku Online Testing Framework provides a configurable basis for testing user interactions to solve visualization-related problems with sudoku puzzles and games. It consists of a server, a client, and test code. The python server implements sudoku puzzle logic and board selection functions. The JavaScript client implements a graphical user interface (GUI) front-end to display the puzzles and facilitate interactions with users. Within the GUI, logical rules may be selected. Cells may be selected to “pivot” on to make a board for each possible value in the cell. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-6409 O

Leger, Michelle↗
DOE OSTI2022

VizBrick

Brick (https://brickschema.org/) is a unified metadata schema to address the problem of building data standardization. Creating Brick models for building datasets means that the contents of the datasets are semantically described using the standard terms defined in the Brick ontology, and it will enable the benefits of data standardization, without having to recollect or reorganize the data. The challenge is that building brick models for building datasets leads to repeated manual trial and error processes, which can be time-consuming. VizBrick is a tool with a graphic/Web-based user interface that can assist users to create Brick models visually and interactively without having to understand the Resource Description Framework (RDF) syntax. VizBrick contains a web server that renders VizBrick web interface pages for browsers. The web server utilizes software components that (1) provide Brick ontology entity mapping to data column suggestions to users so that they can efficiently create their model; (2) provide keyword/Metadata-based search capability for easy find of relevant brick concepts and relations to their data columns

Lee, Sangkeun [Oak Ridge National Lab. (ORNL), Oak↗
DOE OSTI2023

mkite

mkite is a distributed computing platform for materials simulation. mkite is built with the server-client pattern, decoupling production databases from client runners. When used in combination with message brokers, mkite enables any available client to perform calculations without prior hardware specification on the server side. Furthermore, the software enables the creation of complex workflows with multiple inputs and branches, facilitating the exploration of combinatorial chemical spaces. The mkite suite provides recipes and tools to interact with package such as VASP, but is extensible to any other simulation package. Finally, mkite helps keeping the provenance of calculations in a SQL database. A complete description of the software is available at https://arxiv.org/abs/2301.08841.

Schwalbe Koda, Daniel↗
DOE OSTI2021

NSU Hack-a-Thon Workshop v.2.0

SAND2021-6722 O This is a fitness tracking mobile application and server created to train college-level students to build full stack software. The front end is written using an open-source framework called Flutter. The API server is written using an open-source python framework called Flask. The app functions as a workout tracker and is intentionally partially functional for the purpose of training students during the live workshops hosted by Sandia's HR recruiting teams. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Carpenter, Logan↗
DOE OSTI2022

Mappymatch [SWR 22-38]

Mappymatch is a pure-python package developed and open sourced by the National Renewable Energy Laboratory. It contains a collection of "Matchers" that enable matching a GPS trace (series of GPS coordinates) to a map. As of 8/14/2023, The current matchers are: LCSSMatcher: A matcher that implements the LCSS algorithm described in this paper. Works best with high resolution GPS traces. OsrmMatcher: A light matcher that pings an OSRM server to request map matching results. See the official documentation for more info. ValhallaMatcher: A matcher to ping a Valhalla server for map matching results. Currently supported map formats are: Open Street Maps NOTE: This software was formerly known as YAMM (Yet Another Map-Matcher)

Reinicke, Nicholas↗
DOE OSTI2012

Building Agent™ Software (BA) [SWRs 12-05 and 17-21]

Energy interfaces provide a new level of motivational information. Many studies have reported energy savings after dashboards for operators are installed, but these provide little actionable information to the occupant and the occupant cannot report individual comfort. The Building Agent™ (BA), in development at the National Renewable Energy Laboratory (NREL), provides an interface for the occupant to converse with the building control system and aims to actively engage and affect occupant behavior to achieve positive energy outcomes. Energy savings achieved via occupant behavioral changes could be as high as 25%, and could rival savings achieved by technological measures. For example, energy audits often show that more than 50% of the energy used in a building is consumed at night. Occupants leaving equipment and lighting on after they leave work is a large contributing factor. Similar savings opportunities have been identified when occupants were away from their desks 50% of the time, while most of their computers, lights, and other equipment remained powered on. Building Agent aims to create cohesion between building automation system (BAS) data, local measurements, and occupant feedback to provide visualizations, empowering occupants and building engineers to take diagnostic, proactive, energy-saving, and comfort-improving actions. The multilayer BA architecture consists of central server, measurement, and distributed application layers. The server layer aggregates global and localized measurements with occupant feedback to correlate perceived comfort with actual data throughout the building. The measurement layer complements BAS data with localized, environmental condition measurements at workstations. The application layer provides a bidirectional communications mechanism between the building and the occupants that accepts comfort feedback and provides energy performance information. The interface layer also uses three displays to influence building energy use: empathetic and personalized gauges, graphical data, and competitive rankings. Building Agent was implemented at NREL in the Research Support Facility (RSF), a LEED Platinum and Net Zero office building, which has received many awards. The constant feedback from occupants has enabled BA to better enhance the BAS data controls. Continued work is being done on BA to increase visualization of data and enhanced the user experience.

Long, Nicholas↗
DOE OSTI2010

Accessing the SEED Genome Databases via Web Services API: Tools for Programmers

Background: The SEED integrates many publicly available genome sequences into a single resource. The database contains accurate and up-to-date annotations based on the subsystems concept that leverages clustering between genomes and other clues to accurately and efficiently annotate microbial genomes. The backend is used as the foundation for many genome annotation tools, such as the Rapid Annotation using Subsystems Technology (RAST) server for whole genome annotation, the metagenomics RAST server for random community genome annotations, and the annotation clearinghouse for exchanging annotations from different resources. In addition to a web user interface, the SEED also provides Web services based API for programmatic access to the data in the SEED, allowing the development of third-party tools and mash-ups. Results: The currently exposed Web services encompass over forty different methods for accessing data related to microbial genome annotations. The Web services provide comprehensive access to the database back end, allowing any programmer access to the most consistent and accurate genome annotations available. The Web services are deployed using a platform independent service-oriented approach that allows the user to choose the most suitable programming platform for their application. Example code demonstrate that Web services can be used to access the SEED using common bioinformatics programming languages such as Perl, Python, and Java. Conclusions: We present a novel approach to access the SEED database. Using Web services, a robust API for access to genomics data is provided, without requiring large volume downloads all at once. The API ensures timely access to the most current datasets available, including the new genomes as soon as they come online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

WTFgenes: What's The Function of these genes? Static sites for model-based gene set analysis

A common technique for interpreting experimentally-identified lists of genes is to look for enrichment of genes associated with particular ontology terms. The most common test uses the hypergeometric distribution; more recently, a model-based test was proposed. These approaches must typically be run using downloaded software, or on a server. We develop a collapsed likelihood for model-based gene set analysis and present WTFgenes, an implementation of both hypergeometric and model-based approaches, that can be published as a static site with computation run in JavaScript on the user's web browser client. Apart from hosting files, zero server resources are required: the site can (for example) be served directly from Amazon S3 or GitHub Pages. A C++11 implementation yielding identical results runs roughly twice as fast as the JavaScript version. WTFgenes is available from https://github.com/evoldoers/wtfgenes under the BSD3 license. A demonstration for the Gene Ontology is usable at https://evoldoers.github.io/wtfgo.

Mungall, Christopher J.↗
DOE OSTI2017

Control and Information Systems for the National Ignition Facility

Orchestration of every National Ignition Facility (NIF) shot cycle is managed by the Integrated Computer Control System (ICCS), which uses a scalable software architecture running code on more than 1950 front-end processors, embedded controllers, and supervisory servers. The ICCS operates laser and industrial control hardware containing 66 000 control and monitor points to ensure that all of NIF’s laser beams arrive at the target within 30 ps of each other and are aligned to a pointing accuracy of less than 50 μm root-mean-square, while ensuring that a host of diagnostic instruments record data in a few billionths of a second. NIF’s automated control subsystems are built from a common object-oriented software framework that distributes the software across the computer network and achieves interoperation between different software languages and target architectures. A large suite of business and scientific software tools supports experimental planning, experimental setup, facility configuration, and post-shot analysis. Standard business services using open-source software, commercial workflow tools, and database and messaging technologies have been developed. An information technology infrastructure consisting of servers, network devices, and storage provides the foundation for these systems. Thus, this work is an overview of the control and information systems used to support a wide variety of experiments during the National Ignition Campaign.

42 ENGINEERING↗