Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DIRECTORIES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

GUIDE: A Scalable Information Directory Service to Collect, Federate, and Analyze Logs for Operational Insights into a Leadership HPC Facility

In this paper, we describe the GUIDE framework to collect, federate, and analyze log data from the Oak Ridge Leadership Computing Facility's (OLCF), and how we use it derive insights into center operations. We collectsystem logs and extract monitoring data at every level of the various OLCF subsystems, and have developed a suite of pre-processing tools to make the raw data consumable. The cleansed logs are then ingested and federated into a central, scalable data warehouse, Splunk, that offers storage, indexing, querying, and visualization capabilities. We have further developed and deployed a set of analytics tools to analyze these multiple disparate log streams in concert, and derive operational insights. We describe our experience from developing and deploying the GUIDE infrastructure, and deriving valuable insights on the various subsystems, based on two years of operations in the production OLCF environment.

Vazhkudai, Sudharshan↗
DOE OSTI2010

Towards secure virtual directories : a risk analysis framework.

Directory services are used by almost every enterprise computing environment to provide data concerning users, computers, contacts, and other objects. Virtual directories are components that provide directory services in a highly customized manner. Unfortunately, though the use of virtual directory services are widespread, an analysis of risks posed by their unique position and architecture has not been completed. We present a detailed analysis of six attacks to virtual directory services, including steps for detection and prevention. We also describe various categories of attack risks, and discuss what is necessary to launch an attack on virtual directories. Finally, we present a framework to use in analyzing risks to individual enterprise computing virtual directory instances. We show how to apply this framework to an example implementation, and discuss the benefits of doing so.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2018

Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata

A file system is provided having a plurality of distributed metadata servers with separate metadata servers for file metadata and directory metadata. An exemplary distributed metadata storage system comprises a plurality of compute nodes, wherein at least one compute node comprises a file system client that generates a directory metadata request and/or a file metadata request; a plurality of directory metadata servers to serve directory metadata in response to a given directory metadata request; and a plurality of file metadata servers to serve file metadata in response to a given file metadata request. The plurality of directory metadata servers are distinct from the plurality of file metadata servers. The compute nodes optionally comprise an object client that requests object data from an object store system. The number of directory metadata servers and the number of file metadata servers are optionally determined independently to balance for different workloads, and can be dynamically readjusted for evolving workloads.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Label-based Virtual Directories In dCache

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

Sahakyan, Marina [DESY]↗
DOE OSTI2020

Distributed coherence directory subsystem with exclusive data regions

A processing system includes a first set of one or more processing units including a first processing unit, a second set of one or more processing units including a second processing unit, and a memory having an address space shared by the first and second sets. The processing system further includes a distributed coherence directory subsystem having a first coherence directory to support a first subset of one or more address regions of the address space and a second coherence directory to support a second subset of one or more address regions of the address space. In some implementations, the first coherence directory is implemented in the system so as to have a lower access latency for the first set, whereas the second coherence directory is implemented in the system so as to have a lower access latency for the second set.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Distributed coherence directory subsystem with exclusive data regions

A processing system includes a first set of one or more processing units including a first processing unit, a second set of one or more processing units including a second processing unit, and a memory having an address space shared by the first and second sets. The processing system further includes a distributed coherence directory subsystem having a first coherence directory to support a first subset of one or more address regions of the address space and a second coherence directory to support a second subset of one or more address regions of the address space. In some implementations, the first coherence directory is implemented in the system so as to have a lower access latency for the first set, whereas the second coherence directory is implemented in the system so as to have a lower access latency for the second set.

Eckert, Yasuko↗
DOE OSTI2010

Detecting insider activity using enhanced directory virtualization.

Insider threats often target authentication and access control systems, which are frequently based on directory services. Detecting these threats is challenging, because malicious users with the technical ability to modify these structures often have sufficient knowledge and expertise to conceal unauthorized activity. The use of directory virtualization to monitor various systems across an enterprise can be a valuable tool for detecting insider activity. The addition of a policy engine to directory virtualization services enhances monitoring capabilities by allowing greater flexibility in analyzing changes for malicious intent. The resulting architecture is a system-based approach, where the relationships and dependencies between data sources and directory services are used to detect an insider threat, rather than simply relying on point solutions. This paper presents such an architecture in detail, including a description of implementation results.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2013

Cache directory look-up re-use as conflict check mechanism for speculative memory requests

In a cache memory, energy and other efficiencies can be realized by saving a result of a cache directory lookup for sequential accesses to a same memory address. Where the cache is a point of coherence for speculative execution in a multiprocessor system, with directory lookups serving as the point of conflict detection, such saving becomes particularly advantageous.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Reader set encoding for directory of shared cache memory in multiprocessor system

In a parallel processing system with speculative execution, conflict checking occurs in a directory lookup of a cache memory that is shared by all processors. In each case, the same physical memory address will map to the same set of that cache, no matter which processor originated that access. The directory includes a dynamic reader set encoding, indicating what speculative threads have read a particular line. This reader set encoding is used in conflict checking. A bitset encoding is used to specify particular threads that have read the line.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Distributed directory of named data elements in coordination namespace

An approach is described that provides a distributed directory structure within a storage of an information handling system (a local node). A request is received with the request corresponding to a shared virtual address. The shared virtual address that is shared amongst a number of nodes that includes the local node and some remote nodes. A Global Address Space Directory (GASD) is retrieved that corresponds to a global virtual address space. The GASD is stored in a Coordination Namespace that is stored in a memory that is distributed amongst the nodes. A mapping that is included in the GASD is used to determine the node where the shared virtual address currently resides. The shared virtual address is then accessed from the node where it currently resides.

Johns, Charles Ray↗
DOE OSTI2011

A method of searching LDAP directories using XQuery

A method by which an LDAP directory can be searched using XQuery is described. The strategy behind the tool consists of four steps. First the XQuery script is examined and relevant XPath expressions are extracted, determined to be sufficient to define all information needed to perform the query. Then the XPath expressions are converted into their equivalent LDAP search filters by use of the published LDAP schema of the service, and search requests are made to the LDAP host. The search results are then merged and converted to an XML document that conforms to the hierarchy of the LDAP schema. Finally, the XQuery script is executed on the working XML document by conventional means. Examples are given of application of the tool in the Open Science Grid, which for discovery purposes operates an LDAP server that contains Glue schema-based information on site configuration and authorization policies. The XQuery scripts compactly replace hundreds of lines of custom python code that relied on the unix ldapsearch utility. Installation of the tool is available through the Virtual Data Toolkit.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Streaming Data Reorganization at Scale with DeltaFS Indexed Massive Directories

We report complex storage stacks providing data compression, indexing, and analytics help leverage the massive amounts of data generated today to derive insights. It is challenging to perform this computation, however, while fully utilizing the underlying storage media. This is because, while storage servers with large core counts are widely available, single-core performance and memory bandwidth per core grow slower than the core count per die. Computational storage offers a promising solution to this problem by utilizing dedicated compute resources along the storage processing path. We present DeltaFS Indexed Massive Directories (IMDs), a new approach to computational storage. DeltaFS IMDs harvest available (i.e., not dedicated) compute, memory, and network resources on the compute nodes of an application to perform computation on data. We demonstrate the efficiency of DeltaFS IMDs by using them to dynamically reorganize the output of a real-world simulation application across 131,072 CPU cores. DeltaFS IMDs speed up reads by 1,740x while only slightly slowing down the writing of data during simulation I/O for in situ data processing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

The Biosurveillance Analytics Resource Directory (BARD): Facilitating the use of epidemiological models for infectious disease surveillance

Epidemiological modeling for infectious disease is important for disease management and its routine implementation needs to be facilitated through better description of models in an operational context. A standardized model characterization process that allows selection or making manual comparisons of available models and their results is currently lacking. A key need is a universal framework to facilitate model description and understanding of its features. Los Alamos National Laboratory (LANL) has developed a comprehensive framework that can be used to characterize an infectious disease model in an operational context. The framework was developed through a consensus among a panel of subject matter experts. In this paper, we describe the framework, its application to model characterization, and the development of the Biosurveillance Analytics Resource Directory (BARD; http://brd.bsvgateway.org/brd/), to facilitate the rapid selection of operational models for specific infectious/communicable diseases. We offer this framework and associated database to stakeholders of the infectious disease modeling field as a tool for standardizing model description and facilitating the use of epidemiological models.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2014

Global Social Media Directory: A Resource Guide

The Global Social Media Directory is a resource guide providing information on social networking services around the globe. This information changes rapidly, therefore, this document will be updated on a regular basis and as funding permits.

social media, social networking services, user-gen↗
DOE OSTI2023

Input files and WRF run directory for a LASSO-CACTI simulation

Tar file containing files necessary for reproducing a given Weather Research and Forecasting (WRF) simulation from the Large-Eddy Simulation (LES) Atmospheric Radiation Measurement (ARM) Symbiotic Simulation and Observation (LASSO) deep-convection scenario for the Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign. The LASSO-CACTI simulations span grid spacings from 7.5 km to 100 m for convection near the Sierras de Córdoba mountain range, roughly centered on the ARM Mobile Facility. More information can be found at https://www.arm.gov/capabilities/modeling/lasso. Types of files in this tar include the initial and boundary conditions, WRF namelist, and other files present in a WRF run directory.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Cache directory lookup reader set encoding for partial cache line speculation support

In a multiprocessor system, with conflict checking implemented in a directory lookup of a shared cache memory, a reader set encoding permits dynamic recordation of read accesses. The reader set encoding includes an indication of a portion of a line read, for instance by indicating boundaries of read accesses. Different encodings may apply to different types of speculative execution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2013

Analysis of Subject Matter Expert (SME) Feedback on the First Version of the Biosurveillance Analytics Resource directory (BaRD)

As part of a recently funded project, LANL was tasked with the development of the BaRD, a reference tool in support of the Biosurveillance Ecosystem (BSVE) that will serve to enhance situational awareness during an infectious disease outbreak and provide timely and relevant information to support effective decision making to prevent, stop or control an outbreak or epidemic. The BaRD is being designed to enable rapid selection of appropriate models for use in disease prediction, monitoring, or forecasting at various population levels. It is a searchable, relational database that catalogs and classifies model specific information to allow facile reporting on tools of interest.

60 APPLIED LIFE SCIENCES↗