Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “tokenization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Addressing tokens dynamic generation, propagation, storage and renewal to secure the GlideinWMS pilot based jobs and system

GlideinWMS has been one of the first middleware in the WLCG community to transition from X.509 to support also tokens. The first step was to get from the prototype in 2019 to using tokens in production in 2022. This paper will present the challenges introduced by the wider adoption of tokens and the evolution plans for securing the pilot infrastructure of GlideinWMS and supporting the new requirements. In the last couple of years, the GlideinWMS team supported the migration of experiments and resources to tokens. Inadequate support in the current infrastructure, more stringent requirements, and the higher spatial and temporal granularity forced GlideinWMS to revisit once more how credentials are generated, used, and propagated. The new credential modules have been designed to be used in multiple systems (GlideinWMS, HEPCloud) and use a model where credentials have type, purpose, and different flows. Credentials are dynamically generated in order to customize the duration and limit the scope to the targeted resource. This allows to enforce the least privilege principle. Finally, we also considered adding credential storage, renewal, and invalidation mechanisms within the GlideinWMS infrastructure to better serve the experiments' needs.

Coimbra, Bruno Moreira [Fermilab] (ORCID:000900022↗
NASA NTRS1992

Performance comparison of token ring protocols for hard-real-time communication

The ability to guarantee the deadlines of synchronous messages while maintaining a good aggregate throughput is an important consideration in the design of distributed real-time systems. In this paper, we study two token ring protocols, the priority driven protocol and the timed token protocol, for their suitability for hard real-time systems. Both these protocols use a token to control access to the transmission medium. In a priority driven protocol, messages are assigned priorities and the protocol ensures that messages are transmitted in the order of their priorities. Timed token protocols do not provide for priority arbitration but ensure that the maximum access delay for a station is bounded. For both protocols, we first derive the schedulability conditions under which the transmission deadlines of a given set of synchronous messages can be guaranteed. Subsequently, we use these schedulability conditions to quantitatively compare the average case behavior of the protocols. This comparison demonstrates that each of the protocols has its domain of superior performance and neither dominates the other for the entire range of operating conditions.

Kamat, Sanjay↗
NASA NTRS1987

Throughput analysis of the IEEE 802.4 token bus standard under heavy load

It has become clear in the last few years that there is a trend towards integrated digital services. Parallel to the development of public Integrated Services Digital Network (ISDN) is service integration in the local area (e.g., a campus, a building, an aircraft). The types of services to be integrated depend very much on the specific local environment. However, applications tend to generate data traffic belonging to one of two classes. According to IEEE 802.4 terminology, the first major class of traffic is termed synchronous, such as packetized voice and data generated from other applications with real-time constraints, and the second class is called asynchronous which includes most computer data traffic such as file transfer or facsimile. The IEEE 802.4 token bus protocol which was designed to support both synchronous and asynchronous traffic is examined. The protocol is basically a timer-controlled token bus access scheme. By a suitable choice of the design parameters, it can be shown that access delay is bounded for synchronous traffic. As well, the bandwidth allocated to asynchronous traffic can be controlled. A throughput analysis of the protocol under heavy load with constant channel occupation of synchronous traffic and constant token-passing times is presented.

Pang, Joseph↗
DOE OSTI2025

Is tokenization needed for masked particle modeling?

In this work, we significantly enhance masked particle modeling (MPM), a self-supervised learning scheme for constructing highly expressive representations of unordered sets relevant to developing foundation models for high-energy physics. In MPM, a model is trained to recover the missing elements of a set, a learning objective that requires no labels and can be applied directly to experimental data. We achieve significant performance improvements over previous work on MPM by addressing inefficiencies in the implementation and incorporating a more powerful decoder. We compare several pre-training tasks and introduce new reconstruction methods that utilize conditional generative models without data tokenization or discretization. We show that these new methods outperform the tokenized learning objective from the original MPM on a new test bed for foundation models for jets, which includes using a wide variety of downstream tasks relevant to jet physics, such as classification, secondary vertex finding, and track identification.

conditional generative models↗
NASA NTRS1990

Distributed computing system with dual independent communications paths between computers and employing split tokens

This is a distributed computing system providing flexible fault tolerance; ease of software design and concurrency specification; and dynamic balance of the loads. The system comprises a plurality of computers each having a first input/output interface and a second input/output interface for interfacing to communications networks each second input/output interface including a bypass for bypassing the associated computer. A global communications network interconnects the first input/output interfaces for providing each computer the ability to broadcast messages simultaneously to the remainder of the computers. A meshwork communications network interconnects the second input/output interfaces providing each computer with the ability to establish a communications link with another of the computers bypassing the remainder of computers. Each computer is controlled by a resident copy of a common operating system. Communications between respective ones of computers is by means of split tokens each having a moving first portion which is sent from computer to computer and a resident second portion which is disposed in the memory of at least one of computer and wherein the location of the second portion is part of the first portion. The split tokens represent both functions to be executed by the computers and data to be employed in the execution of the functions. The first input/output interfaces each include logic for detecting a collision between messages and for terminating the broadcasting of a message whereby collisions between messages are detected and avoided.

Rasmussen, Robert D.↗
NASA NTRS1990

Documentation for the token ring network simulation system

A manual is presented which describes the language features of the Token Ring Network Simulation System. The simulation system is a powerful simulation tool for token ring networks which allows the specification of various Medium Access Control (MAC) layer protocols as well as the specification of various features of upper layer ISO protocols. In addition to these features, it also allows the user to specify message and station classes virtually to any degree of detail desired. The choice of a language instead of an interactive system to specify network parameters was dictated by both flexibility and time considerations. The language was developed specifically for the simulation system, and is very simple. It is also user friendly in that language elements which do not apply to the case at hand are ignored rather than treated as errors.

Peden, Jeffery H.↗
NASA NTRS1986

A Priority Protocol for Token-Ring Networks

New priority protocol controls access to token-ring local-area network (LAN) of digital-communication stations over widely ranging mix of low- and high-priority traffic. Protocol, called round-robin priority scheme (RRPS), introduces only small overhead and therefore degrades system performance only minimally. Key messages guaranteed access to local-area network during peak loads.

Liu, H. T.↗
NASA NTRS1986

Fairness of channel access for non-time-critical traffic using the FDDI token ring protocol

The Fiber Distributed Data Interface (FDDI) is an ANSI draft proposed standard for a 100 megabit per second fiber optic token ring. FDDI supports two types of traffic, synchronous and asynchronous. Synchronous traffic is time critical traffic; stations are assigned guaranteed bandwidth to support their synchronous needs. Asynchronous traffic is lower priority and is sent only if time permits. It is proved analytically that the FDDI access protocol provides all stations on the ring with equal access to the channel to transmit asynchronous frames, regardless of the relative sizes of synchronous bandwidth allocations for individual stations. Analytic results are supported with data from simulation runs.

Johnson, M. J.↗
NASA NTRS1990

Queueing models for token and slotted ring networks

Currently the end-to-end delay characteristics of very high speed local area networks are not well understood. The transmission speed of computer networks is increasing, and local area networks especially are finding increasing use in real time systems. Ring networks operation is generally well understood for both token rings and slotted rings. There is, however, a severe lack of queueing models for high layer operation. There are several factors which contribute to the processing delay of a packet, as opposed to the transmission delay, e.g., packet priority, its length, the user load, the processor load, the use of priority preemption, the use of preemption at packet reception, the number of processors, the number of protocol processing layers, the speed of each processor, and queue length limitations. Currently existing medium access queueing models are extended by adding modeling techniques which will handle exhaustive limited service both with and without priority traffic, and modeling capabilities are extended into the upper layers of the OSI model. Some of the model are parameterized solution methods, since it is shown that certain models do not exist as parameterized solutions, but rather as solution methods.

Peden, Jeffery H.↗
DOE OSTI2026

Did You Win the GPU Cloud Lottery? Benchmarking from TFLOPS to Tokens/$

Cloud GPUs are commonly assumed to deliver consistent performance for a given GPU model. This assumption does not always hold: cloud providers employ diverse system configurations and virtualization mechanisms, and GPUs themselves exhibit non-negligible manufacturing variability (the silicon lottery). In this work, we present a large-scale measurement study of GPU performance variability across 11 cloud providers, covering over 3,500 physical GPUs and 6,800 benchmark runs. Our hierarchical analysis shows that while execution-level variation stays below 9%, performance varies by up to 38% across devices and providers for the same GPU model. Regression analysis indicates that driver- and OS-related software factors contribute less than 1% of the variance; instead, silicon lottery effects dominate observed performance variation, and cloud providers further amplify them through persistent, systematic second-order effects.

Slynko, Platon [Silicon Data, New York, USA] (ORCI↗
DOE OSTI2022

Security Evaluation of Smart Cards and Secure Tokens: Benefits and Drawbacks for Reducing Supply Chain Risks of Nuclear Power Plants

The supply chain attack pathway is being increasingly used by adversaries to bypass security controls and gain unauthorized access to sensitive networks and equipment (e.g., Critical Digital Assets). Cyber-attacks targeting supply chain generally aim to compromise the environments, products, or services of vendors and suppliers to inject, add, or substitute authentic software and hardware with malicious elements. These malicious elements are deemed to be authentic as they arise from the vendor or supplier (i.e., the supply chain). This research aims to leverage findings and assumptions made from the previous report to determine the security benefits and drawbacks of a smart card- based hardware root of trust. Smart cards can provide devices inside Nuclear Power Plants (NPP) with a secure environment to store keys in and perform sensitive operations such as digital signature generation. These abilities can be leveraged to increase supply chain cybersecurity by autonomously providing NPP Licensees with reports on device integrity, authenticity and measurements of executable and non-executable data.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRS1992

An image compression technique for use on token ring networks

A low complexity technique for compression of images for transmission over local area networks is presented. The technique uses the synchronous traffic as a side channel for improving the performance of an adaptive differential pulse code modulation (ADPCM) based coder.

Gorjala, B.↗
DOE OSTI2022

Name Normalizer

Our name normalizer and enrichment micro-service address the shortcomings of standard text normalization. Initially the text input is split into tokens. This process takes into account some conventions of formatting. Any dashes present between two tokens preserves the relationship of those two tokens. Conversely a comma between two tokens ensures that the separation between the two tokens is maintained. The order of tokens is also preserved. Standard text normalization (conversion to lowercase, trimming extra white space, and canonicalization) is then applied to the tokens. Once normalized every unique and sequence of tokens is given confidence values by comparing the normalized values to publicly available data.

Bleeker, Amelia↗
DOE OSTI2026

MTRE

Multi-Token Reliability Estimation (MTRE) is a lightweight, white-box hallucination detector for vision-language models. Instead of using only the first output token, MTRE aggregates logits from the first ~10 tokens and feeds them to a small attention-based reliability head; per-token scores are combined via a sequential log-likelihood-ratio test with early-stopping, and an MTRE-t variant calibrates thresholds via cross-fitting. MTRE reports average gains of +9.4% Accuracy and +14.8% AUROC over common baselines across MAD-Bench, MM-SafetyBench, MathVista, and arithmetic/counting tasks, while adding ~4.3M params and ~1% inference overhead (~26 MB VRAM, ~0.94 ms per detection). Key limitation: requires access to early token logits and is evaluated on a handful of open-source 7B VLMs.

Bhattarai, Manish [Los Alamos National Labs]↗
DOE OSTI2026

MATEY: multiscale adaptive transformer models for spatiotemporal physical systems

Accurate representation of the multiscale features in spatiotemporal physical systems using vision transformer architectures requires extremely long, computationally prohibitive token sequences. To address this issue, we propose two novel adaptive tokenization schemes that dynamically adjust patch sizes based on local features: one ensures convergent behavior to uniform patch refinement, while the other offers better computational efficiency. Moreover, we present a set of spatiotemporal attention schemes, where the temporal or axial spatial dimensions are decoupled, to evaluate their baseline computational and data efficiencies and to determine whether adaptive tokenization can improve this performance. We assess the performance of the proposed multiscale adaptive model, MATEY, in a sequence of experiments. Compared to a full spatiotemporal attention scheme or a scheme that decouples only the temporal dimension, we find that fully decoupled axial attention is less efficient and expressive, requiring more training time and model parameters to achieve the same accuracy. The experiments on the adaptive tokenization schemes show that, compared to a uniformly refined model, the proposed schemes achieve comparable or improved accuracy at a much lower cost in the tested two-dimensional settings. While the asymptotic analysis suggests the potential for favorable scaling, empirical validation at substantially longer sequence lengths remains to be performed in future work. Finally, we demonstrate in two fine-tuning tasks featuring different physics that models pretrained on PDEBench data outperform the ones trained from scratch, especially in the low data regime with frozen attention.

adaptive tokenization↗