Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “tokenization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Simple, Secure, Internet Delivery of MOOSE-based Applications

Application packaging and distribution are the final steps for delivering software to end-users; both are frequently neglected when creating scientific software. Commercial businesses rely on electronic distribution systems that have rendered disk drives obsolete. Still, national laboratories continue to rely heavily on removable media to distribute and limit access to controlled applications. With increasing concerns of unauthorized copying of sensitive applications, a modern distribution system that utilizes cryptographically secure communication and authentication protocols has been developed. This new distribution system will secure the chain of custody for nuclear software while simultaneously simplifying access to these tools. This report summarizes four primary advancements made toward the secure distribution of Nuclear Energy Advanced Modeling and Simulation (NEAMS)-developed, Multiphysics Object Oriented Simulation Environment (MOOSE)-based applications: application installation, package distribution, automated package building, and distribution of documentation. NEAMS is currently developing more than ten separate applications based on the open-source MOOSE Framework. Distribution of these applications has primarily been accomplished by distributing source code, with end-users compiling the applications themselves. This work created a mechanism where MOOSE applications can be installed in a similar way to any other software. This allows both administrators and end-users simplified access to runnable executables. With this new installation capability, it was then possible to rethink distribution. A new, secure capability for delivering MOOSE-based applications over the internet has been created. This system requires unique cryptographic tokens for authentication, greatly securing the custody chain for software. Once granted access, installation of any NEAMS code can be accomplished with these terminal commands: "conda install ncrc" "ncrc install ncrc-bison." After these two commands (and authenticating) the BISON application will be securely down- loaded from Idaho National Laboratory (INL)’s servers, installed, and ready to use. To enable this new distribution capability to be successful, the open-source Continuous Integration, Verification, Enhancement, and Testing (CIVET) Continuous Integration (CI) capability was augmented to add Continuous Delivery (CD). CD enables the automated building and packaging of MOOSE-based applications as they are modified by development teams, ensuring that our customers can obtain up-to-date versions of the software at any time. The need for instruction on how to use these applications was addressed through modifications to the MOOSE documentation system. The MooseDocs capability, which enables robust documentation of MOOSE-based applications, has been extended to allow both for the installation of documentation and the packaging of documentation with installed applications. Together, these enhancements form the core of a new, secure distribution mechanism for nuclear simulation tools. In concert with the Nuclear Computational Resource Center (NCRC), NEAMS- developed applications will now be straightforward to obtain securely.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Hypothesis testing via AI: Generating physically interpretable models of scientific data with machine learning (Full Technical Report)

Deep learning has demonstrated an exceptional ability to solve complex tasks (an engineering success); however, it has done so at the expense of the ability to generate new knowledge (a scientific failure). We propose an alternative framework—entitled Deep Symbolic Regression (DSR)—in which artificial neural networks (NNs) rapidly generate hypotheses about physical relationships among inputs. This framework bypasses the need to interpret an NN altogether, while still leveraging the representational power of deep learning. The resulting models are tractable mathematical expressions, which are inherently and readily human interpretable and can provide insights into underlying physical phenomena. Further, we fold this methodology into the scientific process by allowing the scientist to directly integrate a priori knowledge and beliefs to accelerate learning. We demonstrate this methodology on symbolic regression—the problem of rediscovering underlying expressions describing a dataset—and achieve state-of-the-art performance across a wide variety of symbolic regression problems. Further, we generalize our DSR framework to apply to the more general class of symbolic optimization problems, in which one seeks to optimize a sequence of symbols or “tokens” under a black-box reward function. Examples of other symbolic optimization problems include neural architecture search and computational antibody design. Our generalized tool, Deep Symbolic Optimization (DSO), has been demonstrated on the task of learning symbolic control policies for reinforcement learning environments, and has been adopted as an enabling capability for computational antibody design.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

2024 OES-Environmental 2024 State of the Science Report, Chapter 5: Stakeholder Engagement for Marine Renewable Energy

Stakeholder engagement is a critical piece of any new development project that affects public or private interests. Effective, thoughtful engagement and participatory activities early in the planning process of a project can help planners and project developers understand local concerns, adjust designs to avoid negative environmental impacts, select the best site for a project, answer questions, reduce delay, enhance opportunities and benefits, and build support for a project (Cuppen et al. 2016; Portman 2009; Wiersma & DevineWright 2014). On the other hand, cursory or inadequate engagement that is viewed as “checking the box” or tokenism is unlikely to be effective, and can result in project failures, diminished trust, strong opposition, or costly, drawn-out processes (Butcher & MacLennan 2020; Garard & Kowarsch 2017; Gill & Rand 2022; Jolivet & Heiskanen 2010; Pizzi et al. 2021; Sterling et al. 2017).

16 TIDAL AND WAVE POWER↗
DOE OSTI2024

IRI Technology Landscape – A survey of re-usable components and methodologies

This document describes technical implementation details on network access schemes connecting API-driven workflows to supercomputer centers. API-driven workflows are a central theme in connected computing, since they bring the terminal-mainframe' access pattern present since the 1970s up to the task of interfacing with modern web browser technologies. Both security (HTTPS/TLS/IPSec/VPNs/public key cryptography/digital signatures) and network protocol stacks (HTTP-REST APIs, tokens, gRPC, SRTP) have evolved to the point where implementing API-driven workflows is possible using stable, secure off-the-shelf software.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Lyngaas, Isaac↗
DOE OSTI2024

Optimizing Distributed Training on Frontier for Large Language Models

Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Loss scaling studies have demonstrated the superior performance of larger LLMs compared to their smaller counterparts. Nevertheless, training LLMs with billions of parameters poses significant challenges and requires considerable computational resources. For example, training a one trillion parameter GPT-style model on 20 trillion tokens requires a staggering 120 million exaflops. This research explores efficient distributed training strategies to extract this computation from Frontier, the world's first exascale supercomputer. We enable and investigate various model and data parallel training techniques, such as tensor parallelism, pipeline parallelism, and sharded data parallelism, to facilitate training a trillion-parameter model on Frontier. We empirically assess these techniques and their associated parameters to determine their impact on memory footprint, communication latency, and GPU's computational efficiency. We analyze the complex interplay among these techniques and find a strategy to combine them to achieve high throughput through hyperparameter tuning. We have identified efficient strategies for training large LLMs of varying sizes through empirical analysis and hyperparameter tuning. For 22 Billion, 175 Billion, and 1 Trillion parameters, we achieved GPU throughputs of 38.38%, 36.14%, and 31.96%, respectively. For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved 100% weak scaling efficiency on 1024 and 3072 Mi250X GPUs, respectively. We also achieved strong scaling efficiencies of 89% and 87% for these two models. We trained these models only tens of iterations instead of training till completion.

Yin, Junqi↗
NASA NTRS1971

Study of aircraft in intraurban transportation systems, San Francisco Bay area

The nine-county San Francisco Bay area is examined in two time periods (1975-1980 and 1985-1990) as a scenario for analyzing the characteristics of an intraurban, commuter-oriented aircraft transportation system. Aircraft have dominated the long-haul passenger market for some time, but efforts to penetrate the very-short-haul intraurban market have met with only token success. Yet, the characteristics of an aircraft transportation system-speed and flexibility-are very much needed to solve the transportation ills of our major urban areas. This study attempts to determine if the aircraft can contribute toward solving the transportation problems of major metropolitan areas and be economically viable in such an environment.

Source record↗
NASA NTRS1972

Study of aircraft in intraurban transportation systems, San Francisco Bay area

The nine-county San Francisco Bay area is examined in two time periods (1975-1980 and 1985-1990) as a scenario for analyzing the characteristics of an intraurban, commuter-oriented aircraft transportation system. Aircraft have dominated the long-haul passenger market for some time, but efforts to penetrate the very-short-haul intraurban market have met with only token success. Yet, the characteristics of an aircraft transportation system, speed and flexibility, are very much needed to solve the transportation ills of our major urban areas. The aircraft intraurban system is a technically feasible alternative to ground transportation systems. Although requiring some subsidy, it becomes socially viable where substantial commuter traffic exists at ranges of 10 to 15 mi or more and where topographic features constrain ground travel. The general problem areas of community noise, air traffic congestion, ground transportation interface, pollution, and safety appear to have workable solutions.

Source record↗
NASA NTRS1975

Gas bearing operates in vacuum

Bearing has restrictions to reduce air leaks and is connected to external pumpout facility which removes exhausted air. Token amount of air which is lost to vacuum is easily removed by conventional vacuum pump.

Perkins, G. S.↗
NASA NTRS1975

Mars - A planet with a complex surface evolution

The surface of Mars has evolved to its present form through a complex sequence of tectonism and associated volcanism, impact processes, water erosion, mass movements, and wind action. The diversity of geological processes active in past Martian history far exceeded most predictions. By the same token, predictions of processes modifying the satellites of the outer planets may fall far short of the true range of phenomena. A summary of present though with regard to Martian surface evolution is presented to serve as a case in point of the value of imagery and topography data in making interpretations of geological histories.

Arvidson, R. E.↗
NASA NTRS1977

Studies in interactive communication. II - The effects of four communication modes on the linguistic performance of teams during cooperative problem solving

Two-man teams solved credible, 'real world' problems for which computer assistance has been or could be useful. Conversations were carried on in one of four modes of communication: typewriting, handwriting, voice, and natural unrestricted communication. Performance was assessed on three classes of dependent measures: time to solution, behavioral measures of activity, and linguistic measures. Significant differences among the communication modes were found in each of the three classes. This paper is concerned mainly with the results of the linguistic analyses. Linguistic performance was assessed with 182 measures, most of which turned out to be redundant and some of which were useless or meaningless. Those that remain show that although problems can be solved faster in the oral modes than in the hard-copy modes, the oral modes are characterized by many more messages, sentences, words, and unique words; much higher communication rates; but lower type-token ratios. Although a number of significant problem and job-role effects were found, there were relatively few significant interactions of modes with thsse variables. It appears, therefore, that the mode effects hold for both problems and for both job roles assigned to the subjects.

Chapanis, A.↗
NASA NTRS1979

An algorithm for retrieval of ocean surface and atmospheric parameters from the observations of the scanning multichannel microwave radiometer (SMMR)

A formalism was developed which can be used to interpret the data in terms of sea surface temperature, sea surface wind speed, and the atmospheric overburden of water vapor and liquid water. It was shown with reasonable instrumental performance assumptions, these parameters could be derived to useful accuracies. Although the algorithms were not derived for use in rain, it is shown that, at least, token rain rates can be tolerated without invalidating the retrieved geophysical parameters.

Wilheit, T. T.↗
NASA NTRS1980

An algorithm for retrieval of ocean surface and atmospheric parameters from the observations of the scanning multichannel microwave radiometer

The scanning multichannel microwave radiometer is a five-frequency (6.6, 10.7, 18, 21, and 37 GHz), dual-polarized microwave radiometer, which was launched in two separate satellites, Nimbus 7 and Seasat, in 1978. A formalism is developed which can be used to interpret the data in terms of sea surface temperature, sea surface wind speed, and the atmospheric content of water vapor and liquid water. It is shown with reasonable instrumental performance assumptions that these parameters can be derived to useful accuracies. Although the algorithms are not derived for use in rain, it is shown that at least token rain rates can be tolerated without invalidating the retrieved geophysical parameters.

Wilheit, T. T.↗
NASA NTRS1985

RATFOR user's guide version 2.0

This document is a user's guide for RATFOR at Ames Research Center. The main part of the document is a general description of RATFOR, and the appendix is devoted to a machine specific implementation for the Cray X-MP. The general stylistic features of RATFOR are discussed, including the block structure, keywords, source code, format, and the notion of tokens. There is a section on the basic control structures (IF-ELSE, ELSE IF, WHILE, FOR, DO, REPEAT-UNTIL, BREAK, NEXT), and there is a section on the statements that extend FORTRAN's capabilities (DEFINE, MACRO, INCLUDE, STRING). THE appendix discusses everything needed to compile and run a basic job, the preprocessor options, the supported character sets, the generated listings, fatal errors, and program limitations and the differences from standard FORTRAN.

Helmle, L. C.↗
NASA NTRS1985

A 250-Mbit/s ring local computer network using 1.3-microns single-mode optical fibers

A 250-Mbit/s three-station fiber-optic ring local computer network was built and successfully demonstrated. A conventional token protocol was employed for bus arbitration to maximize the bus efficiency under high loading conditions, and a non-return-to-zero (NRS) data encoding format was selected for simplicity and maximum utilization of the ECL-circuit bandwidth.

Eng, S. T.↗
NASA NTRS1987

A static data flow simulation study at Ames Research Center

Demands in computational power, particularly in the area of computational fluid dynamics (CFD), led NASA Ames Research Center to study advanced computer architectures. One architecture being studied is the static data flow architecture based on research done by Jack B. Dennis at MIT. To improve understanding of this architecture, a static data flow simulator, written in Pascal, has been implemented for use on a Cray X-MP/48. A matrix multiply and a two-dimensional fast Fourier transform (FFT), two algorithms used in CFD work at Ames, have been run on the simulator. Execution times can vary by a factor of more than 2 depending on the partitioning method used to assign instructions to processing elements. Service time for matching tokens has proved to be a major bottleneck. Loop control and array address calculation overhead can double the execution time. The best sustained MFLOPS rates were less than 50% of the maximum capability of the machine.

Barszcz, Eric↗
NASA NTRS1987

Network protocols for real-time applications

The Fiber Distributed Data Interface (FDDI) and the SAE AE-9B High Speed Ring Bus (HSRB) are emerging standards for high-performance token ring local area networks. FDDI was designed to be a general-purpose high-performance network. HSRB was designed specifically for military real-time applications. A workshop was conducted at NASA Ames Research Center in January, 1987 to compare and contrast these protocols with respect to their ability to support real-time applications. This report summarizes workshop presentations and includes an independent comparison of the two protocols. A conclusion reached at the workshop was that current protocols for the upper layers of the Open Systems Interconnection (OSI) network model are inadequate for real-time applications.

Johnson, Marjory J.↗
NASA NTRS1988

Intelligent Resource Management for Local Area Networks: Approach and Evolution

The Data Management System network is a complex and important part of manned space platforms. Its efficient operation is vital to crew, subsystems and experiments. AI is being considered to aid in the initial design of the network and to augment the management of its operation. The Intelligent Resource Management for Local Area Networks (IRMA-LAN) project is concerned with the application of AI techniques to network configuration and management. A network simulation was constructed employing real time process scheduling for realistic loads, and utilizing the IEEE 802.4 token passing scheme. This simulation is an integral part of the construction of the IRMA-LAN system. From it, a causal model is being constructed for use in prediction and deep reasoning about the system configuration. An AI network design advisor is being added to help in the design of an efficient network. The AI portion of the system is planned to evolve into a dynamic network management aid. The approach, the integrated simulation, project evolution, and some initial results are described.

Meike, Roger↗