Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “job execution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

The VLBA correlator: Real-time in the distributed era

The correlator is the signal processing engine of the Very Long Baseline Array (VLBA). Radio signals are recorded on special wideband (128 Mb/s) digital recorders at the 10 telescopes, with sampling times controlled by hydrogen maser clocks. The magnetic tapes are shipped to the Array Operations Center in Socorro, New Mexico, where they are played back simultaneously into the correlator. Real-time software and firmware controls the playback drives to achieve synchronization, compute models of the wavefront delay, control the numerous modules of the correlator, and record FITS files of the fringe visibilities at the back-end of the correlator. In addition to the more than 3000 custom VLSI chips which handle the massive data flow of the signal processing, the correlator contains a total of more than 100 programmable computers, 8-, 16- and 32-bit CPUs. Code is downloaded into front-end CPU's dependent on operating mode. Low-level code is assembly language, high-level code is C running under a RT OS. We use VxWorks on Motorola MVME147 CPU's. Code development is on a complex of SPARC workstations connected to the RT CPU's by Ethernet. The overall management of the correlation process is dependent on a database management system. We use Ingres running on a Sparcstation-2. We transfer logging information from the database of the VLBA Monitor and Control System to our database using Ingres/NET. Job scripts are computed and are transferred to the real-time computers using NFS, and correlation job execution logs and status flow back by the route. Operator status and control displays use windows on workstations, interfaced to the real-time processes by network protocols. The extensive network protocol support provided by VxWorks is invaluable. The VLBA Correlator's dependence on network protocols is an example of the radical transformation of the real-time world over the past five years. Real-time is becoming more like conventional computing. Paradoxically, 'conventional' computing is also adopting practices from the real-time world: semaphores, shared memory, light-weight threads, and concurrency. This appears to be a convergence of thinking.

Wells, D. C.↗
NASA NTRS2004

An Execution Service for Grid Computing

This paper describes the design and implementation of the IPG Execution Service that reliably executes complex jobs on a computational grid. Our Execution Service is part of the IPG service architecture whose goal is to support location-independent computing. In such an environment, once n user ports an npplicntion to one or more hardware/software platfrms, the user can describe this environment to the grid the grid can locate instances of this platfrm, configure the platfrm as required for the application, and then execute the application. Our Execution Service runs jobs that set up such environments for applications and executes them. These jobs consist of a set of tasks for executing applications and managing data. The tasks have user-defined starting conditions that allow users to specih complex dependencies including task to execute when tasks fail, afiequent occurrence in a large distributed system, or are cancelled. The execution task provided by our service also configures the application environment exactly as specified by the user and captures the exit code of the application, features that many grid execution services do not support due to dflculties interfacing to local scheduling systems.

Smith, Warren↗
NASA NTRS2007

Grid Task Execution

IPG Execution Service is a framework that reliably executes complex jobs on a computational grid, and is part of the IPG service architecture designed to support location-independent computing. The new grid service enables users to describe the platform on which they need a job to run, which allows the service to locate the desired platform, configure it for the required application, and execute the job. After a job is submitted, users can monitor it through periodic notifications, or through queries. Each job consists of a set of tasks that performs actions such as executing applications and managing data. Each task is executed based on a starting condition that is an expression of the states of other tasks. This formulation allows tasks to be executed in parallel, and also allows a user to specify tasks to execute when other tasks succeed, fail, or are canceled. The two core components of the Execution Service are the Task Database, which stores tasks that have been submitted for execution, and the Task Manager, which executes tasks in the proper order, based on the user-specified starting conditions, and avoids overloading local and remote resources while executing tasks.

Hu, Chaumin↗
DOE OSTI2021

Game-Theoretic Approach for Grace-Period Policy in Supercomputers

Job scheduling at supercomputing facilities is important for achieving high utilization of these valuable resources while ensuring effective execution of jobs submitted by users. The jobs are scheduled according to their specified resource demands such as expected job completion times, and the available resources based on allocations. Jobs that overrun their allocated times are terminated, for example, after a grace-period. It is non-trivial and often very complex for users to accurately estimate the completion times of their jobs, and consequently they face a dilemma: underestimate the job time to have a higher priority and risk job termination due to overrun, or overestimate it to ensure its completion and risk its delayed execution. In this paper, we investigate whether providing grace-period can benefit facility performance by developing a game- theoretic model between a facility provider and multiple users for a simplified scheduling scenario based on job execution times. We present closed-form expressions for the provider’s and user’s best-response strategies to maximize their respective utility functions. We describe conditions under which offering a grace-period is advantageous to both facility provider and users by deriving the Nash equilibrium of the game.

He, Fei↗
DOE OSTI2026

AQDrop Quantum Service (AQDrop) v1.0

AQDrop is a job management system designed to streamline access to the Advanced Quantum Testbed (AQT) at NERSC (National Energy Research Scientific Computing Center). It serves as a centralized middleware layer between researchers and quantum processing hardware. Key Features: AQDrop provides a FastAPI-based server backed by PostgreSQL for job submission, queue management, and role-based access control (members, operators, and administrators). Users submit Qiskit circuits via JSON payloads, which are queued, dispatched to the QPU through the Qubic API, and returned as measurement counts. A Python client library and web dashboard round out the interface options. Primary Use: Researchers submit quantum circuit jobs from a laptop or login node; an operator client executes those jobs on the AQT's physical QPU and returns results — all coordinated through the central API. Advantages: Compared to ad-hoc or direct hardware access, AQDrop adds structured queue management, auditable job-status tracking and OAuth2 authentication — reducing scheduling conflicts and unauthorized access. Its containerized deployment also improves reproducibility and scalability. Overall, AQDrop functions as a purpose-built quantum job broker tailored to NERSC's specific hardware and institutional access requirements.

Caplinger, Evan [Lawrence Berkeley National Labora↗
NASA NTRS1995

Job Scheduling Under the Portable Batch System

The typical batch queuing system schedules jobs for execution by a set of queue controls. The controls determine from which queues jobs may be selected. Within the queue, jobs are ordered first-in, first-run. This limits the set of scheduling policies available to a site. The Portable Batch System removes this limitation by providing an external scheduling module. This separate program has full knowledge of the available queued jobs, running jobs, and system resource usage. Sites are able to implement any policy expressible in one of several procedural language. Policies may range from "bet fit" to "fair share" to purely political. Scheduling decisions can be made over the full set of jobs regardless of queue or order. The scheduling policy can be changed to fit a wide variety of computing environments and scheduling goals. This is demonstrated by the use of PBS on an IBM SP-2 system at NASA Ames.

Henderson, Robert L.↗
DOE OSTI2022

Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms

Many extreme scale scientific applications have workloads comprised of a large number of individual highperformance tasks. The Pilot abstraction decouples workload specification, resource management, and task execution via job placeholders and late-binding. As such, suitable implementations of the Pilot abstraction can support the collective execution of large number of tasks on supercomputers. We introduce RADICAL-Pilot (RP) as a portable, modular and extensible Pilot enabled runtime system. We describe RP's design, architecture and implementation. We characterize its performance and show its ability to scalably execute workloads comprised of tens of thousands heterogeneous tasks on DOE and NSF leadership-class HPC platforms. Specifically, we investigate RP's weak/strong scaling with CPU/GPU, single/multi core, (non)MPI tasks and python functions when using most of ORNL Summit and TACC Frontera. RADICAL-Pilot can be used stand-alone, as well as the runtime for third-party workflow systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Thermal images collected during large-scale 3D printing with Ingersoll MasterPrint

This data set contains images produced to test the performance of anomaly and fault detection methods in the context of additive manufacturing. Two print jobs were executed using the Ingersoll MasterPrint with the Model 30 Strangpresse extruder. The material used was Techmer compounded polylactic acid (PLA) with wood flour as a filler (80/20 PLA/WF by weight). The first print job consists of the 3D printing of a hexagonal cylinder with a two-bead wall. This print was sliced at a gantry velocity of 3000 millimeter per minute, and an extruder screw speed of 68.14 rotations per minute. This are considered the normal operating conditions. A second hexagonal cylinder was printed using a reduced extruder screw speed, 15% lower than under normal operating conditions. The images were collected with a Teledyne FLIR Lepton 3.5 infra-red camera, a small form factor radiometric long-wave infrared camera with a spectral range of 8 µm to 14 µm. Sensor resolution was 160x120 pixels, with a pixel size of 12 µm, a temperature range of -10 - 450°C, and an accuracy of +/- 10°C in its low gain configuration. Images in this data set were collected with cameras oriented at the printer nozzle. The nozzle camera setup consisted of two cameras located 12.5cm from the nozzle center. These were mounted directly to the print head, so that the camera positions relative to the print direction would remain constant as it rotated around its C-axis to follow the print path. One camera was placed ahead of the nozzle to capture the previous layer immediately before being covered by the new layer of material, while the second camera was placed behind the nozzle and captured the freshly extruded bead. Images are collecting during each phase of the print: (a) idle (i.e., no material deposited), (b) extrusion (i.e., to prime the extruder), and (c) printing (deposition of material to manufacture the hexagonal cylinder).

36 MATERIALS SCIENCE↗
DOE OSTI2026

Thermal images collected during 3D printing with Cincinnati BAAM

This data set contains images produced to test the performance of anomaly and fault detection methods in the context of additive manufacturing. A total of 10 print jobs were executed using a Cincinnati BAAM (model 606) with a Cincinnati medium compression screw. The material used was compounded polylactic acid (PLA) with wood flour as a filler produced by Jabil (80% NatureWorks Ingeo 6060D amorphous PLA, % 100 mesh pine flour from American Wood Fiber). Print sheets were 1/4in polycarbonate.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Offloading Calculations to Computational Storage Devices: Spark and HDFS [Slides]

The objective is to evaluate the capabilities of multiple CSDs (provided by NDG Systems) using Hadoop Filesystem and Apache Spark. The independent variables are: number of CSDs, 0, 1, 2, 4, or 6; size of dataset, 1 GB, 5 GB, 10 GB; type of dataset, one large file with all of the data, 10 files, 100 files. The dependent variables are: job time; execution time. the constants are operations on the dataset.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1979

ATLAS, an integrated structural analysis and design system. Volume 1: ATLAS user's guide

Some of the many analytical capabilities provided by the ATLAS Version 4.0 System in the logical sequence are described in which model-definition data are prepared and the subsequent computer job is executed. The example data presented and the fundamental technical considerations that are highlighted can be used as guides during the problem solving process. This guide does not describe the details of the ATLAS capabilities, but provides an introduction to the new user of ATLAS to the level at which the complete array of capabilities described in the ATLAS User's Manual can be exploited fully.

Dreisbach, R. L.↗
NASA NTRS1979

Performance and economy of a fault-tolerant multiprocessor

The FTMP (Fault-Tolerant Multiprocessor) is one of two central aircraft fault-tolerant architectures now in the prototype phase under NASA sponsorship. The intended application of the computer includes such critical real-time tasks as 'fly-by-wire' active control and completely automatic Category III landings of commercial aircraft. The FTMP architecture is briefly described and it is shown that it is a viable solution to the multi-faceted problems of safety, speed, and cost. Three job dispatch strategies are described, and their results with respect to job-starting delay are presented. The first strategy is a simple First-Come-First-Serve (FCFS) job dispatch executive. The other two schedulers are an adaptive FCFS and an interrupt driven scheduler. Three failure modes are discussed, and the FTMP survival probability in the face of random hard failures is evaluated. It is noted that the hourly cost of operating two FTMPs in a transport aircraft can be as little as one-to-two percent of the total flight-hour cost of the aircraft.

Lala, J. H.↗
NASA NTRS2010

Analysis of Skeletal Muscle Metrics as Predictors of Functional Task Performance

PURPOSE: The ability to predict task performance using physiological performance metrics is vital to ensure that astronauts can execute their jobs safely and effectively. This investigation used a weighted suit to evaluate task performance at various ratios of strength, power, and endurance to body weight. METHODS: Twenty subjects completed muscle performance tests and functional tasks representative of those that would be required of astronauts during planetary exploration (see table for specific tests/tasks). Subjects performed functional tasks while wearing a weighted suit with additional loads ranging from 0-120% of initial body weight. Performance metrics were time to completion for all tasks except hatch opening, which consisted of total work. Task performance metrics were plotted against muscle metrics normalized to "body weight" (subject weight + external load; BW) for each trial. Fractional polynomial regression was used to model the relationship between muscle and task performance. CONCLUSION: LPMIF/BW is the best predictor of performance for predominantly lower-body tasks that are ambulatory and of short duration. LPMIF/BW is a very practical predictor of occupational task performance as it is quick and relatively safe to perform. Accordingly, bench press work best predicts hatch-opening work performance.

Ryder, Jeffrey W.↗
DOE OSTI2022

Quantum Computing in the Cloud: Analyzing job and machine characteristics

As the popularity of quantum computing continues to grow, quantum machine access over the cloud is critical to both academic and industry researchers across the globe. And as cloud quantum computing demands increase exponentially, the analysis of resource consumption and execution characteristics are key to efficient management of jobs and resources at both the vendor-end as well as the client-end. While the analysis of resource consumption and management are popular in the classical HPC domain, it is severely lacking for more nascent technology like quantum computing. This paper is a first-of-its-kind academic study, analyzing various trends in job execution and resources consumption / utilization on quantum cloud systems. We focus on IBM Furthermore, quantum systems and analyze characteristics over a two year period, encompassing over 6000 jobs which contain over 600,000 quantum circuit executions and correspond to almost 10 billion “shots” or trials over 20+ quantum machines. Specifically, we analyze trends focused on, but not limited to, execution times on quantum machines, queuing/waiting times in the cloud, circuit compilation times, machine utilization, as well as the impact of job and machine characteristics on all of these trends. Furthermore, our analysis identifies several similarities and differences with classical HPC cloud systems. Based on our insights, we make recommendations and contributions to improve the management of resources and jobs on future quantum cloud systems.

42 ENGINEERING↗
NASA NTRS2003

Master Software Requirements Specification

A basic function of a computational grid such as the NASA Information Power Grid (IPG) is to allow users to execute applications on remote computer systems. The Globus Resource Allocation Manager (GRAM) provides this functionality in the IPG and many other grids at this time. While the functionality provided by GRAM clients is adequate, GRAM does not support useful features such as staging several sets of files, running more than one executable in a single job submission, and maintaining historical information about execution operations. This specification is intended to provide the environmental and software functional requirements for the IPG Job Manager V2.0 being developed by AMTI for NASA.

Hu, Chaumin↗
NASA NTRS1998

The SGI/Cray T3E: Experiences and Insights

The NASA Goddard Space Flight Center is home to the fifth most powerful supercomputer in the world, a 1024 processor SGI/Cray T3E-600. The original 512 processor system was placed at Goddard in March, 1997 as part of a cooperative agreement between the High Performance Computing and Communications Program's Earth and Space Sciences Project (ESS) and SGI/Cray Research. The goal of this system is to facilitate achievement of the Project milestones of 10, 50 and 100 GFLOPS sustained performance on selected Earth and space science application codes. The additional 512 processors were purchased in March, 1998 by the NASA Earth Science Enterprise for the NASA Seasonal to Interannual Prediction Project (NSIPP). These two "halves" still operate as a single system, and must satisfy the unique requirements of both aforementioned groups, as well as guest researchers from the Earth, space, microgravity, manned space flight and aeronautics communities. Few large scalable parallel systems are configured for capability computing, so models are hard to find. This unique environment has created a challenging system administration task, and has yielded some insights into the supercomputing needs of the various NASA Enterprises, as well as insights into the strengths and weaknesses of the T3E architecture and software. The T3E is a distributed memory system in which the processing elements (PE's) are connected by a low latency, high bandwidth bidirectional 3-D torus. Due to the focus on high speed communication between PE's, the T3E requires PE's to be allocated contiguously per job. Further, jobs will only execute on the user specified number of PE's and PE timesharing is possible but impractical. With a highly varied job mix in both size and runtime of jobs, the resulting scenario is PE fragmentation and an inability to achieve near 100% utilization. SGI/Cray has provided several scheduling and configuration tools to minimize the impact of fragmentation. These tools include PScheD (the political scheduler), GRM (the global resource manager) and NQE (the Network Queuing Environment). Features and impact of these tools will be discussed, as will resulting performance and utilization data. As a distributed memory system, the T3E is designed to be programmed through explicit message passing. Consequently, certain assumptions related to code design are made by the operating system (UNICOS/mk) and its scheduling tools. With the exception of HPF, which does run on the T3E, however poorly, alternative programming styles have the potential to impact the T3E in unexpected and undesirable ways. Several examples will be presented (preceeded with the disclaimer, "Don't try this at home! Violators will be prosecuted!")

Bernard, Lisa Hamet↗
DOE OSTI2022

Adaptive elasticity policies for staging-based in situ visualization

In situ processing aims to alleviate the growing gap between computation and I/O capabilities by performing data processing close to the data source. In situ processing is widely used to process data generated by multiple data sources, including observation data from edge devices or scientific observational facilities and the simulation data generated by scientific computation on a high-performance computing (HPC) platform. For a scientific workflow that is run on an HPC platform and composed of a simulation program and an in situ data analytics or visualization (abbreviated as ana/vis) task, there is an implicit assumption that the computing resources assigned to the workflow keep static during the workflow execution. However, with the converging trend between the HPC and cloud computing platform, running the in situ ana/vis task in an elastic way is promising to decrease its overhead and improve its resource utilization rate. Resource elasticity represents the ability to change resource configurations such as the number of computing nodes/processes during workflow execution. An elastic job may dynamically adjust resource configurations; it may use a few resources at the beginning and more resources toward the end of the job when interesting data appear. However, it is hard to predict a priori how many computing nodes/processes need to be added/removed during the workflow execution to adapt to changing workflow needs. How to efficiently guide elasticity operations, such as growing or shrinking the number of processes used for in situ analysis during workflow execution, is an open-ended research question. In this article, we present adaptive elasticity policies that adopt workflow runtime information collected during workflow execution to predict how to trigger the addition/removal of processes in order to minimize in situ processing overhead. Taking in situ visualization tasks as an example, we integrate the presented elasticity policies into a staging-based elastic workflow and evaluate its efficiency in multiple elasticity scenarios. Compared with the situation without elasticity or with a static elasticity policy that uses a fixed number of processes for each rescaling operation, the adaptive elasticity policy can save overhead in finding a proper resource configuration and improve resource utilization efficiency. Furthermore, one experiment illustrates that the adaptive elasticity policy saves 41% of core-hours compared with the situation without the resource elasticity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

PDQ Users Manual. Manual Version 2, for PDQ Code Version 1.20

PDQ is a tool for the management of the input and execution of batch jobs for simulation codes that use a text based input system. It accomplishes this goal by operating at two levels. First, it takes input file templates (commonly known at LANL as input deck templates) and creates multiple instantiations by performing substitutions of data from table files into symbols (variables) found in the template. Second, it provides commands to submit the created files to the SLURM batch system for execution. These two activities taken together produce a whole that is greater than the sum of its parts and provides an elegant way of executing studies across multiple similar simulations while minimizing the risk of typographical errors in the input files. PDQ was originally developed as a job management system called XVS by Jeff McAninch while he was at LANL. Besides the capabilities described here, XVS had many other features specific for interactions with particular simulation codes. After Jeff’s departure, maintenance of XVS was taken over by Rendell Carver; he added some new features as well as kept it functioning as the batch system at LANL was changed from LSF to MOAB to SLURM. In 2017, Rob Pelak decided to develop a different version that removed the additional features (many of which were rendered obsolete with the retirement of the simulation code or batch system that they supported) and produced a cleaner “bare bones” version of XVS. A few other behaviors of XVS that Rob found irksome were altered. Rob gave the resulting code a new name: PDQ. In 2022 Danielle McDermott developed a version that runs under Python 3.X. As suggested by Rob, she used the python2to3 utility to identify most changes. Given that PDQ continues to operate with Python version 2.7 we have advanced the version number to 1.20.

97 MATHEMATICS AND COMPUTING↗