SEARCH · Engineering Papers

Results for “distributed scheduling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Frontier Job-Centric Telemetry Dataset

Comprehensive analysis of high-performance computing (HPC) systems requires linking workload execution to system behavior. This kind of analysis is vital for diagnosing performance issues, managing capacity, detecting anomalous workloads, and understanding how applications interact with system hardware. This job-centric telemetry dataset unifies scheduler job records with node-level measurements, enabling direct association between workloads and their corresponding power, thermal, and performance characteristics. It contains sanitized, scheduler related metadata for 152,400 individual jobs that ran on the Frontier supercomputer and ended on selected days throughout 2024 and 2025, a subpopulation of ~6.8% of the total number of allocated jobs with non-zero run time on the system over that same period. Each is linked with files that contain telemetry time series records of the power utilization and temperature behavior of its allocated nodes and their processors during the run time of the job. Where available, a portion of the job files also contain network performance time series. Jobs are sampled from select days that reflect normal levels of user activity and possess job size distributions with large numbers of leadership class jobs (>20% of Frontier nodes). Jobs in this dataset attempt to best represent successful user workflows.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2021

Asynchronous distributed-memory task-parallel algorithm for compressible flows on unstructured 3D Eulerian grids

Here, we discuss the implementation of a finite element method, used to numerically solve the Euler equations of compressible flows, using an asynchronous runtime system (RTS). The algorithm is implemented for distributed-memory machines, using stationary unstructured 3D meshes, combining data-, and task-parallelism on top of the Charm++ RTS. Charm++’s execution model is asynchronous by default, allowing arbitrary overlap of computation and communication. Task-parallelism allows scheduling parts of an algorithm independently of, or dependent on, each other. Built-in automatic load balancing enables continuous redistribution of computational load by migration of work units based on real-time CPU load measurement. The RTS also features automatic checkpointing, fault tolerance, resilience against hardware failure, and supports power-, and energy-aware computation. We demonstrate scalability up to 25 x 10 9 cells at $\mathscr{O}$10 4 compute cores and the benefits of automatic load balancing for irregular workloads. The full source code with documentation is available at https://quinoacomputing.org.

42 ENGINEERING↗

DOE OSTI2022

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2001

Method for resource control in parallel environments using program organization and run-time support

A system and method for dynamic scheduling and allocation of resources to parallel applications during the course of their execution. By establishing well-defined interactions between an executing job and the parallel system, the system and method support dynamic reconfiguration of processor partitions, dynamic distribution and redistribution of data, communication among cooperating applications, and various other monitoring actions. The interactions occur only at specific points in the execution of the program where the aforementioned operations can be performed efficiently.

Ekanadham, Kattamuri↗

NASA NTRS1999

Method for resource control in parallel environments using program organization and run-time support

Ekanadham, Kattamuri↗

NASA NTRSDate not supplied

Planning Bias: Planning as a Source of Sampling Bias

Many data-driven planning methods are trained on data generated by planners. It is well known that many statistical learning methods are sensitive to sampling bias, and yet there has been little or no attention to planning as a sampling method and its role in introducing sampling bias into planner-generated training data. Recently, it has been demonstrated that A**,* in the presence of problems with variable heuristic error, prefers some solutions over other equally cost-optimal solutions. But, as we discuss in this paper, mitigation may not be as simple as resolving arbitrary tie-breaking by sampling from ties uniformly at random. In this paper, we formalize an intuition of planning bias. We focus on problems which output a single solution. Diverse planning only complicates the problem by generalizing it to bias in the set of sets; we show how it is subject to bias in the single solution. We make some useful observations about deterministic algorithms in contrast to non-deterministic algorithms. We explain how information entropy may be a good way to measure planning bias, and discuss some issues in evaluating practical approaches to measurement. We address the intuition that uniform random tiebreaking should mitigate bias; and sketch a novel approach to constructing an appropriate random distribution for duplicate detection during forward search for unbiased A*. Finally, we suggest directions for future work.

Planning Scheduling Algorithms↗

DOE OSTI2022

Dynamic Risk-Aware Patch Scheduling

A program that defines and assesses the dynamic risk of software vulnerabilities and considers the dynamic risks into patch scheduling to reduce security risks posed by vulnerabilities and provide formal guidance to security operations at various organizations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

Preparing Distribution Utilities for the Future - Unlocking Demand-Side Management Potential: A Novel Analytical Framework

The balance of supply and demand in the power systems has traditionally been served solely through generation and network capacity planning and operations. However, with increased requirements for flexibility due to the uptake in variable renewable generation sources such as wind and solar there is a need to increased demand-side flexibility. In addition, there are increased communications and flexibility capabilities emerging on the demand-side from the adoption of advanced metering infrastructures and smart meter deployment and intelligent loads such as smart thermostats and schedulable white goods (e.g. dishwashers and washing machines). Unlocking demand-side flexibility can bring system benefits from peak load reduction bringing about generation capacity and network upgrade deferral, to reducing demand and more efficient utilization of generation and network capacity. Unlocking demand-side flexibility is an evolving process for utilities and solutions must be tailored to each specific customer group. Demand-side management (DSM) is a broad set of tools that can include demand response (both dispatchable and non-dispatchable), energy efficiency and distributed energy resources and demand-side technologies. The National Renewable Energy Laboratory (NREL), in collaboration with BSES Rajdhani Power Ltd. (BRPL) and Deloitte, examined the potential of DSM in BRPL’s service territory, developing detailed information on customer classes and willingness to participate in DSM. The study developed modeling frameworks for load analysis and the analysis tools to assess the potential of time-of-use tariffs in motivating customers to reduce their peak period energy consumption. The study shows that BRPL customers, specifically their domestic customers, are willing to participate in DSM programs and that time-of-use pricing can help BRPL reduce their peak demand and help unlock demand-side flexibility.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1999

Backscatter Modeling at 2.1 Micron Wavelength for Space-Based and Airborne Lidars Using Aerosol Physico-Chemical and Lidar Datasets

Space-based and airborne coherent Doppler lidars designed for measuring global tropospheric wind profiles in cloud-free air rely on backscatter, beta from aerosols acting as passive wind tracers. Aerosol beta distribution in the vertical can vary over as much as 5-6 orders of magnitude. Thus, the design of a wave length-specific, space-borne or airborne lidar must account for the magnitude of 8 in the region or features of interest. The SPAce Readiness Coherent Lidar Experiment under development by the National Aeronautics and Space Administration (NASA) and scheduled for launch on the Space Shuttle in 2001, will demonstrate wind measurements from space using a solid-state 2 micrometer coherent Doppler lidar. Consequently, there is a critical need to understand variability of aerosol beta at 2.1 micrometers, to evaluate signal detection under varying aerosol loading conditions. Although few direct measurements of beta at 2.1 micrometers exist, extensive datasets, including climatologies in widely-separated locations, do exist for other wavelengths based on CO2 and Nd:YAG lidars. Datasets also exist for the associated microphysical and chemical properties. An example of a multi-parametric dataset is that of the NASA GLObal Backscatter Experiment (GLOBE) in 1990 in which aerosol chemistry and size distributions were measured concurrently with multi-wavelength lidar backscatter observations. More recently, continuous-wave (CW) lidar backscatter measurements at mid-infrared wavelengths have been made during the Multicenter Airborne Coherent Atmospheric Wind Sensor (MACAWS) experiment in 1995. Using Lorenz-Mie theory, these datasets have been used to develop a method to convert lidar backscatter to the 2.1 micrometer wavelength. This paper presents comparison of modeled backscatter at wavelengths for which backscatter measurements exist including converted beta (sub 2.1).

Srivastava, V.↗

NASA NTRS1999

BIO-Plex Information System Concept

This paper describes a suggested design for an integrated information system for the proposed BIO-Plex (Bioregenerative Planetary Life Support Systems Test Complex) at Johnson Space Center (JSC), including distributed control systems, central control, networks, database servers, personal computers and workstations, applications software, and external communications. The system will have an open commercial computing and networking, architecture. The network will provide automatic real-time transfer of information to database server computers which perform data collection and validation. This information system will support integrated, data sharing applications for everything, from system alarms to management summaries. Most existing complex process control systems have information gaps between the different real time subsystems, between these subsystems and central controller, between the central controller and system level planning and analysis application software, and between the system level applications and management overview reporting. An integrated information system is vitally necessary as the basis for the integration of planning, scheduling, modeling, monitoring, and control, which will allow improved monitoring and control based on timely, accurate and complete data. Data describing the system configuration and the real time processes can be collected, checked and reconciled, analyzed and stored in database servers that can be accessed by all applications. The required technology is available. The only opportunity to design a distributed, nonredundant, integrated system is before it is built. Retrofit is extremely difficult and costly.

Jones, Harry↗

DOE OSTI2021

Monitoring of corrosion effects in pipes with multi-mode acoustic signals

Current commercial long-range guided wave inspection and monitoring technologies for pipes usually require establishment of certain specific acoustic modes for wave propagation and use a large number of sensors in a certain geometric arrangement (for example, uniformly-spaced arrays of sensors on a collar, placed around a pipe). This paper, in contrast, presents a long-range corrosion detection method in pipes based on multi-mode guided acoustic waves, generated and received using simple, single-element sensors. In this technique, available guided wave modes in any interrogated zone of the structure are automatically down-selected by the medium, and changes in energy distribution over the selected modes are analyzed through signal processing to monitor material loss or addition. Here, to investigate the feasibility of the proposed method, we have performed both experimental and numerical simulation studies on 50.8-mm (2-in)-diameter schedule-40 carbon steel pipes. In these studies, a linear chirp signal is injected in compression mode, perpendicular to the surface of the pipe, generating multi-mode, multi-directional guided waves that travel within the pipe wall. Corrosion effects in the pipes are simulated numerically and experimentally. To reduce errors in the measurement technique due to changes in the local or global temperature of the measured media, a universal temperature compensation algorithm is introduced. The experimental results show good agreement with simulation results, indicating that the proposed multi-mode wave generation and detection in a pipe could be used for long-range detection and monitoring of corrosion effects, such as pipe wall losses or solid material deposits.

36 MATERIALS SCIENCE↗

NASA NTRS1995

Job Management Requirements for NAS Parallel Systems and Clusters

A job management system is a critical component of a production supercomputing environment, permitting oversubscribed resources to be shared fairly and efficiently. Job management systems that were originally designed for traditional vector supercomputers are not appropriate for the distributed-memory parallel supercomputers that are becoming increasingly important in the high performance computing industry. Newer job management systems offer new functionality but do not solve fundamental problems. We address some of the main issues in resource allocation and job scheduling we have encountered on two parallel computers - a 160-node IBM SP2 and a cluster of 20 high performance workstations located at the Numerical Aerodynamic Simulation facility. We describe the requirements for resource allocation and job management that are necessary to provide a production supercomputing environment on these machines, prioritizing according to difficulty and importance, and advocating a return to fundamental issues.

Saphir, William↗

NASA NTRS2001

ASCA Observations of the Dwarf Novae SS Cyg and VW Hyi And Observations of the Precessing Disk Cataclysmic Variable TV Col

The observations for both SS Cyg and VW Hyi were to be scheduled as Targets of Opportunity jointly with other satellites. The VW Hyi observation was obtained jointly with EUVE during a superoutburst. The XTE data were initially processed, revealing no detection. However, the XTE team improved the instrumental background model and distributed it in July 1998. A further improvement was made in August 1999. The improved models allow a better background subtraction, thereby detecting previously un-detected sources.

Schlegel, E.↗

NASA NTRS2004

In Situ Water Vapor Measurements Using Coupled UV Fragment Fluorescence/Absorption Spectroscopy in Support of CRYSTAL-FACE

Understanding the coupling of dynamics, chemistry, and radiation within the context of the NASA Earth Science Enterprise (ESE) and the national Climate Change Science Program (CCSP) requires, as a first-order priority, high spatial resolution, high-accuracy observations of water in its various phases. Given the powerful diagnostic importance of the condensed phases of water for dynamics and the impact of phase changes in water on the radiation field, the accurate, in situ observation of water vapor is of central importance to CRYSTAL FACE (CF). This is clear both from the defined scientific objectives of the NRA and from developments in the coupled fields of stratosphere/troposphere exchange, cirrus cloud formation/removal and mechanisms for the distribution of water vapor in the middle/upper troposphere. Accordingly, we were funded under NASA Grant NAG5-11548 to perform the following tasks for the CF mission: 1. Prepare the water vapor instrument for integration into the WB57F and test flights scheduled for Spring 2002. 2. Calibrate and prepare the water vapor instrument for the Summer 2002 CF science flights based in Jacksonville, Florida. 3. Provide both science and engineering support for the above-mentioned efforts. 4. Analyze and interpret the CF data in collaboration with other mission scientists. 5. Attend the science workshop in Spring 2003. 6. Publish the data and analysis in peer-reviewed journals.

Anderson, James G.↗

NASA NTRS2008

Microwave Radiometers from 0.6 to 22 GHz for Juno, a Polar Orbiter around Jupiter

A compact radiometer instrument is under development at JPL for Juno, the next NASA New Frontiers mission, scheduled to launch in 2011. This instrument is called the MWR (MicroWave Radiometer), and its purpose is to measure the thermal emission from Jupiter's atmosphere at selected frequencies from 0.6 to 22 GHz. The objective is to measure the distributions and abundances of water and ammonia in Jupiter's atmosphere, with the goal of understanding the previously unobserved dynamics of the subcloud atmosphere, and to discriminate among models for planetary formation in our solar system. The MWR instrument is currently being developed to address these science questions for the Juno mission. As part of a deep space mission aboard a solar-powered spacecraft, MWR is designed to be compact, lightweight, and low power. The entire MWR instrument consists of six individual radiometer channels with approximately 4% bandwidth at 0.6, 1.25,2.6,5.2, 10,22 GHz operating in direct detection mode. Each radiometer channel has up to 80 dB of gain with a noise figure of several dB. The highest frequency channel uses a corrugated feedhorn and waveguide transmission lines, whereas all other channels use highly phase stable coaxial cables and either patch array or waveguide slot array antennas. Slot waveguide array antennas were chosen for the low loss at the next three highest frequencies and patch array antennas were implemented due to the mass constraint at the two lowest frequencies. The six radiometer channels receive their voltage supplies and control lines from an electronics unit that also provides the instrument communication interface to the Juno spacecraft. For calibration purposes each receiver has integrated noise diodes, a Dicke switch, and temperature sensors near each component that contributes to the noise figure. In addition, multiple sensors will be placed along the RF transmission lines and the antennas in order to measure temperature gradients. All antennas and RF transmission lines must withstand low temperatures and the harsh radiation environment surrounding Jupiter; the receivers and control electronics are protected by a radiation-shielding enclosure on the Juno spacecraft that also provides for a benign and stable operating temperature environment. This paper will focus on the concept of the MWR instrument and will present results of one breadboard receiver channel.

receiver↗

DOE OSTI2021

Optimizing $\mathrm{ACRT}$ to reduce inclusion formation during the $\mathrm{VGF}$ growth of cadmium zinc telluride: II. Application to experiments

Here two different accelerated crucible rotation technique (ACRT) rotation schedules are assessed, via theoretical computations and growth experiments, according to their ability to reduce tellurium-rich inclusions during the vertical gradient freeze (VGF) of cadmium zinc telluride (CZT). A vigorous rotation schedule based on classical ACRT guidelines produces a well-mixed melt that is expected to reduce tellurium inclusions compared to growth without crucible rotation. In contrast, a new ACRT schedule, derived from model-based optimization for interface stability, employs slower rotation with longer periods and is predicted to further reduce inclusions. Growth experiments corroborate these expectations. Namely, both ACRT schedules result in crystals with inclusion size and volume significantly decreased from levels found in material grown with no rotation, and material grown using the computationally optimized ACRT parameters exhibits a median inclusion size that is smaller and with a sharper distribution than in material grown via classical ACRT.

36 MATERIALS SCIENCE↗

NASA NTRS1988

Space Station Photovoltaic power modules

Silicon cell Photovoltaic (PV) power modules are key components of the Space Station Electrical Power System (EPS) scheduled to begin deployment in 1994. Four PV power modules, providing 75 KWe of user ac power, form the cornerstone of the EPS; which is comprised of Photovoltaic (PV) power modules, Solar Dynamic (SD) power modules, and the Power Management and Distribution (PMAD) system. The PV modules are located on rotating outboard sections of the Space Station (SS) structure and each module incorporates its own nickel-hydrogen energy storage batteries, its own thermal control system, and some autonomous control features. The PV modules are a cost-effective and technologically mature approach for providing reliable SS electrical power and are a solid base for EPS growth, which is expected to reach 300 KWe by the end of the Space Station's 30-year design lifetime.

Tatro, Charles A.↗

NASA NTRS2001

Computational Analysis of the Combustion Processes in an Axisymmetric, RBCC Flowpath

Computational fluid dynamic simulations have been used to study the combustion processes within an axisymmetric, RBCC flowpath. Two distinct operating modes have been analyzed to date, including the independent ramjet stream (IRS) cycle and the supersonic combustion ramjet (scramJet) cycle. The IRS cycle investigation examined the influence of fuel-air ratio, fuel distribution, and rocket chamber pressure upon the combustion physics and thermal choke characteristics. Results indicate that adjustment of the amount and radial distribution of fuel can control the thermal choke point. The secondary massflow rate was very sensitive to the fuel-air ratio and the rocket chamber pressure. The scramjet investigation examined the influence of fuel-air ratio and fuel injection schedule upon combustion performance estimates. An analysis of the mesh-dependence of these calculations was presented. Jet penetration data was extracted from the three-dimensional simulations and compared favorably with experimental correlations of similar flows. Results indicate that combustion efficiency was very sensitive to the fuel schedule.

Steffen, Christopher J., Jr.↗