SEARCH · Engineering Papers

Results for “performance forecasting”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73

All sources NASA NTRS DOE OSTI

NASA NTRS2023

Tensor-to-scalar ratio forecasts for extended LiteBIRD frequency configurations

LiteBIRD is a planned JAXA-led cosmic microwave background (CMB) B -mode satellite experiment aiming for launch in the late 2020s, with a primary goal of detecting the imprint of primordial inflationary gravitational waves. Its current baseline focal-plane configuration includes 15 frequency bands between 40 and 402 GHz, fulfilling the mission requirements to detect the amplitude of gravitational waves with the total uncertainty on the tensor-to-scalar ratio, δr , down to δr < 0.001. A key aspect of this performance is accurate astrophysical component separation, and the ability to remove polarized thermal dust emission is particularly important. In this paper we note that the CMB frequency spectrum falls off nearly exponentially above 300 GHz relative to the thermal dust spectral energy distribution, and a relatively minor high frequency extension can therefore result in even lower uncertainties and better model reconstructions. Specifically, we compared the baseline design with five extended configurations, while varying the underlying dust modeling, in each of which the High-Frequency Telescope (HFT) frequency range was shifted logarithmically toward higher frequencies, with an upper cutoff ranging between 400 and 600 GHz. In each case, we measured the tensor-to-scalar ratio r uncertainty and bias using both parametric and minimum-variance component-separation algorithms. When the thermal dust sky model includes a spatially varying spectral index and temperature, we find that the statistical uncertainty on r after foreground cleaning may be reduced by as much as 30–50% by extending the upper limit of the frequency range from 400 to 600 GHz, with most of the improvement already gained at 500 GHz. We also note that a broader frequency range leads to higher residuals when fitting an incorrect dust model, but also it is easier to discriminate between models through higher χ 2 sensitivity. Even in the case in which the fitting procedure does not correspond to the underlying dust model in the sky, and when the highest frequency data cannot be modeled with sufficient fidelity and must be excluded from the analysis, the uncertainty on r increases by only about 5% for a 500 GHz configuration compared to the baseline.

U. Fuskeland↗

DOE OSTI2024

Machine Learning Classification Strategy to Improve Streamflow Estimates in Diverse River Basins in the Colorado River Basin

Streamflow in the Colorado River Basin (CRB) is significantly altered by human activities including land use/cover alterations, reservoir operation, irrigation, and water exports. Climate is also highly varied across the CRB which contains snowpack-dominated watersheds and arid, precipitation-dominated basins. Recently, machine learning methods have improved the generalizability and accuracy of streamflow models. Previous successes with LSTM modeling have primarily focused on unimpacted basins, and few studies have included human impacted systems in either regional or single-basin modeling. We demonstrate that the diverse hydrological behavior of river basins in the CRB are too difficult to model with a single, regional model. We propose a method to delineate catchments into categories based on the level of predictability, hydrological characteristics, and the level of human influence. Lastly, we model streamflow in each category with climate and anthropogenic proxy data sets and use feature importance methods to assess whether model performance improves with additional relevant data. Overall, land use cover data at a low temporal resolution was not sufficient to capture the irregular patterns of reservoir releases, demonstrating the importance of having high-resolution reservoir release data sets at a global scale. On the other hand, the classification approach reduced the complexity of the data and has the potential to improve streamflow forecasts in human-altered regions.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Tensor-to-scalar ratio forecasts for extended LiteBIRD frequency configurations

LiteBIRD is a planned JAXA-led cosmic microwave background (CMB) B-mode satellite experiment aiming for launch in the late 2020s, with a primary goal of detecting the imprint of primordial inflationary gravitational waves. Its current baseline focal-plane configuration includes 15 frequency bands between 40 and 402 GHz, fulfilling the mission requirements to detect the amplitude of gravitational waves with the total uncertainty on the tensor-to-scalar ratio, δr, down to δr < 0.001. A key aspect of this performance is accurate astrophysical component separation, and the ability to remove polarized thermal dust emission is particularly important. In this paper we note that the CMB frequency spectrum falls off nearly exponentially above 300 GHz relative to the thermal dust spectral energy distribution, and a relatively minor high frequency extension can therefore result in even lower uncertainties and better model reconstructions. Specifically, we compared the baseline design with five extended configurations, while varying the underlying dust modeling, in each of which the High-Frequency Telescope (HFT) frequency range was shifted logarithmically toward higher frequencies, with an upper cutoff ranging between 400 and 600 GHz. In each case, we measured the tensor-to-scalar ratio r uncertainty and bias using both parametric and minimum-variance component-separation algorithms. When the thermal dust sky model includes a spatially varying spectral index and temperature, we find that the statistical uncertainty on r after foreground cleaning may be reduced by as much as 30–50% by extending the upper limit of the frequency range from 400 to 600 GHz, with most of the improvement already gained at 500 GHz. We also note that a broader frequency range leads to higher residuals when fitting an incorrect dust model, but also it is easier to discriminate between models through higher χ 2 sensitivity. Even in the case in which the fitting procedure does not correspond to the underlying dust model in the sky, and when the highest frequency data cannot be modeled with sufficient fidelity and must be excluded from the analysis, the uncertainty on r increases by only about 5% for a 500 GHz configuration compared to the baseline.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

A trait-based framework for linking microbial communities with carbon transformations under precipitation change

Droughts are common throughout the world. As the climate changes, droughts may become more frequent and intense. Still, scientists are uncertain about how drought will affect the natural world, particularly the bacteria, fungi, and other microbes that live in soils. These tiny life forms are crucial because they control the Earth’s flows of carbon and essential nutrients. Researchers at the University of California, Irvine, and Lawrence Berkeley National Laboratory teamed up to study how the microbiome, or collection of bacteria and fungi in the soil, deals with drought. Since 2007, the researchers have used shelters with retractable roofs to prevent nearly half of normal rainfall from reaching grass and shrub ecosystems, and their soil microbiomes, in Southern California. The study team discovered that microbes have some clever tricks up their sleeve for surviving drought. When growing on dead grass as a food source, microbes ramp up production of specialized chemicals called osmolytes that keep their cells from drying out. But microbes growing on dead shrubs face another problem. Unlike grass, shrub leaves are a lousy food source. To digest and consume shrub leaves, microbes have to exude more enzymes, which act like biochemical chef knives that chop complex leaf molecules into bite-sized pieces. Carbon is the coin of the microbial realm, earned via enzyme action or slurping up dead plant juices. Microbes growing under normal conditions on tasty dead grass have it easy: they can spend most of their carbon currency on growth. With drought, life gets harder as microbes need to pay up for osmolytes and ramp down their growth. It gets worse with shrub leaves because microbes have to multi-task among growth, enzyme secretion, and osmolyte production. When drought hits, microbes on shrub leaves forgo the osmolytes, probably because losing the carbon revenue from enzyme investment would be a deal-breaker for survival. The next question tackled by the researchers asked how the genes controlling microbial lifestyles sort out across the tree of life. Most microbiologists thought these lifestyles would correspond to rather large branches on the tree. But the study team found that in fact, very closely related microbes differ in important and interesting ways. For example, bacteria that have nearly identical housekeeping genes respond distinctively to warming, rainfall, and plant chemistry. As a result, soil microbiomes are much richer in diversity than originally thought. And studying microbial diversity in much greater detail could open up many more possibilities for how microbial life deals with changing environmental conditions. Trying to understand microbial life without the fine details of genetic diversity is like trying to stream Netflix on a dial-up connection. By looking across the landscape, the researchers revealed that microbial diversity is absolutely critical for maintaining the planet’s flows of carbon and nutrients. The study team designed a new technology—microbial cages—for transplanting intact microbiomes. With the cages, the researchers could move microbiomes to novel environments and compare their ability to cycle carbon and nutrients. In some cases, performance tailed off when microbiomes found themselves in a new environment, but in other cases, performance rivaled or even exceeded that of the resident microbiome. And even the low performers eventually caught up to the native microbiomes if given sufficient time. These findings mean that microbiomes—at least in Southern California—may be resilient to climate shifts due to a high diversity of microbial lifestyles. Coping with heat and drought may literally be in their DNA. The last piece of the research puzzle, and a “Holy Grail” for microbial ecologists, is to forecast the behavior of diverse microbiomes. To meet this challenge, the study team developed new theory and computer models. For the first time, these models account for hundreds of different microbes and how their intricate lifestyles cope with drought. The models are starting to connect the tiniest microbes with the global cycles that sustain Earth’s farms, fields, and forests. With that connection, it will be easier for society to plan for a world with more droughts and other climate disruptions.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1971

Study of aircraft in intraurban transportation systems, volume 1

An analysis of an effective short range, high density computer transportation system for intraurban systems is presented. The seven county Detroit, Michigan, metropolitan area, was chosen as the scenario for the analysis. The study consisted of an analysis and forecast of the Detroit market through 1985, a parametric analysis of appropriate short haul aircraft concepts and associated ground systems, and a preliminary overall economic analysis of a simplified total system designed to evaluate the candidate vehicles and select the most promising VTOL and STOL aircraft. Data are also included on the impact of advanced technology on the system, the sensitivity of mission performance to changes in aircraft characteristics and system operations, and identification of key problem areas that may be improved by additional research. The approach, logic, and computer models used are adaptable to other intraurban or interurban areas.

Stout, E. G.↗

DOE OSTI2024

Integration of Condition-Based, Diagnostic, Prognostic, And Anomaly Detection Data into Reliability Models to Support a Predictive Maintenance Context

Reliability data employed in plant reliability models are an approximated integral representation of the past industrywide operational experience, and they neglect the present asset health status (available, for example, from online monitoring data and diagnostic assessments) and forecasted health projection (when available from prognostic models). Ideally, in a predictive maintenance context, system reliability models should support decision making by propagating actual health information from the asset to the system level in order to provide a quantitative snapshot of system health and identify the most critical assets. Asset health should be informed solely by that specific asset’s current and historical performance data and should not be an approximated integral representation of the past industrywide operational experience (as currently performed by system reliability models through Bayesian updating processes). This paper proposes a reliability modeling approach that relies on asset diagnostic and prognostic assessments, along with monitoring data to measure asset health. We show how state-of-the art condition-based, diagnostic, prognostic, and anomaly detection models can be linked to system reliability models not in probability terms, but in terms of margin where margin is defined as the “distance” between the present status and an undesired event (e.g., failure or unacceptable performance). Then, we show how the propagation of margin data from the asset to the system level is performed through classical reliability models such as fault trees or reliability block diagrams. The described method is in fact able to propagate heterogenous health data from the asset to the system level in order to analytically assess system health.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Connect the Dots: In Situ 4-D Seismic Monitoring of CO 2 Storage With Spatio-Temporal CNNs

4-D seismic imaging has been widely used in CO 2 sequestration projects to monitor the fluid flow in the volumetric subsurface region that is not sampled by wells. Ideally, real-time monitoring and near-future forecasting would provide site operators with great insights to understand the dynamics of the subsurface reservoir and assess any potential risks. However, due to obstacles such as high deployment cost, availability of acquisition equipment, exclusion zones around surface structures, only very sparse seismic imaging data can be obtained during monitoring. That leads to an unavoidable and growing knowledge gap over time. The operator needs to understand the fluid flow throughout the project lifetime and the seismic data are only available at a limited number of times. This is insufficient for understanding reservoir behavior. To overcome those challenges, we have developed spatio-temporal neural-network-based models that can produce high-fidelity interpolated or extrapolated images effectively and efficiently. Specifically, our models are built on an autoencoder, and incorporate the long short-term memory (LSTM) structure with a new loss function regularized by optical flow. We validate the performance of our models using real 4-D post-stack seismic imaging data acquired at the Sleipner CO 2 sequestration field. We employ two different strategies in evaluating our models. Numerically, we compare our models with different baseline approaches using classic pixel-based metrics. We also conduct a blind survey and collect a total of 20 responses from domain experts to evaluate the quality of data generated by our models. Finally, via both numerical and expert evaluation, we conclude that our models can produce high-quality 2-D/3-D seismic imaging data at a reasonable cost, offering the possibility of real-time monitoring or even near-future forecasting of the CO 2 storage reservoir.

4-D seismic imaging↗

DOE OSTI2021

Quantifying wind plant blockage under stable atmospheric conditions

Wind plant blockage reduces the wind velocity upstream undermining turbine performance for the first row of the plant. We assess how atmospheric stability modifies the induction zone of a wind plant in flat terrain. We also explore different approaches to quantifying the magnitude and extent of the induction zone from field-like observations. To investigate the influence from atmospheric stability, we compare simulations of two stable boundary layers using the Weather Research and Forecasting model in large-eddy simulation mode, representing wind turbines using the generalized actuator disk approach. We find a faster cooling rate at the surface, which produces a stronger stably stratified boundary layer, amplifies the induction zone of both an isolated turbine and of a large wind plant. A statistical analysis on the hub-height wind speed field shows wind slowdowns only extend far upstream (up to 15D) of a wind plant in strong stable boundary layers. To evaluate different ways of measuring wind plant blockage from field-like observations, we consider various ways of estimating the freestream velocity upstream of the plant. Sampling a large area upstream is the most accurate approach to estimating the freestream conditions, and thus of measuring the blockage effect. Also, the choice of sampling method may induce errors of the same order as the velocity deficit in the induction zone.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2003

Intelligent Launch and Range Operations Virtual Test Bed (ILRO-VTB)

Intelligent Launch and Range Operations Virtual Test Bed (ILRO-VTB) is a real-time web-based command and control, communication, and intelligent simulation environment of ground-vehicle, launch and range operation activities. ILRO-VTB consists of a variety of simulation models combined with commercial and indigenous software developments (NASA Ames). It creates a hybrid software/hardware environment suitable for testing various integrated control system components of launch and range. The dynamic interactions of the integrated simulated control systems are not well understood. Insight into such systems can only be achieved through simulation/emulation. For that reason, NASA has established a VTB where we can learn the actual control and dynamics of designs for future space programs, including testing and performance evaluation. The current implementation of the VTB simulates the operations of a sub-orbital vehicle of mission, control, ground-vehicle engineering, launch and range operations. The present development of the test bed simulates the operations of Space Shuttle Vehicle (SSV) at NASA Kennedy Space Center. The test bed supports a wide variety of shuttle missions with ancillary modeling capabilities like weather forecasting, lightning tracker, toxic gas dispersion model, debris dispersion model, telemetry, trajectory modeling, ground operations, payload models and etc. To achieve the simulations, all models are linked using Common Object Request Broker Architecture (CORBA). The test bed provides opportunities for government, universities, researchers and industries to do a real time of shuttle launch in cyber space.

Bardina, Jorge↗

NASA NTRS1988

Photochemical trajectory modelling studies of the 1987 Antarctic spring vortex

Simulations of Antarctic ozone photochemistry performed using a photochemical model integrated along air parcel trajectories are described. This type of model has a major advantage at high latitudes of being able to simulate correctly the complex interaction between photolysis and temperature fields, which, because of the polar night cannot be represented accurately in a zonally averaged framework. Isentropic air parcel trajectories were computed using Meteorological Office global model analyses and forecast fields from positions along the ER-2 flight paths during the Airborne Antarctic Ozone Experiment in Austral Spring 1987. A photochemical model is integrated along these trajectories using the aircraft observations to initialize constituent concentrations. The model includes additional reactions of the ClO dimer and also bromine reactions, which are thought to play a significant role in Antarctica. The model also includes heterogeneous reactions which are invoked when the air parcel passes through a polar stratospheric cloud (PSC). The existence of a PSC is determined throughout the course of the model integration from the parcel temperature and the saturated vapour pressure of water over an assumed H2O/HNO3 mixture. The air parcel temperature is used to determine the saturated vapor pressure of HNO3 over the same mixture. Mixing ratios which exceed saturation result in condensation of the excess in the model and hence lead to a reduction of the amount of gas phase NO2 available for chemical reaction.

Austin, J.↗

NASA NTRS1994

An Approach to Studying the Reliability of Microgravity Experiments

The identification of key factors that influence the nonsuccess of experiments conducted under microgravity conditions will aid in the planning, design, and implementation of future space shuttle experiments, as well as other microgravity experiments (i.e., experiments conducted on the space station). Similarly, knowledge of the experiments' reliability will assist in forecasting the success of forthcoming experiments. Since a relatively large number of space shuttle experiments have been conducted to date, a substantial pool of data exists for assessing the possible causes or factors which influence experiment nonsuccesses. This report details the task being undertaken at NASA Lewis Research Center (LeRC) to measure the space shuttle experiments' nonsuccess trends and identify causes that significantly affect their performance. It addresses the activities associated with correlating experiment macro-factors with experiment nonsuccesses. The development and implementation of a microgravity database to be used for tracking and correlating experiment nonsuccess factors, as well as the criteria for measuring experiment success and nonsuccess, are also discussed.

Thaggard, Michael↗

NASA NTRSDate not supplied

Orion Artemis I As Flown MMOD Analysis

Introduction The Lockheed Martin Orion spacecraft conducted the Artemis I flight around the Moon from November 16 through December 11, 2022. After the flight, an engineer from the NASA Johnson Space Center (JSC) Hypervelocity Impact Technology (HVIT) Group performed a Micrometeoroid and Orbital Debris MMOD analysis using the Bumper 3 risk assessment tool to predict the number of small impacts that would likely have occurred during the mission. Separately, a team from the same group inspected the Orion capsule for hypervelocity impact damage features. The results of the inspection were compared to those of the analysis to aid in improving the analysis, including the environment models. Scope of Work The spacecraft geometry model was created by Lockheed Martin during construction of the Artemis I Orion vehicle based on Computer Aided Design (CAD) models of the vehicle. New hypervelocity impact testing was performed to verify Ballistic Limit Equations (BLEs) used in the analysis to link impactor size and damage to the Thermal Protection System (TPS). The exact trajectory flown was recorded during the flight, including vehicle attitude. This data was used in conjunction with the ORDEM 3.2 and MEM 3 environment modeling tools to create models of particle flux impacting the spacecraft throughout the mission. Meteoroid shower forecast information was also included to account for additional particle flux associated with meteoroid showers. Inspection of the Orion capsule included the Backshell thermal tiles and the tape covering it, windows, fabric thermal materials, and small areas of other materials. Potential MMOD damage found was characterized using various techniques, including optical microscopy, computed tomography scanning, and X-ray spectroscopy. Findings The number of craters found in the Backshell tile, and their size distribution, matches well with the Bumper analysis prediction. Tape, window, and other material impacts recorded similarly align to Bumper analysis predictions. Conclusions and Recommendations This comparison of analysis with inspection of the hardware provides valuable insight into the MMOD environment and how accurately analysis tools assess the impact risk to spacecraft. As this was the first large, non-ablative returned surface from a lunar mission, this analysis extends the MMOD community’s insight beyond low Earth orbit into cis-lunar space.

MMOD↗

NASA NTRSDate not supplied

Orion Artemis I As Flown MMOD Analysis

MMOD↗

NASA NTRS1980

Estimation of within-stratum variance for sample allocation: Foreign commodity production forecasting

The problem of determining the stratum variances required for an optimum sample allocation for remotely sensed crop surveys is investigated with emphasis on an approach based on the concept of stratum variance as a function of the sampling unit size. A methodology using the existing and easily available information of historical statistics is developed for obtaining initial estimates of stratum variances. The procedure is applied to variance for wheat in the U.S. Great Plains and is evaluated based on the numerical results obtained. It is shown that the proposed technique is viable and performs satisfactorily with the use of a conservative value (smaller than the expected value) for the field size and with the use of crop statistics from the small political division level.

Chhikara, R. S.↗

DOE OSTI2023

Measurements and Model Improvement: Insight into NWP Model Error Using Doppler Lidar and Other WFIP2 Measurement Systems

Abstract Doppler-lidar wind-profile measurements at three sites were used to evaluate NWP model errors from two versions of NOAA’s 3-km-grid HRRR model, to see whether updates in the latest version 4 reduced errors when compared against the original version 1. Nested (750-m grid) versions of each were also tested to see how grid spacing affected forecast skill. The measurements were part of the field phase of the Second Wind Forecasting Improvement Project (WFIP2), an 18-month deployment into central Oregon–Washington, a major wind-energy-producing region. This study focuses on errors in simulating marine intrusions, a summertime, 600–800-m-deep, regional sea-breeze flow found to generate large errors. HRRR errors proved to be complex and site dependent. The most prominent error resulted from a premature drop in modeled marine-intrusion wind speeds after local midnight, when lidar-measured winds of greater than 8 m s −1 persisted through the next morning. These large negative errors were offset at low levels by positive errors due to excessive mixing, complicating the interpretation of model “improvement,” such that the updates to the full-scale versions produced mixed results, sometimes enhancing but sometimes degrading model skill. Nesting consistently improved model performance, with version 1’s nest producing the smallest errors overall. HRRR’s ability to represent the stages of sea-breeze forcing was evaluated using radiation budget, surface-energy balance, and near-surface temperature measurements available during WFIP2. The significant site-to-site differences in model error and the complex nature of these errors mean that field-measurement campaigns having dense arrays of profiling sensors are necessary to properly diagnose and characterize model errors, as part of a systematic approach to NWP model improvement. Significance Statement Dramatic increases in NWP model skill will be required over the coming decades. This paper describes the role of major deployments of accurate profiling sensors in achieving that goal and presents an example from the Second Wind Forecast Improvement Program (WFIP2). Wind-profile data from scanning Doppler lidars were used to evaluate two versions of HRRR, the original and an updated version, and nested versions of each. This study focuses on the ability of updated HRRR versions to improve upon predicting a regional sea-breeze flow, which was found to generate large errors by the original HRRR. Updates to the full-scale HRRR versions produced mixed results, but the finer-mesh versions consistently reduced model errors.

Meteorology & Atmospheric Sciences↗

DOE OSTI2020

Integrating Predictions for Improving Defect Classification Accuracy in NDT-based Assessment of Concrete - 20229

There is an increasing need to create predictive models for defect classification in concrete using the output of non-destructive testing (NDT) techniques. Recent advancement of machine-learning algorithms has offered several techniques for developing classification models for different types of data sets. However, the performance of these algorithms is very uncertain, mainly when applied to small and noisy datasets. For example, when human access is limited (e.g., nuclear facility), robot-based NDT is preferred. But compared to manual tests with humans present on site, the data sets are small, and more noise can exist. Therefore, it is imperative to develop new approaches to ensure a consistently high classification accuracy for inadequate data sets. This study explores the classification performance on NDT dataset using classifiers from different machine-learning algorithms, namely k-Nearest Neighbor (kNN), Decision Tree, Naive Bayes, Logistic Regression, and Support Vector Machine (SVM). The authors further integrated the predictions from these classifiers using proposed methods. The integration strategy combines the output of the classifiers based on two different measures, accuracy, and performance (ACC and PERF), using equations such as sum, average, and square-root-of-sums-of-squares (SRSS). Our results reveal varying classification accuracies across individual classifiers with different misclassifications across the test data set. The integration strategy provided significant improvement in the classification accuracy compared to the individual classifiers. Furthermore, the results indicate minimal variation across the integration methods as compared to the variation across the individual classifiers. To conclude, prediction integration offers a unique approach for combining the output of multiple classifiers to create redundancies with the potential of achieving high classification performance and improved reliability in predictive models for defect detection in concrete. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

Design considerations for a Digital Twin built to improve nitrification performance at a water resource recovery facility

A Digital Twin built around Activated Sludge Model No. 1 was deployed at a full-scale water resource recovery facility. Its design included a waste rate recommender system based on automatic scenario analyses, where influent loads and waste rates are varied to determine their impact on nitrification. At the same frequency as these scenario analyses, scheduled auto-calibrations allow for nitrifier maximum specific growth rate (μmax-NITO) soft sensing, the only kinetic parameter shown to require adjustment if the objective is aeration tank effluent ammonia forecasting accuracy. By integrating temperature forecasting over the next three sludge ages, this Digital Twin approach creates opportunities for advancing waste rate decisions in anticipation of seasonal temperature changes, optimizing ammonia control authority under varying influent loads, and furnishing valuable insights for future capital projects requiring nitrifier kinetic understanding and modelling.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Operational Analytics Studies for ATLAS Distributed Computing: Data Popularity Forecast and Utilization of the WLCG Centers

Operational analytics is the direction of research related to the analysis of the current state of computing processes and the prediction of future states in order to anticipate imbalances and take timely measures to stabilize a complex system. There are two relevant areas in ATLAS Distributed Computing that are currently the focus of studies: user physics analysis including the forecast of popularity of data samples among users, and evaluating WLCG centers for their readiness to process user analysis payloads. Studying these areas is challenging due to the complexity involved, as it requires a comprehensive understanding of numerous boundary conditions typically found in large-scale distributed computing infrastructures. Forecasts of data popularity are problematic without the categorization of user tasks by their types (data transformation or physics analysis), which do not always appear on the surface but may induce noise, which introduces significant distortions for predictive analysis. Evaluating the WLCG resources by their analysis workloads is also a challenging task as it is necessary to find a balance between the workload of the resource, its performance, the waiting time for jobs on it, as well as the volume of jobs that it processes. This is especially difficult in a heterogeneous computing environment, where legacy resources are used along with modern high-performance machines. We will look at these areas of research in detail and discuss what tools and methods are used in our work, demonstrating results already obtained.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗