Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83

The Suomi National Polar-Orbiting Partnership (SNPP): Continuing NASA Research and Applications

The Suomi National Polar-orbiting Partnership (SNPP) satellite was successfully launched into a polar orbit on October 28, 2011 carrying 5 remote sensing instruments designed to provide data to improve weather forecasts and to increase understanding of long-term climate change. SNPP provides operational continuity of satellite-based observations for NOAA's Polar-orbiting Operational Environmental Satellites (POES) and continues the long-term record of climate quality observations established by NASA's Earth Observing System (EOS) satellites. In the 2003 to 2011 pre-launch timeframe, NASA's SNPP Science Team assessed the adequacy of the operational Raw Data Records (RDRs), Sensor Data Records (SDRs), and Environmental Data Records (EDRs) from the SNPP instruments for use in NASA Earth Science research, examined the operational algorithms used to produce those data records, and proposed a path forward for the production of climate quality products from SNPP. In order to perform these tasks, a distributed data system, the NASA Science Data Segment (SDS), ingested RDRs, SDRs, and EDRs from the NOAA Archive and Distribution and Interface Data Processing Segments, ADS and IDPS, respectively. The SDS also obtained operational algorithms for evaluation purposes from the NOAA Government Resource for Algorithm Verification, Independent Testing and Evaluation (GRAVITE). Within the NASA SDS, five Product Evaluation and Test Elements (PEATEs) received, ingested, and stored data and performed NASA's data processing, evaluation, and analysis activities. The distributed nature of this data distribution system was established by physically housing each PEATE within one of five Climate Analysis Research Systems (CARS) located at either at a NASA or a university institution. The CARS were organized around 5 key EDRs directly in support of the following NASA Earth Science focus areas: atmospheric sounding, ocean, land, ozone, and atmospheric composition products. The PEATES provided the system level interface with members of the NASA SNPP Science Team and other science investigators within each CARS. A sixth Earth Radiation Budget CARS was established at NASA Langley Research Center (NASA LaRC) to support instrument performance, data evaluation, and analysis for the SNPP Clouds and the Earth's Radiant Budget Energy System (CERES) instrument. Following the 2011 launch of SNPP, spacecraft commissioning, and instrument activation, the NASA SNPP Science Team evaluated the operational RDRs, SDRs, and EDRs produced by the NOAA ADS and IDPS. A key part in that evaluation was the NASA Science Team's independent processing of operational RDRs and SDRs to EDRs using the latest NASA science algorithms. The NASA science evaluation was completed in the December 2012 to April 2014 timeframe with the release of a series of NASA Science Team Discipline Reports. In summary, these reports indicated that the RDRs produced by the SNPP instruments were of sufficiently high quality to be used to create data products suitable for NASA Earth System science and applications. However, the quality of the SDRs and EDRs were found to vary greatly when considering suitability for NASA science. The need for improvements in operational algorithms, adoption of different algorithmic approaches, greater monitoring of on-orbit instrument calibration, greater attention to data product validation, and data reprocessing were prominent findings in the reports. In response to these findings, NASA, in late 2013, directed the NASA SNPP Science Team to use SNPP instrument data to develop data products of sufficiently high quality to enable the continuation of EOS time series data records and to develop innovative, practical applications of SNPP data. This direction necessitated a transition of the SDS data system from its pre-launch assessment mode to one of full data processing and production. To do this, the PEATES, which served as NASA's data product testing environment during the prelaunch and early on-orbit periods, were transitioned to Science Investigator-led Processing Systems (SIPS). The distributed data architecture was maintained in this new system by locating the SIPS at the same institutions at which the CARS and PEATES were located. The SIPS acquire raw SNPP instrument Level 0 (i.e. RDR) data over the full SNPP mission from the NOAA ADS and IDPS through the NASA SDS Data Distribution and Depository Element (SD3E). The SIPS process those data into NASA Level 1, Level 2, and global, gridded Level 3 standard products using peer-reviewed algorithms provided by members of the NASA Science Team. The SIPS work with the NASA SNPP Science Team in obtaining enhanced, refined, or alternate real-time algorithms to support the capabilities of the Direct Readout Laboratory (DRL). All data products, algorithm source codes, coefficients, and auxiliary data used in product generation are archived in an assigned NASA Distributed Active Archive Center (DAAC).

remote sensing↗

Trade-off analysis of modes of data handling for earth resources (ERS), volume 1

Data handling requirements are reviewed for earth observation missions along with likely technology advances. Parametric techniques for synthesizing potential systems are developed. Major tasks include: (1) review of the sensors under development and extensions of or improvements in these sensors; (2) development of mission models for missions spanning land, ocean, and atmosphere observations; (3) summary of data handling requirements including the frequency of coverage, timeliness of dissemination, and geographic relationships between points of collection and points of dissemination; (4) review of data routing to establish ways of getting data from the collection point to the user; (5) on-board data processing; (6) communications link; and (7) ground data processing. A detailed synthesis of three specific missions is included.

Source record↗

Cloud Algorithm Design and Performance for the 2002 Geoscience Laser Altimeter System Mission

A satellite borne lidar instrument, the Geoscience Laser Altimeter System (GLAS), is to be launched in late 2002 and will provide continuous profiling of atmospheric clouds and aerosol on a global basis. Data processing algorithms have been developed to provide operational data products in near real time. Basic data products for cloud observations are the height of the top and bottom of single to multiple cloud layers and the lidar calibrated observed backscatter cross section up to the level of signal attenuation. In addition the optical depth and vertical profile of visible extinction cross section of many transmissive cloud layers and most haze layers are to be derived. The optical thickness is derivable in some cases from the attenuation of the molecular scattering below cloud base. In other cases an assumption of the scattering phase function is required. In both cases a estimated correction for multiple scattering is required. The data processing algorithms have been tested in part from aircraft measurements used to simulated satellite data. The GLAS lidar observations will be made from an orbit that will allow inter comparison with all other existing satellite cloud measurements.

Spinhirne, J. D.↗

Twist Model Development and Results From the Active Aeroelastic Wing F/A-18 Aircraft

Understanding the wing twist of the active aeroelastic wing F/A-18 aircraft is a fundamental research objective for the program and offers numerous benefits. In order to clearly understand the wing flexibility characteristics, a model was created to predict real-time wing twist. A reliable twist model allows the prediction of twist for flight simulation, provides insight into aircraft performance uncertainties, and assists with computational fluid dynamic and aeroelastic issues. The left wing of the aircraft was heavily instrumented during the first phase of the active aeroelastic wing program allowing deflection data collection. Traditional data processing steps were taken to reduce flight data, and twist predictions were made using linear regression techniques. The model predictions determined a consistent linear relationship between the measured twist and aircraft parameters, such as surface positions and aircraft state variables. Error in the original model was reduced in some cases by using a dynamic pressure-based assumption and by using neural networks. These techniques produced excellent predictions for flight between the standard test points and accounted for nonlinearities in the data. This report discusses data processing techniques and twist prediction validation, and provides illustrative and quantitative results.

Lizotte, Andrew↗

Twist Model Development and Results from the Active Aeroelastic Wing F/A-18 Aircraft

Understanding the wing twist of the active aeroelastic wing (AAW) F/A-18 aircraft is a fundamental research objective for the program and offers numerous benefits. In order to clearly understand the wing flexibility characteristics, a model was created to predict real-time wing twist. A reliable twist model allows the prediction of twist for flight simulation, provides insight into aircraft performance uncertainties, and assists with computational fluid dynamic and aeroelastic issues. The left wing of the aircraft was heavily instrumented during the first phase of the active aeroelastic wing program allowing deflection data collection. Traditional data processing steps were taken to reduce flight data, and twist predictions were made using linear regression techniques. The model predictions determined a consistent linear relationship between the measured twist and aircraft parameters, such as surface positions and aircraft state variables. Error in the original model was reduced in some cases by using a dynamic pressure-based assumption. This technique produced excellent predictions for flight between the standard test points and accounted for nonlinearities in the data. This report discusses data processing techniques and twist prediction validation, and provides illustrative and quantitative results.

Lizotte, Andrew M.↗

Architectural Improvements and New Processing Tools for the Open XAL Online Model

The online model is the component of Open XAL providing accelerator modeling, simulation, and dynamic synchronization to live hardware. Significant architectural changes and feature additions have been recently made in two separate areas: 1) the managing and processing of simulation data, and 2) the modeling of RF cavities. Simulation data and data processing have been completely decoupled. A single class manages all simulation data while standard tools were developed for processing the simulation results. RF accelerating cavities are now modeled as composite structures where parameter and dynamics computations are distributed. The beam and hardware models both maintain their relative phase information, which allows for dynamic phase slip and elapsed time computation.

Accelerator Control↗

Landscape-scale Characterization of Arctic Tundra Vegetation Composition, Structure, and Function with a Multi-sensor Unoccupied Aerial System: Supporting Data.

Airborne remote sensing data collected using the Brookhaven National Laboratory's (BNL) heavy-lift unoccupied aerial system (UAS) octocopter platform, the Osprey, operated by the Terrestrial Ecosystem Science and Technology (TEST) group (https://www.bnl.gov/testgroup). This package includes data from three flights flown over the NGEE-Arctic Council, Kougarok and Teller sites in July, 2018. The Osprey is a multi-sensor UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) and thermal infrared (TIR) surface 'skin' temperature imagery, as well as surface reflectance at 1 nm intervals in the visible to near-infrared spectral range 350 - 1000 nm measured at regular intervals along each flight path. Derived image products include ortho-mosaiced RGB and TIR images, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, digital terrain model (DTM), and a canopy height model (CHM). In addition, a VNIR surface reflectance file is provided for the trigger locations collected during each flight campaign. Ancillary aircraft data, flight mission parameters, and general flight conditions provided by the onboard flight and data collection computers are also included. Unprocessed and processed data products are included in this package (processing levels 0-3). Data and metadata are provided as text (*.txt, *.json, *.kml, *hdr, *.enp), tabular (*.dat, *.csv, *.waypoint, ENVI format (no extension)), point cloud (*.laz) and image (*.jpg, *.tif, *png) formats. This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy?s Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Scanning structural mapping at the Life Science X-ray Scattering Beamline

This work describes the instrumentation and software for microbeam scattering and structural mapping at the Life Science X-ray Scattering (LiX) beamline at NSLS-II. Using a two-stage focusing scheme, an adjustable beam size between a few micrometres and a fraction of a millimetre is produced at the sample position. Scattering data at small and wide angles are collected simultaneously on multiple Pilatus detectors. A recent addition of an in-vacuum Pilatus 900k detector, with the detector modules arranged in a C-shaped configuration, has improved the azimuthal angle coverage in the wide-angle data. As an option, fluorescence data can be collected simultaneously. Fly scans have been implemented to minimize the time interval between scattering patterns and to avoid unnecessary radiation damage to the sample. For weakly scattering samples, an in-vacuum sample environment has been developed here to minimize background scattering. Data processing for these measurements is highly sample-specific. To establish a generalized data process workflow, first the data are reduced to reciprocal coordinates at the time of data collection. The users can then quantify features of their choosing from these intermediate data and construct structural maps. As examples, results from in-vacuum mapping of onion epidermal cell walls and 2D tomographic sectioning of an intact poplar stem are presented.

36 MATERIALS SCIENCE↗

The Role of Remotely Sensed and Relayed Data in the Delaware River Basin

The planned integration of the existing water quality monitoring and data processing systems in the Delaware River Basin with a data relay experiment proposed for the ERTS-1 is discussed. The experiment is designed to use ERTS-1 as a data relay link for a maximum of 20 hydrologic stations in the basin, including stream gaging, reservoir level, ground water level, and water quality monitoring stations. This experiment has the potential for reducing the time lag between data collection and dissemination to less than 12 hours. The experiment will also provide impetus to develop an operational system of real time data processing and dissemination to handle the large quantity of data that will be obtained from the stations in the basin. The results of this experiment will demonstrate the relative merits of satellite relay of data versus conventional means of data telemetry and will provide a basis for the development of operational satellite relay of hydrologic data.

Paulson, R. W.↗

Workflow-based automatic processing for Internet of Floating Things crowdsourced data

Data from sensors incorporated into mobile devices, such as networked navigational sensors, can be used to capture detailed environmental information. We describe here a workflow and framework for using sensors on boats to construct unique new datasets of underwater topography (bathymetry). Starting with a large number of measurements of position, depth, etc., obtained from such an Internet of Floating Things, we illustrate how, with a specialized protocol, data can be communicated to cloud resources, even when using delayed, intermittent, or disconnected networks. We then propose a method for automatic sensor calibration based on a novel reputation approach. Sampled depth data are interpolated efficiently on a cloud computing platform in order to provide a continuously updated bathymetric database. Our prototype implementation uses the FACE-IT Galaxy workflow engine to manage network communication and exploits the computational power of GPGPUs in a virtualized cloud environment, working with a CUDA-parallel algorithm, for efficient data processing. We report on an initial evaluation involving data from a sailing vessel in Italian coastal waters.

97 MATHEMATICS AND COMPUTING↗

2017 Western Passage Tidal Energy Resource Characterization Measurements

These data are from tidal resource characterization measurements collected between April and July 2017 in Western Passage near Eastport, Maine, USA. The dataset contains the following four sub-datasets, each of which is described in more detail in the README.pdf. 1. A bottom-mounted Teledyne RDI Workhorse 600 kHz acoustic Doppler current profiler (ADCP) was deployed at 44.92015 N, 66.98915 W in ~50 m of water from 3 April to 18 July (106 days). Data were recorded in 6-minute increments in the ENU (East, magnetic North, Up) coordinate system with bin-mapping enabled. 2. A bottom-mounted Nortek Signature 500 kHz ADCP was deployed at 44.92192 N, 66.98913 W in ~50 m of water from 4 April to 18 July (105 days). Data were sampled and recorded at 2 Hz and recorded in the ENU (East, magnetic North, Up) coordinate system. 3. Between those stations along a cross-channel transect, a Stable Tidal Turbulence Mooring (STTM) positioned ~10 m above the seabed was deployed for one week during a spring tide. The STTM was outfitted with two Nortek Vector acoustic Doppler velocimeters equipped with inertial motion units (ADVs), a bottom-tracking downward-looking Teledyne RDI Workhorse 600 kHz ADCP to provide motion-corrected flow and turbulence characteristics at high temporal resolution, and an upward-looking Teledyne RDI Sentinel V20 ADCP. The STTM was deployed at 44.92098 N, 66.98922 W from 24-31 May. 4. A vessel-mounted Teledyne RDI Workhorse 300 kHz ADCP collected current data along three transects over two days, 4-5 April. The data processing used DOLfYN version 0.11.2. All hdf5 files (i.e., files ending in `.h5`) contained here can be opened using that version of DOLfYN (e.g., `dat = dolfyn.load('')`). All distances are in meters (e.g., depth, range, MLLW, hab, eta, z_), and all velocities in m/s. See the DOLfYN documentation https://lkilcher.github.io/dolfyn/), and/or the Nortek and Teledyne RDI documentation for additional details. Additional details on the dataset can be found in the README.pdf, including: - Format details of each data file. - How to regenerate the data-processing (using the files in the `wp2017_processing.zip` archive).

16 TIDAL AND WAVE POWER↗

Applying Hanford Tank Mixing Data to Define Pulse Jet Mixer Operation

Pulse jet mixed (PJM) process vessels are being developed for storing, blending, and chemical processing of nuclear waste slurries at the Waste Treatment and Immobilization Plant (WTP) to be built at Hanford, Washington. These waste slurries exhibit variable process feed characteristics including Newtonian to non-Newtonian rheologies over a range of solids loadings. Waste feed to the WTP from the Hanford Tank Farms will be accomplished via the Waste Feed Delivery (WFD) system which includes million-gallon underground storage double-shell tanks (DSTs) with dual-opposed jet mixer pumps. Experience using WFD type jet mixer pumps to mobilize actual Hanford waste in DSTs may be used to establish design threshold criteria of interest to pulse jet mixed process vessel operation. This paper describes a method to evaluate the pulse jet mixed vessel capability to process waste based on information obtained during mobilizing and suspending waste by the WFD system jet mixer pumps in a DST. Calculations of jet velocity and wall shear stress in a specific pulse jet mixed process vessel were performed using a commercial computational fluid dynamics (CFD) code. The CFD-modelled process vessel consists of a 4.9-m- (16-ft-) diameter tank with a 2:1 semi-elliptical head, a single, 10-cm (4-in.) downward facing 60-degree conical nozzle, and a 0.61-m (24-in.) inside diameter PJM. The PJM is located at 70% of the vessel radius with the nozzle stand-off-distance 14 cm (6 in.) above the vessel head. The CFD modeled fluid velocity and wall shear stress can be used to estimate vessel waste-processing performance by comparison to available actual WFD system process data. Test data from the operation of jet mixer pumps in the 23-m (75-ft) diameter DSTs have demonstrated mobilization, solid particles in a sediment matrix were moved from their initial location, and suspension, mobilized solid particles were moved to a higher elevation in the vessel than their initial location, of waste solids. Jet mixer pumps were used in Hanford waste tank 241-AZ-101, and at least 95% of the 0.46-m (18-in.) deep sediment, with a shear strength of 1,500 to 4,200 Pa, was mobilized. Solids with a median particle size of 43 μm, 90th percentile of 94μm, were suspended in tank 241-AZ-101 to at least 5.5 m (216 in.) above the vessel bottom. Analytical calculations for this jet mixer pump test were used to estimate the velocities and wall shear stress that mobilized and suspended the waste. These velocities and wall shear stresses provide design threshold criteria which are metrics for system performance that can be evaluated via testing. If the fluid motion in a specific pulse jet mixed process vessel meets or exceeds the fluid motion of the demonstrated performance in the WFD system, confidence is provided that that vessel will similarly mobilize and suspend those solids if they were within the WTP. The single PJM CFD-calculated jet velocity and wall shear stress compare favorably with the design threshold criterion estimated for the tank 241-AZ-101 process data. Therefore, for both mobilization and suspension, the performance data evaluated from the WFD system testing increases confidence that the performance of the pulse jet mixed process vessels will be sufficient to process that waste even if that waste is not fully characterized.

mixing' slurry↗

Raman Lidar Profiles Best Estimate Value-Added Product Technical Report

The ARM Raman lidars are semi-autonomous ground-based systems that transmit at a wavelength of 355 nm with 300 mJ, ~5 ns pulses, and a pulse repetition frequency of 30Hz. Signals from the various detection channels are processed to produce time- and height-resolved estimates of several geophysical quantities, such as water vapor mixing ratio, relative humidity, aerosol scattering ratio, backscatter, optical depth, extinction, and depolarization ratio. Data processing is currently handled by a suite of six value-added product (VAP) processes. Collectively, these processes are known as the Raman Lidar Profiles VAP (RLPROF). The top-level best-estimate (BE) VAP process was introduced in order to bring together the most relevant information from the intermediate-level VAPs. As such, the BE process represents the final stage in data processing for the Raman lidar. Its principal function is to extract the primary variables from each of the intermediate-level VAPs, perform additional quality control, and combine all of this information into a single output file for the end-user. The focus of this document is to describe the processing performed by the BE VAP process.

54 ENVIRONMENTAL SCIENCES↗

Comparison Analysis of the Eastern Range False Cape 915-MHz Doppler Radar Wind Profiler

The United States Space Force (USSF) is responsible for space vehicle launches at its Eastern Range (ER), which includes the Cape Canaveral Space Force Station (CCSFS). Multiple systems are used to measure the atmosphere at the ER, including suites of Doppler Radar Wind Profilers (DRWPs) operating at 915MHz that measure winds within the lowest few kilometers of the atmosphere. Observations of boundary layer winds can be used for multiple applications, including serving as input to toxic dispersion models and characterizing winds for low-level aborts. In fall 2020 the USSF requested NASA’s Marshall Space Flight Center (MSFC) Natural Environments Branch (NE) to evaluate wind output from the False Cape(FC) DRWP system (MSFC NE 2021). In that analysis, it was found that data availability was low in the lowest few hundred meters of a profile, shown in Figure 1. To increase the available data, the USSF modified the quality control (QC) algorithm for the FCDRWP, which collected data from October 2020 to January 2021. In Fall 2021, the USSF requested NASA’s MSFC NE to evaluate the modified QC algorithm by comparing the data processed by the modified method (referred to as beam data) to data processed with the original QC method (referred to as original data). This report describes the system and the analyses that MSFC NE conducted to compare the availability and quality of data from the two QC methods

Maggire Fielder↗

TEAMER: Pitching Foil Crossflow Turbine Efficiency Data

This dataset documents the efficiency testing of a pitching foil crossflow turbine, conducted at the University of New Hampshire's (UNH) Chase Ocean Engineering Laboratory tow tank facility. The tests explored various pitch phases and amplitudes, ranging from 0 to 18 degrees, across different flow speeds and turbine RPMs. Specifically, testing was performed at pitch amplitudes of 0,3,6,9,12,15 and 18 degrees. The results indicate that turbine efficiency improves with increased pitch amplitude, reaching optimal performance at 12 degrees. Modifications were made to the test frame by UNH to enhance the stiffness of the support structure. The testing specifically mapped the efficiency of a single quadrant of the Ocean Renewable Power Company (ORPC) Autonomous Turbine Generator Unit (ATGU), covering the full range of allowable pitching amplitudes and phases. The dataset provides both raw and processed data, including drag and torque measurements, and contains Python scripts used for data processing and visualization. This research was supported by funding from TEAMER RFTS 1 (Request for Technical Support).

16 TIDAL AND WAVE POWER↗

In Situ Transmission Electron Microscopy: Signal processing challenges and examples

Transmission electron microscopy (TEM) is a powerful tool for imaging material structure and characterizing material chemistry. Recent advances in data collection technology for TEM have enabled high-volume and high-resolution data collection at a microsecond frame rate. Here, taking advantage of these advances in data collection rates requires the development and application of data processing tools, including image analysis, feature extraction, and streaming data processing techniques. In this article, we highlight a few areas in materials science that have benefited from combining signal processing and statistical analysis with data collection capabilities in TEM and present a future outlook on opportunities of integrating signal processing with automated TEM data analysis.

36 MATERIALS SCIENCE↗

Rocket observations of solar radiation during the eclipse of 26 February 1979

Three Nike Tomahawk rockets were launched in Red Lake, Ontario, one previous to, and two during, the total eclipse of the Sun, for the purpose of studying the atmosphere and its interaction with solar radiation. The method and preliminary results of three experiments that were used to measure solar radiation in the X-ray, Lyman-alpha, and visible parts of the spectrum are described. The instrumentation designed for this investigation is discussed as well as post-flight data processing techniques. The retrieved data were processed to some extent to verify that a valid representation of the solar radiation was obtained. The Lyman-alpha experiment yielded very good results, and preliminary data are included. The visible radiation experiment served as support for the other experiments and also functioned well. Due to a high level of energetic particles during the eclipse, the X-ray data were contaminated and are not presented. However, satellite observations indicate no appreciable level of X-rays from the Sun during the time of the eclipse.

Bliss, H. M.↗

BigPanDA: PanDA Workload Management System and its Applications beyond ATLAS

Modern experiments collect peta-scale volumes of data and utilize vast, geographically distributed computing infrastructure that serves thousands of scientists around the world. Requirements for rapid, near real-time data processing, fast analysis cycles and need to run massive detector simulations to support data analysis pose special premium on efficient use of available computational resources. A sophisticated Workload Management System (WMS) is needed to coordinate the distribution and processing of data and jobs in such environment. The ATLAS experiment at CERN uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. While PanDAcurrently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, it runs around 2 million jobs per day on hundreds of Grid sites and serving thousands of ATLAS users. In 2017 about 1.5 exabytes of data were processed with PanDA.In 2012 BigPanDA project project was started with aim to introduce new types of computing resources into ATLAS computing infrastructure, but also to offering PanDA features to different data-intensive applications for projects and experiments outside of ATLAS and High-Energy and Nuclear Physics. In this article we will present accomplishments and discuss possible directions for future work.

Svirin, Pavlo↗