SEARCH · Engineering Papers

Results for “scalable performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,549 records · Page 86

All sources NASA NTRS DOE OSTI

DOE OSTI2021

MOSIQS: Persistent Memory Object Storage With Metadata Indexing and Querying for Scientific Computing

Scientific applications often require high-bandwidth shared storage to perform joint simulations and collaborative data analytics. Shared memory pools provide a chance to satisfy such needs. Recently, a high-speed network such as Gen-Z utilizing persistent memory (PM) offers an opportunity to create a shared memory pool connected to compute nodes. However, there are several challenges to use scientific applications on the shared memory pool directly such as scalability, failure-atomicity, and lack of scientific metadata-based search and query. In this paper, we propose MOSIQS, a persistent memory object storage framework with metadata indexing and querying for scientific computing. We design MOSIQS based on the key idea that memory objects on PM pool can live beyond the application lifetime and can become the sharing currency for applications and scientists. MOSIQS provides an aggregate memory pool atop an array of persistent memory devices to store and access memory objects to accelerate scientific computing. MOSIQS uses a lightweight persistent memory key-value store to manage the metadata of memory objects, which enables memory object sharing. To facilitate metadata search and query over millions of memory objects resident on memory pool, we introduce Group Split and Merge (GSM), a novel persistent index data structure designed primarily for scientific datasets. GSM splits and merges dynamically to minimize the query search space and maintains low query processing time while overcoming the index storage overhead. MOSIQS is implemented on top of PMDK. We evaluate the proposed approach on many-core server with an array of real PM devices. Experimental results show that MOSIQS gains a 100% write performance improvement and executes multi-attribute queries efficiently with 2.7× less index storage overhead offering significant potential to speed up scientific computing applications.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2000

HTMT-class Latency Tolerant Parallel Architecture for Petaflops Scale Computation

Computational Aero Sciences and other numeric intensive computation disciplines demand computing throughputs substantially greater than the Teraflops scale systems only now becoming available. The related fields of fluids, structures, thermal, combustion, and dynamic controls are among the interdisciplinary areas that in combination with sufficient resolution and advanced adaptive techniques may force performance requirements towards Petaflops. This will be especially true for compute intensive models such as Navier-Stokes are or when such system models are only part of a larger design optimization computation involving many design points. Yet recent experience with conventional MPP configurations comprising commodity processing and memory components has shown that larger scale frequently results in higher programming difficulty and lower system efficiency. While important advances in system software and algorithms techniques have had some impact on efficiency and programmability for certain classes of problems, in general it is unlikely that software alone will resolve the challenges to higher scalability. As in the past, future generations of high-end computers may require a combination of hardware architecture and system software advances to enable efficient operation at a Petaflops level. The NASA led HTMT project has engaged the talents of a broad interdisciplinary team to develop a new strategy in high-end system architecture to deliver petaflops scale computing in the 2004/5 timeframe. The Hybrid-Technology, MultiThreaded parallel computer architecture incorporates several advanced technologies in combination with an innovative dynamic adaptive scheduling mechanism to provide unprecedented performance and efficiency within practical constraints of cost, complexity, and power consumption. The emerging superconductor Rapid Single Flux Quantum electronics can operate at 100 GHz (the record is 770 GHz) and one percent of the power required by convention semiconductor logic. Wave Division Multiplexing optical communications can approach a peak per fiber bandwidth of 1 Tbps and the new Data Vortex network topology employing this technology can connect tens of thousands of ports providing a bi-section bandwidth on the order of a Petabyte per second with latencies well below 100 nanoseconds, even under heavy loads. Processor-in-Memory (PIM) technology combines logic and memory on the same chip exposing the internal bandwidth of the memory row buffers at low latency. And holographic storage photorefractive storage technologies provide high-density memory with access a thousand times faster than conventional disk technologies. Together these technologies enable a new class of shared memory system architecture with a peak performance in the range of a Petaflops but size and power requirements comparable to today's largest Teraflops scale systems. To achieve high-sustained performance, HTMT combines an advanced multithreading processor architecture with a memory-driven coarse-grained latency management strategy called "percolation", yielding high efficiency while reducing the much of the parallel programming burden. This paper will present the basic system architecture characteristics made possible through this series of advanced technologies and then give a detailed description of the new percolation approach to runtime latency management.

Sterling, Thomas↗

DOE OSTI2021

Synthesis of High-Performance Lignin-Based Inverse Thermoplastic Vulcanizates with Tailored Morphology and Properties

In this work, we report synthesis of a high-strength renewable phenolic composition with linear large deformation strain without a thermoplastic-like yielding while retaining thermal processability. Small molecule carboxylic acid derivatives with varying molecular architectures act as esterifying crosslinkers in an equal mass mixture of lignin and acrylonitrile–butadiene copolymers in a highly scalable, solvent-free process. These “inverse thermoplastic vulcanizates” (iTPVs)—unique in their approach of crosslinking the rigid lignin phase rather than the soft phase—exhibit ordered self-assembly, tunable nanoscale morphology, and processability. The first of its kind iTPV compositions exhibit engineering stress–strain curves with two- to sixfold linear extensibility, a twofold rise in strength, and an order of magnitude enhanced modulus compared to a simple lignin-rubber blend. Viscoelastic properties correlate well with crosslinker architecture and the resulting morphology, allowing competing properties of toughness and stiffness to be tuned. This research finds a path for identifying the potential of lignin as a sustainable feedstock.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication

Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.

Tan, Nigel↗

DOE OSTI2024

Development of Thin, Robust, Lithium-Impenetrable, HighConductivity, Electrochemically Stable, Scalable, and Low-Cost Glassy Solid Electrolytes for Solid State Lithium Batteries (Final Report)

Solid-state electrolytes (SSEs) must have thicknesses of < 50 microns in order to be competitive with current state-of-the-art organic liquid electrolyte batteries. Mixed oxy-sulfide-nitride (MOSN) glassy solid electrolytes (GSEs) can be drawn into thin ribbons using the glass redraw process, to form glasses with superior performance in a safe, lower-cost, monolithic glass film. It is expected that these new GSEs in solid-state lithium batteries (SSLBs) increase the energy density (anode basis) from ~300 mAh/g to ~ 4000 mAh/g. This project is focused on the development of candidate MOS GSE compositions, and optimization of the thin film drawing process to generate thin film glass ribbons for use in SSLBs.

25 ENERGY STORAGE↗

DOE OSTI2024

Single Crystalline GeSe Van Der Waals Ribbons With Uniform Layer Stacking, High Carrier Mobility, and Adjustable Edge Morphology

Abstract Performance of the group IV monochalcogenide GeSe in solar cells, electronic, and optoelectronic devices is expected to improve when high‐quality single crystalline material is used rather than polycrystalline films. Crystalline flakes represent an attractive alternative to bulk single crystals as their synthesis may be developed to be scalable, faster, and with higher overall yield. However, large – and especially large and thin – single crystal flakes are notoriously hard to synthesize. Here it is demonstrated that vapor‐liquid‐solid growth combined with direct lateral vapor‐solid incorporation produces high‐quality single crystalline GeSe ribbons with tens of micrometers size and controllable thickness. Electron microscopy shows that the ribbons exhibit perfect equilibrium (AB) van der Waals stacking order without extended defects across the entire thickness, in contrast to the conventional case of substrate‐supported flakes where material is added via layer‐by‐layer nucleation and growth on the basal plane. Electrical measurements show anisotropic transport and a high Hall mobility of 85 cm 2 V −1 s −1 , on par with the best single crystals to date. Growth from mixed GeSe and SnSe vapors, finally, yields ribbons with unchanged structure and composition but with jagged edges, promising for applications that rely on ample chemically active edge sites, such as catalysis or photocatalysis.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2025

Etching-assisted upcycling of Ni-lean to Ni-rich cathode materials

Upcycling is recognized as a sustainable recycling approach for spent lithium-ion batteries. However, existing upcycling methods typically involve intricate pretreatment or post-treatment steps, complicating their practical application. Here, in this study, we propose a straightforward, etching-assisted upcycling method that effectively transforms polycrystalline Ni-lean cathodes into high-performance single crystal Ni-rich cathodes. During the etching step, nickel acetate was dissolved into acetic acid and then polycrystalline NMC111 are etched in the solution. Finally, polycrystalline NMC111 are converted into single crystal particles coated with amorphous nickel acetate. This significantly enhances elemental diffusion during subsequent sintering by minimizing both particle size and the contact distance between NMC111 and nickel acetate. As elemental diffusion is improved and acetate ions decompose completely during sintering, the process requires neither additional pretreatment nor post-treatment. The resulting cathode materials (Etched-UP622) exhibit superior structural and electrochemical properties compared to the Control622, achieving an energy density of 719.7 Wh/kg, approximately 56.7 mAh/g higher than Control622 and 125.5 Wh/kg higher than NMC111. Etched-UP622 also delivers higher discharge capacity, improved rate performance and cycling stability, surpassing Control622 and NMC111. Meanwhile, NMC811 also can be synthesized by the proposed strategy, and the discharge capacity can reach 166.9 mAh/g at 1C, similar to 14 mAh/g higher than Control811. Overall, this etching-assisted strategy simplifies the upcycling process and offers a scalable, sustainable route for producing high-quality cathode materials.

Acid etching↗

DOE OSTI2024

Transferring predictions of formation energy across lattices of increasing size*

In this study, we show the transferability of graph convolutional neural network (GCNN) predictions of the formation energy of the nickel-platinum solid solution alloy across atomic structures of increasing sizes. The original dataset was generated with the large-scale atomic/molecular massively parallel simulator using the second nearest-neighbor modified embedded-atom method empirical interatomic potential. Geometry optimization was performed on the initially randomly generated face centered cubic crystal structures and the formation energy has been calculated at each step of the geometry optimization, with configurations spanning the whole compositional range. Using data from various steps of the geometry optimization, we first trained our open-source, scalable implementation of GCNN called HydraGNN on a lattice of 256 atoms, which accounts well for the short-range interactions. Using this data, we predicted the formation energy for lattices of 864 atoms and 2048 atoms, which resulted in lower-than-expected accuracy due to the long-range interactions present in these larger lattices. We accounted for the long-range interactions by including a small amount of training data representative for those two larger sizes, whereupon the predictions of HydraGNN scaled linearly with the size of the lattice. Therefore, our strategy ensured scalability while reducing significantly the computational cost of training on larger lattice sizes.

36 MATERIALS SCIENCE↗

DOE OSTI2025

State Dependent Optimization with Quantum Circuit Cutting

Quantum circuits can be reduced through optimization to better fit the constraints of quantum hardware. One such method, initial-state dependent optimization (ISDO), reduces gate count by leveraging knowledge of the input quantum states. Surprisingly, we found that ISDO is broadly applicable to the downstream circuits produced by circuit cutting. Circuit cutting also requires measuring upstream qubits and has some flexibility of selection observables to do reconstruction. Therefore, we propose a state-dependent optimization (SDO) framework that incorporates ISDO, our newly proposed measure-state dependent optimization (MSDO), and a biased observable selection strategy. Building on the strengths of the SDO framework and recognizing the scalability challenges of circuit cutting, we propose nonseparate circuit cutting-a more flexible approach that enables optimizing gates without fully separating them. We validate our methods on noisy simulations of QAOA, QFT, and BV circuits. Results show that our approach consistently mitigates noise and improves overall circuit performance, demonstrating its promise for enhancing quantum algorithm execution on near-term hardware.

Li, Xinpeng↗

DOE OSTI2025

Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models

Semantic schemas like Haystack 4, Brick and ASHRAE standard 223 enable the structured, standardized, and machine-readable representation of building data, facilitating interoperability, data integration, and advanced analytics. However, extracting information from these models requires specialized expertise in SPARQL and other programming languages, skills that are not commonly found among building professionals. Recent advancements in Large Language Models (LLMs), such as ChatGPT, enable the construction of queries using natural language, making it easier for individuals to interact with these systems in a manner that resembles everyday speech. However, these methods have not yet been tested on building semantic ontologies. This paper introduces a novel workflow and tool for enabling users to ask questions about a specific building's data, using natural language and receive answers automatically generated by GPT-4o. Our approach integrates semantic ontologies with advanced LLM capabilities to automate three critical steps: (1) generating SPARQL queries to retrieve time series references from ontological models, (2) extracting the corresponding time series data from the Building Automation System, and (3) performing computations and visualizations tailored to the user's query. The proposed method simplifies access to BAS data, allowing both domain experts and non-specialists to conduct sophisticated analyses without needing extensive technical knowledge of semantic web technologies. By demonstrating this pipeline, we facilitate more accessible and scalable data-driven decision-making in building operations and management.

Mulayim, Ozan Baris↗

DOE OSTI2021

Real Time - Optimal Power Flow Based Distributed Energy Resources Management System (DERMS) (CRADA Number CRD-20-16909 Final Report)

The integration of behind-the-meter distributed energy resources (DERs) into distribution systems brings transformative changes to power systems. This requires operators and planners to find solutions to modernize electric grids and to effectively manage DERs for grid services. NREL developed novel DER management algorithms (referred to as Real-Time Optimal Power Flow, RT-OPF) through U.S. Department of Energy (DOE)-funded efforts, including Advanced Research Projects Agency–Energy (ARPA-E) Network Optimized Distributed Energy Systems (NODES) funding. This cutting-edge control technology aims to modernize distribution systems with large amounts of DER integration, which will help utilities solve issues brought by renewable integration and build resilient and renewable-based electric grids nationwide. Utilidata worked with NREL to investigate the commercialization opportunity of this RT-OPF-based distributed energy resource management system (DERMS). In this project, NREL performs the technology transfer of the RT-OPF to Utilidata to help them fully understand the RT-OPF solution, to identify potential engineering hurdles, and to assess the expected commercial value of various RT-OPF use cases and deployment. The technology transfer work includes two major tasks. First, NREL performs an in-depth knowledge transfer of the entire RT-OPF solution to Utilidata to help them gain an extensive and detailed understanding of the complete RT-OPF solution. In this task, NREL provides exhaustive information (e.g., documentation, code packages, laboratory and field trial data, performance results) while conducting in-depth training sessions to provide a thorough explanation of the entire solution. NREL hosts meetings to present different topics related to the RT-OPF solution, and question-and-answer sessions are included in each meeting to better explain the RT-OPF-related work. Second, the RT-OPF simulations are performed in a laboratory environment. The main objectives of this task are to walk through with Utilidata engineers how to set up a simulation of RT-OPF, identifying each RT-OPF code block/component in operation, learning how these components interact with each other, and eventually running RT-OPF simulations under various system conditions. This task helps Utilidata engineers understand performance limitations and constraints while also quantifying the commercial value of the RT-OPF for different use cases. Based on these two tasks, Utilidata engineers should be able to define and prioritize the next steps of RT-OPF implementation with an eye toward commercial success and scalability of the solution. The next steps are expected to be part of a new project following the conclusion of this project.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Enhancing Lattice Kinetic Schemes for Fluid Dynamics with Lattice-Equivariant Neural Networks

A new class of equivariant neural networks is presented, hereby dubbed lattice-equivariant neural networks (LENNs), designed to satisfy local symmetries of a lattice structure. The approach develops within a recently introduced framework aimed at learning neural network-based surrogate models’ lattice Boltzmann collision operators. Whenever neural networks are employed to model physical systems, respecting symmetries and equivariance properties has been shown to be key for accuracy, numerical stability, and performance. Here, hinging on ideas from group representation theory, trainable layers are defined whose algebraic structure is equivariant with respect to the symmetries of the lattice cell. In this work, the presented method naturally allows for efficient implementations, in terms of both memory usage and computational costs, supporting scalable training/testing for lattices in two spatial dimensions and higher (in which the size of symmetry group grows). The approach is validated and tested considering 2D and 3D flowing dynamics, both in laminar and turbulent regimes. It is compared with group-averaged-based symmetric networks and with plain, nonsymmetric, networks, showing how the presented approach unlocks the (a posteriori) accuracy and training stability of the former models and the train/inference speed of the latter networks. (LENNs are about one order of magnitude faster than group-averaged networks in 3D.) The work in this paper opens toward practical use of machine learning-augmented lattice Boltzmann CFD in real-world simulations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

A Scalable Gaussian Process Approach to Shear Mapping with MuyGPs

Analysis of cosmic shear is an integral part of understanding structure growth across cosmic time, which in turn provides us with information about the nature of dark energy. Conventional methods generate shear maps from which we can infer the matter distribution in the universe. Current methods (e.g., Kaiser–Squires inversion) for generating these maps, however, are tricky to implement and can introduce bias. Recent alternatives construct a spatial process prior for the lensing potential, which allows for inference of the convergence and shear parameters given lensing shear measurements. Realizing these spatial processes, however, scales cubically in the number of observations—an unacceptable expense as near-term surveys expect billions of correlated measurements. Therefore, we present a linearly scaling shear map construction alternative using a scalable Gaussian process prior called MuyGPs. MuyGPs avoids cubic scaling by conditioning interpolation on only nearest neighbors and fits hyperparameters using batched leave-one-out cross-validation. This work is the first step toward a full, scalable mass mapping method. We work in a simplified regime where we validate our method by interpolating and analyzing maps given noisy point-estimate data from all three shear fields, taken from a suite of N -body ray-tracing simulations. We also show that we can perform these operations at the scale of billions of galaxies on high-performance computing platforms.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models

Mulayim, Ozan Baris↗

NASA NTRS2007

Antenna Technology and other Radio Frequency (RF) Communications Activities at the Glenn Research Center in Support of NASA's Exploration Vision

NASA s Vision for Space Exploration outlines a very ambitious program for the next several decades of the Space Agency endeavors. Ahead is the completion of the International Space Station (ISS); safely flight the shuttle (STS) until 2010; develop and fly the Crew Exploration Vehicle (Orion) by no later than 2014; return to the moon by no later than 2020; extend human presence across the solar system and beyond; implement a sustainable and affordable human and robotic program; develop supporting innovative technologies, knowledge and infrastructure; and promote international and commercial participation in exploration. To achieve these goals, a series of enabling technologies must be developed or matured in a timely manner. Some of these technologies are: spacecraft RF technology (e.g., high power sources and large antennas which using surface receive arrays can get up to 1 Gbps from Mars), uplink arraying (reduce reliance on large ground-based antennas and high operation costs; single point of failure; enable greater data-rates or greater effective distance; scalable, evolvable, flexible scheduling), software define radio (i.e., reconfigurable, flexible interoperability allows for in flight updates open architecture; reduces mass, power, volume), and optical communications (high capacity communications with low mass/power required; significantly increases data rates for deep space). This presentation will discuss some of the work being performed at the NASA Glenn Research Center, Cleveland, Ohio, in antenna technology as well as other on-going RF communications efforts.

Miranda, Felix A.↗

NASA NTRS2007

Stretched Lens Array Squarerigger (SLASR) Technology Maturation

Since April 2005, our team has been underway on a competitively awarded program sponsored by NASA s Exploration Systems Mission Directorate to develop, refine, and mature the unique solar array technology known as Stretched Lens Array SquareRigger (SLASR). SLASR offers an unprecedented portfolio of performance metrics, SLASR offers an unprecedented portfolio of performance metrics, including the following: Areal Power Density = 300 W/m2 (2005) - 400 W/m2 (2008 Target) Specific Power = 300 W/kg (2005) - 500 W/kg (2008 Target) for a Full 100 kW Solar Array Stowed Power = 80 kW/cu m (2005) - 120 kW/m3 (2008 Target) for a Full 100 kW Solar Array Scalable Array Capacity = 100 s of W s to 100 s of kW s Super-Insulated Small Cell Circuit = High-Voltage (300-600 V) Operation at Low Mass Penalty Super-Shielded Small Cell Circuit = Excellent Radiation Hardness at Low Mass Penalty 85% Cell Area Savings = 75% Lower Array Cost per Watt than One-Sun Array Modular, Scalable, & Mass-Producible at MW s per Year Using Existing Processes and Capacities

O'Neill, Mark↗

NASA NTRS2013

Nano-Ceramic Coated Plastics

Plastic products, due to their durability, safety, and low manufacturing cost, are now rapidly replacing cookware items traditionally made of glass and ceramics. Despite this trend, some still prefer relatively expensive and more fragile ceramic/glassware because plastics can deteriorate over time after exposure to foods, which can generate odors, bad appearance, and/or color change. Nano-ceramic coatings can eliminate these drawbacks while still retaining the advantages of the plastic, since the coating only alters the surface of the plastic. The surface coating adds functionality to the plastics such as self-cleaning and disinfectant capabilities that result from a photocatalytic effect of certain ceramic systems. These ceramic coatings can also provide non-stick surfaces and higher temperature capabilities for the base plastics without resorting to ceramic or glass materials. Titanium dioxide (TiO2) and zinc oxide (ZnO) are the candidates for a nano-ceramic coating to deposit on the plastics or plastic films used in cookware and kitchenware. Both are wide-bandgap semiconductors (3.0 to 3.2 eV for TiO2 and 3.2 to 3.3 eV for ZnO), so they exhibit a photocatalytic property under ultraviolet (UV) light. This will lead to decomposition of organic compounds. Decomposed products can be easily washed off by water, so the use of detergents will be minimal. High-crystalline film with large surface area for the reaction is essential to guarantee good photocatalytic performance of these oxides. Low-temperature processing (<100 C) is also a key to generating these ceramic coatings on the plastics. One possible way of processing nanoceramic coatings at low temperatures (< 90 C) is to take advantage of in-situ precipitated nanoparticles and nanostructures grown from aqueous solution. These nanostructures can be tailored to ceramic film formation and the subsequent microstructure development. In addition, the process provides environment- friendly processing because of the aqueous solution. Low-temperature processing has also shown versatility to generate various nanostructures. The growth of low-dimensional nanostructures (0-D, 1-D) provides a means of enhancing the crystallinity of the solution-prepared films that is of importance for photocatalytic performance. This technology can generate durable, fully functional nano-ceramic coatings (TiO2, ZnO) on plastic materials (silicone, Teflon, PET, etc.) that can possess both photocatalytic oxide properties and flexible plastic properties. Processing cost is low and it does not require any expensive equipment investment. Processing can be scalable to current manufacturing infrastructure.

Cho, Junghyun↗

NASA NTRS2016

A Spatiotemporal Indexing Approach for Efficient Processing of Big Array-Based Climate Data with MapReduce

Climate observations and model simulations are producing vast amounts of array-based spatiotemporal data. Efficient processing of these data is essential for assessing global challenges such as climate change, natural disasters, and diseases. This is challenging not only because of the large data volume, but also because of the intrinsic high-dimensional nature of geoscience data. To tackle this challenge, we propose a spatiotemporal indexing approach to efficiently manage and process big climate data with MapReduce in a highly scalable environment. Using this approach, big climate data are directly stored in a Hadoop Distributed File System in its original, native file format. A spatiotemporal index is built to bridge the logical array-based data model and the physical data layout, which enables fast data retrieval when performing spatiotemporal queries. Based on the index, a data-partitioning algorithm is applied to enable MapReduce to achieve high data locality, as well as balancing the workload. The proposed indexing approach is evaluated using the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications (MERRA) climate reanalysis dataset. The experimental results show that the index can significantly accelerate querying and processing (10 speedup compared to the baseline test using the same computing cluster), while keeping the index-to-data ratio small (0.0328). The applicability of the indexing approach is demonstrated by a climate anomaly detection deployed on a NASA Hadoop cluster. This approach is also able to support efficient processing of general array-based spatiotemporal data in various geoscience domains without special configuration on a Hadoop cluster.

big data↗