SEARCH · Engineering Papers

Results for “Big data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

All sources NASA NTRS DOE OSTI

DOE OSTI2016

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility (OLCF), MIRA supercomputer at Argonne Leadership Computing Facilities (ALCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava and others). Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full production for the ATLAS experiment since September 2015. We will present our current accomplishments with running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

Data-intensive science↗

DOE OSTI2017

Los Alamos on Radio Café: Ludmil Alexandrov

In a creative breakthrough in cancer research, Ludmil Alexandrov, the J. Robert Oppenheimer Distinguished Postdoctoral Fellow at Los Alamos National Laboratory, combines Big Data, supercomputing and machine-learning to identify the telltale mutations of cancer. Knowing these mutational signatures can help researchers develop new methods of prevention.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

Unleashing the Power of Industrial Big Data through Scalable Manual Labeling

Big Data plays a central role in the remarkable results achieved by Machine Learning (ML) and especially Deep Learning (DL) in the recent years. However, the difficulty in obtaining a reasonable amount of labeled samples limits ML/DL application in various domains, including industrial equipment and system monitoring. In this paper the need for methods that turn manual labeling into a scalable process is highlighted. A real world problem is analyzed for which weak supervision methods, successfully employed in other domains, did not produce acceptable results. An alternative approach based on clustering ensembles is described and tested, achieving good performance.

Paes Leao, Bruno↗

NASA NTRSDate not supplied

Open Source Application of Fusing Aerosol Products from GEO and LEO Satellites

Retrieving aerosol optical depths (AODs) from sun-synchronous polar orbiting (aka low earth orbit, LEO) satellites, such as MODISs, and VIIRSs, OMI, TROPOMI, etc, has become well-established as a tool for extracting information on particulate matter (PM) and related processes in the atmosphere. However, with recently launched geostationary satellites (GEO), such as GOES-16/17/18, and Himawari-8/9, and Meteosat Third Generation (MTG) they provide a much higher temporal resolution (order of 10 minutes), typically an image once or more per hour during daylight compared to LEO once per day. By combining these observations, we may be able to characterize the diurnal cycle of global AOD at the local, regional and global scale. While the science community is still exploring the new data from GEO observations, we have been thinking about how to properly combine/merge/fuse those data considering differences in their spatial and temporal resolutions. However, this poses a “Big Data” challenge. The big data challenge is not just about data storage, but also about data discoverability, and accessibility, and even more, about data migration/mirroring in the cloud-computing environment. This paper is merely showing some of the efforts and approaches we have attempted in fusing six satellites’ Level 2 aerosol data (three are from GEO (GOES-16/17 and Himawari-8), and the other three are from LEO (TERRA/MODIS, AQUA/MODIS, SNPP-VIIRS) from Dark Target (DT) aerosol retrieval algorithm. Having the on-demand capability of fusing remote sensing products onto the desired temporal and spatial domain enables researchers and application practitioners to better manipulate and work with satellite and sensor data. It is our hopeWe hope that by making such an open-source package, and the accompanying functionality, the scientific community will be granted easier access to aerosol data processing resources. The MEaSUREs Program (Making Earth System Data Records for Use in Research Environments) expands our understanding of the Earth's current system through atmospheric and surface measurements. In an effort to aid the scientific research component and improve open source methods, this project developed Python code for fusing six satellite Level 2 aerosol data (three are from geostationary satellites (GEO), and the other three are from low earth orbital satellites (LEO)) from Dark Target Aerosol Retrieval Algorithm.

Jennifer Wei↗

NASA NTRS2017

GeoDash: Assisting Visual Image Interpretation in Collect Earth Online by Leveraging Big Data on Google Earth Engine

Collect Earth Online (CEO) is a free and open online implementation of the FAO Collect Earth system for collaboratively collecting environmental data through the visual interpretation of Earth observation imagery. The primary collection mechanism in CEO is human interpretation of land surface characteristics in imagery served via Web Map Services (WMS). However, interpreters may not have enough contextual information to classify samples by only viewing the imagery served via WMS, be they high resolution or otherwise. To assist in the interpretation and collection processes in CEO, SERVIR, a joint NASA-USAID initiative that brings Earth observations to improve environmental decision making in developing countries, developed the GeoDash system, an embedded and critical component of CEO. GeoDash leverages Google Earth Engine (GEE) by allowing users to set up custom browser-based widgets that pull from GEE's massive public data catalog. These widgets can be quick looks of other satellite imagery, time series graphs of environmental variables, and statistics panels of the same. Users can customize widgets with any of GEE's image collections, such as the historical Landsat collection with data available since the 1970s, select date ranges, image stretch parameters, graph characteristics, and create custom layouts, all on-the-fly to support plot interpretation in CEO. This presentation focuses on the implementation and potential applications, including the back-end links to GEE and the user interface with custom widget building. GeoDash takes large data volumes and condenses them into meaningful, relevant information for interpreters. While designed initially with national and global forest resource assessments in mind, the system will complement disaster assessments, agriculture management, project monitoring and evaluation, and more.

SERVI↗

DOE OSTI2021

Facilitating better and faster simulations of aerosol-cloud interactions in Earth system models

Focal Area(s): 1. Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising a hierarchy of models. 2. Insight gleaned from complex data (both observed and simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI. Science Challenge: One major challenge that Earth system models (ESMs) face in providing credible prediction of the Earth system and its water cycle characteristics (e.g., mean state, variability, and extreme events) is to accurately simulate aerosol-cloud interactions (ACI). The physical, chemical, and dynamical processes affecting ACI are extremely complex and they range from nanoscale to planetary scale. In each model development cycle, scientists spend significant efforts investigating model deficiencies and uncertainties associated with aerosols (e.g., emissions, chemical processes, aerosol microphysics, and transport) and clouds (e.g., macrophysics, microphysics, turbulence, and large-scale circulation) in order to develop improved treatments. However, despite decades of active research, ACI is still a major source of uncertainty in climate projections, even though great progress has been made. Specific scientific challenges include: (i) Parameterizations are developed based on limited data; (ii) The complexity of a parameterization required for accurate predictions is not understood; (iii) Incomplete and unknown physics leads to errors in the fully coupled Earth system; and (iv) Complex physics is computationally too expensive to employ in ESMs.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2008

Proceedings of the 39th Lunar and Planetary Science Conference

Sessions with oral presentations include: A SPECIAL SESSION: MESSENGER at Mercury, Mars: Pingos, Polygons, and Other Puzzles, Solar Wind and Genesis: Measurements and Interpretation, Asteroids, Comets, and Small Bodies, Mars: Ice On the Ground and In the Ground, SPECIAL SESSION: Results from Kaguya (SELENE) Mission to the Moon, Outer Planet Satellites: Not Titan, Not Enceladus, SPECIAL SESSION: Lunar Science: Past, Present, and Future, Mars: North Pole, South Pole - Structure and Evolution, Refractory Inclusions, Impact Events: Modeling, Experiments, and Observations, Mars Sedimentary Processes from Victoria Crater to the Columbia Hills, Formation and Alteration of Carbonaceous Chondrites, New Achondrite GRA 06128/GRA 06129 - Origins Unknown, The Science Behind Lunar Missions, Mars Volcanics and Tectonics, From Dust to Planets (Planetary Formation and Planetesimals):When, Where, and Kaboom! Astrobiology: Biosignatures, Impacts, Habitability, Excavating a Comet, Mars Interior Dynamics to Exterior Impacts, Achondrites, Lunar Remote Sensing, Mars Aeolian Processes and Gully Formation Mechanisms, Solar Nebula Shake and Bake: Mixing and Isotopes, Lunar Geophysics, Meteorites from Mars: Shergottite and Nakhlite Invasion, Mars Fluvial Geomorphology, Chondrules and Chondrule Formation, Lunar Samples: Chronology, Geochemistry, and Petrology, Enceladus, Venus: Resurfacing and Topography (with Pancakes!), Overview of the Lunar Reconnaissance Orbiter Mission, Mars Sulfates, Phyllosilicates, and Their Aqueous Sources, Ordinary and Enstatite Chondrites, Impact Calibration and Effects, Comparative Planetology, Analogs: Environments and Materials, Mars: The Orbital View of Sediments and Aqueous Mineralogy, Planetary Differentiation, Titan, Presolar Grains: Still More Isotopes Out of This World, Poster sessions include: Education and Public Outreach Programs, Early Solar System and Planet Formation, Solar Wind and Genesis, Asteroids, Comets, and Small Bodies, Carbonaceous Chondrites, Chondrules and Chondrule Formation, Chondrites, Refractory Inclusions, Organics in Chondrites, Meteorites: Techniques, Experiments, and Physical Properties, MESSENGER and Mercury, Lunar Science Present: Kaguya (SELENE) Results, Lunar Remote Sensing: Basins and Mapping of Geology and Geochemistry, Lunar Science: Dust and Ice, Lunar Science: Missions and Planning, Mars: Layered, Icy, and Polygonal, Mars Stratigraphy and Sedimentology, Mars (Peri)Glacial, Mars Polar (and Vast), Mars, You are Here: Landing Sites and Imagery, Mars Volcanics and Magmas, Mars Atmosphere, Impact Events: Modeling, Experiments, and Observation, Ice is Nice: Mostly Outer Planet Satellites, Galilean Satellites, The Big Giant Planets, Astrobiology, In Situ Instrumentation, Rocket Scientist's Toolbox: Mission Science and Operations, Spacecraft Missions, Presolar Grains, Micrometeorites, Condensation-Evaporation: Stardust Ties, Comet Dust, Comparative Planetology, Planetary Differentiation, Lunar Meteorites, Nonchondritic Meteorites, Martian Meteorites, Apollo Samples and Lunar Interior, Lunar Geophysics, Lunar Science: Geophysics, Surface Science, and Extralunar Components, Mars, Remotely, Mars Orbital Data - Methods and Interpretation, Mars Tectonics and Dynamics, Mars Craters: Tiny to Humongous, Mars Sedimentary Mineralogy, Martian Gullies and Slope Streaks, Mars Fluvial Geomorphology, Mars Aeolian Processes, Mars Data and Mission,s Venus Mapping, Modeling, and Data Analysis, Titan, Icy Dwarf Satellites, Rocket Scientist's Toolbox: In Situ Analysis, Remote Sensing Approaches, Advances, and Applications, Analogs: Sulfates - Earth and Lab to Mars, Analogs: Remote Sensing and Spectroscopy, Analogs: Methods and Instruments, Analogs: Weird Places!. Print Only Early Solar System, Solar Wind, IDPs, Presolar/Solar Grains, Stardust, Comets, Asteroids, and Phobos, Venus, Mercury, Moon, Meteorites, Mars, Astrobiology, Impacts, Outer Planets, Satellites, and Rings, Support for Mission Operations, Analog Education and Public Outreach.

Source record↗

DOE OSTI2017

mdtmFTP and its evaluation on ESNET SDN testbed

In this paper, to address the high-performance challenges of data transfer in the big data era, we are developing and implementing mdtmFTP: a high-performance data transfer tool for big data. mdtmFTP has four salient features. First, it adopts an I/O centric architecture to execute data transfer tasks. Second, it more efficiently utilizes the underlying multicore platform through optimized thread scheduling. Third, it implements a large virtual file mechanism to address the lots-of-small-files (LOSF) problem. In conclusion, mdtmFTP integrates multiple optimization mechanisms, including–zero copy, asynchronous I/O, pipelining, batch processing, and pre-allocated buffer pools–to enhance performance. mdtmFTP has been extensively tested and evaluated within the ESNET 100G testbed. Evaluations show that mdtmFTP can achieve higher performance than existing data transfer tools, such as GridFTP, FDT, and BBCP.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Condition Monitoring of Wind Turbine Planetary Gearboxes Under Different Operating Conditions

Digitally enhanced services for wind power could reduce Operations and Maintenance (O&M) costs as well as the Levelised Cost Of Energy (LCOE). Therefore, there is a continuous need for advanced monitoring techniques which can exploit the opportunities of Internet of Things (IoT) and Big Data Analytics, revolutionizing the future of the energy sector. The heart of wind turbines is a rather complex epicyclic gearbox. Among others, extremely critical gearbox components which are often responsible for machinery stops are the rolling element bearings. The vibration signatures of bearings are rather weak compared to other components, such as gears, and as a result an extended number of signal processing techniques and tools have been proposed during the last decades, focusing towards accurate, early, and on time bearing fault detection with limited false alarms and missed detections. Envelope Analysis is one of the most important methodologies, where an envelope of the vibration signal is estimated usually after filtering around a frequency band excited by impacts due to the bearing faults. Different tools, such as Kurtogram, have been proposed in order to accurately select the optimum filter parameters (center frequency and bandwidth). Cyclic Spectral Correlation and Cyclic Spectral Coherence, based on Cyclostationary Analysis, have been proved as very powerful tools for condition monitoring. The monitoring techniques seem to have reached a mature level in case a machinery operates under steady speed and load. On the other hand, in case the operating conditions change, it is still unclear whether the change of the monitoring indicators is due to the change of the health of the machinery or due to the change of the operating parameters. Recently, the authors have proposed a new tool called IESFOgram, which is based on Cyclic Spectral Coherence and can automatically select the filtering band. Furthermore, the Cyclic Spectral Coherence is integrated along the selected frequency band leading to an Improved Envelope Spectrum. In this paper, the performance of the tool is evaluated and further extended on cases operating under different speeds and different loads. The effectiveness of the methodology is tested and validated on the National Renewable Energy Laboratory (NREL) wind turbine gearbox vibration condition monitoring benchmarking data set which includes various faults with different levels of diagnostic complexity as well as various speed and load operating conditions.

17 WIND ENERGY↗

DOE OSTI2016

INTEGRATION OF PANDA WORKLOAD MANAGEMENT SYSTEM WITH SUPERCOMPUTERS

Abstract The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the funda- mental nature of matter and the basic forces that shape our universe, and were recently credited for the dis- covery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 140 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data cen- ters are physically scattered all over the world. While PanDA currently uses more than 250000 cores with a peak performance of 0.3+ petaFLOPS, next LHC data taking runs will require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Com- puting Facility (OLCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava, and others). The current approach utilizes a modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single- threaded workloads in parallel on Titan s multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms. We will present our current accom- plishments in running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facility s infrastructure for High Energy and Nuclear Physics, as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

De, K↗

DOE OSTI2025

Release of ENDF81SaB: ENDF/B-VIII.1-Based ACE Data Files for Thermal Scattering

On August 30, 2024, the National Nuclear Data Center (NNDC) released the ENDF/B-VIII.1 nuclear data library. The library was released in the standard Evaluated Nuclear Data File (ENDF) format. These files can be accessed on the NNDC's website (www.nndc.bnl.gov). The files provided in the thermal neutron scattering sublibrary were processed into A Compact ENDF (ACE)-formatted files, verified, and validated by the XCP-5 Nuclear Data Team, resulting in the ENDF81SaB application library. This report details the processing of these files and the quality assurance approach taken. This is not intended to be a full validation effort; rather, this library is intended to simply reproduce the released files for further validation testing by the community. The validation basis and details of the evaluations are documented in the forthcoming ``Big Paper''.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRSDate not supplied

Big-data Efficient and Automated Science Transfer (BEAST): An Open-Source Software Architecture for Arc Jet Data Management, Modeling, and Automation

Big-data Efficient and Automated Science Transfer (BEAST) was conceived to address the existing ground testing data management of the NASA Ames arc jet facilities (e.g., manually entered Excel files and USB drive data transfers). These data management practices were seen as a choke point for future thermal protection system (TPS) development as they limit statistical tracking, resolution of diagnostics, coordination between video/time series, data throughput, and data processing speed/efficiency. Consequently, BEAST was developed to provide a new data infrastructure with streamlined data collection, processing, transfer, and analysis. This new framework also seeks to implement the FAIR principles of data stewardship: Findable, Accessible, Interoperable, and Reusable. The BEAST framework is based on a combination of the Python Django web framework and the Python data stack to provide a monolithic, open-source platform for data management, automation, and machine learning. This architecture was chosen for maintainability and scalability for a small, in-house development team. This paper will describe the application framework, deployment, and discuss the benefits and future plans for the system.

Data management↗

NASA NTRSDate not supplied

Big-data Efficient Automated Science Transfer (BEAST): an open-source software architecture for arc jet data management, modeling, and automation

Data management↗

DOE OSTI2013

Synthesis and structure of high-quality films of copper polyphthalocyanine – 2D conductive polymer

Graphical abstract: - Highlights: • 2D polymers show a big promise for science and technology. • We develop a new procedure for the direct synthesis of copper polyphthalocyanine. • We obtain reliable experimental data on the CuPPC structure. • With the support of quantum chemical calculations we describe electronic structure of CuPPC. - Abstract: Copper polyphthalocyanine (CuPPC), a 2D conjugated polymer, is a promising material for electronics and photovoltaics, but its applications were hindered by a poor processability. We propose an experimental approach, by which thin films of CuPPC, can be directly synthesized in a chemical vapor deposition (CVD) set-up at mild temperature (420 °C). High polymerization degree and high crystallinity of the films were confirmed by TEM, FTIR and UV–vis studies. From XRD and TEM electron diffraction, we conclude that the polymer has AA layer stacking with the inter-layer distance of 0.32 nm. The assignment of X-ray and TEM diffraction patterns was based on quantum-chemical calculations. Based on the latter, we also discuss electronic structure and conclude that CuPPC is rather a semi-metal than semi-conductor.

36 MATERIALS SCIENCE↗

DOE OSTI2016

WE-AB-207B-07: Dose Cloud: Generating “Big Data” for Radiation Therapy Treatment Plan Optimization Research

Purpose: To provide a tool to generate large sets of realistic virtual patient geometries and beamlet doses for treatment optimization research. This tool enables countless studies exploring the fundamental interplay between patient geometry, objective functions, weight selections, and achievable dose distributions for various algorithms and modalities. Methods: Generating realistic virtual patient geometries requires a small set of real patient data. We developed a normalized patient shape model (PSM) which captures organ and target contours in a correspondence-preserving manner. Using PSM-processed data, we perform principal component analysis (PCA) to extract major modes of variation from the population. These PCA modes can be shared without exposing patient information. The modes are re-combined with different weights to produce sets of realistic virtual patient contours. Because virtual patients lack imaging information, we developed a shape-based dose calculation (SBD) relying on the assumption that the region inside the body contour is water. SBD utilizes a 2D fluence-convolved scatter kernel, derived from Monte Carlo simulations, and can compute both full dose for a given set of fluence maps, or produce a dose matrix (dose per fluence pixel) for many modalities. Combining the shape model with SBD provides the data needed for treatment plan optimization research. Results: We used PSM to capture organ and target contours for 96 prostate cases, extracted the first 20 PCA modes, and generated 2048 virtual patient shapes by randomly sampling mode scores. Nearly half of the shapes were thrown out for failing anatomical checks, the remaining 1124 were used in computing dose matrices via SBD and a standard 7-beam protocol. As a proof of concept, and to generate data for later study, we performed fluence map optimization emphasizing PTV coverage. Conclusions: We successfully developed and tested a tool for creating customizable sets of virtual patients suitable for large-scale radiation therapy optimization research.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2020

Experimentally Constrained (\(n,\gamma \)) Reaction Rates Relevant to r- and i-Process Nucleosynthesis

The element distribution that we observe in the Universe today, tells a fascinating story of nucleosynthesis events that have taken place throughout the 13.7-billion-years-long history starting with the Big Bang. It is known for a long time that radiative neutron-capture reactions play a major role in synthesizing elements heavier than iron. However, many questions remain when it comes to our understanding of neutron-capture processes in extreme stellar environments. In particular, the intermediate and rapid neutron-capture processes are very challenging to describe, as there exist little or no data on the much-needed neutron-capture rates. In this work we discuss possibilities to obtain indirect, experimental constrains on these rates by means of the Oslo and the β-Oslo methods.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS2009

On Organization of Information: Approach and Early Work

In this report we describe an approach for organizing information for presentation and display. "e approach stems from the observation that there is a stepwise progression in the way signals (from the environment and the system under consideration) are extracted and transformed into data, and then analyzed and abstracted to form representations (e.g., indications and icons) on the user interface. In physical environments such as aerospace and process control, many system components and their corresponding data and information are interrelated (e.g., an increase in a chamber s temperature results in an increase in its pressure). "ese interrelationships, when presented clearly, allow users to understand linkages among system components and how they may affect one another. Organization of these interrelationships by means of an orderly structure provides for the so-called "big picture" that pilots, astronauts, and operators strive for.

Degani, Asaf↗

DOE OSTI2014

THE NEW ONLINE METADATA EDITOR FOR GENERATING STRUCTURED METADATA

Nobody is better suited to “describe” data than the scientist who created it. This “description” about a data is called Metadata. In general terms, Metadata represents the who, what, when, where, why and how of the dataset [1]. eXtensible Markup Language (XML) is the preferred output format for metadata, as it makes it portable and, more importantly, suitable for system discoverability. The newly developed ORNL Metadata Editor (OME) is a Web-based tool that allows users to create and maintain XML files containing key information, or metadata, about the research. Metadata include information about the specific projects, parameters, time periods, and locations associated with the data. Such information helps put the research findings in context. In addition, the metadata produced using OME will allow other researchers to find these data via Metadata clearinghouses like Mercury [2][4]. OME is part of ORNL’s Mercury software fleet [2][3]. It was jointly developed to support projects funded by the United States Geological Survey (USGS), U.S. Department of Energy (DOE), National Aeronautics and Space Administration (NASA) and National Oceanic and Atmospheric Administration (NOAA). OME’s architecture provides a customizable interface to support project-specific requirements. Using this new architecture, the ORNL team developed OME instances for USGS’s Core Science Analytics, Synthesis, and Libraries (CSAS&L), DOE’s Next Generation Ecosystem Experiments (NGEE) and Atmospheric Radiation Measurement (ARM) Program, and the international Surface Ocean Carbon Dioxide ATlas (SOCAT).Researchers simply use the ORNL Metadata Editor to enter relevant metadata into a Web-based form. From the information on the form, the Metadata Editor can create an XML file on the server that the editor is installed or to the user’s personal computer. Researchers can also use the ORNL Metadata Editor to modify existing XML metadata files.As an example, an NGEE Arctic scientist use OME to register their datasets to the NGEE data archive and allows the NGEE archive to publish these datasets via a data search portal (http://ngee.ornl.gov/data). These highly descriptive metadata created using OME allows the Archive to enable advanced data search options using keyword, geo-spatial, temporal and ontology filters. Similarly, ARM OME allows scientists or principal investigators (PIs) to submit their data products to the ARM data archive.How would OME help Big Data Centers like the Oak Ridge National Laboratory Distributed Active Archive Center (ORNL DAAC)?The ORNL DAAC is one of NASA’s Earth Observing System Data and Information System (EOSDIS) data centers managed by the Earth Science Data and Information System (ESDIS) Project. The ORNL DAAC archives data produced by NASA's Terrestrial Ecology Program. The DAAC provides data and information relevant to biogeochemical dynamics, ecological data, and environmental processes, critical for understanding the dynamics relating to the biological, geological, and chemical components of the Earth's environment. Typically data produced, archived and analyzed is at a scale of multiple petabytes, which makes the discoverability of the data very challenging. Without proper metadata associated with the data, it is difficult to find the data you are looking for and equally difficult to use and understand the data.OME will allow data centers like the NGEE and ORNL DAAC to produce meaningful, high quality, standards-based, descriptive information about their data products in-turn helping with the data discoverability and interoperability. Useful Links:USGS OME: http://mercury.ornl.gov/OME/NGEE OME: http://ngee-arctic.ornl.gov/ngeemetadata/ARM OME: http://archive2.ornl.gov/armome/Contact: Ranjeet Devarakonda (devarakondar@ornl.gov)References:[1] Federal Geographic Data Committee. Content standard for digital geospatial metadata. Federal Geographic Data Committee, 1998.[2] Devarakonda, Ranjeet, et al. "Mercury: reusable metadata management, data discovery and access system." Earth Science Informatics 3.1-2 (2010): 87-94.[3] Wilson, B. E., Palanisamy, G., Devarakonda, R., Rhyne, B. T., Lindsley, C., & Green, J. (2010). Mercury Toolset for Spatiotemporal Metadata.[4] Pouchard, L. C., Branstetter, M. L., Cook, R. B., Devarakonda, R., Green, J., Palanisamy, G., ... & Noy, N. F. (2013). A Linked Science investigation: enhancing climate change data discovery with semantic technologies. Earth science informatics, 6(3), 175-185.

FGDC CSDGM↗