SEARCH · Engineering Papers

Results for “Earth Science Data Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

All sources NASA NTRS DOE OSTI

NASA NTRS2005

Global monitoring of Sea Surface Salinity with Aquarius

Aquarius is a microwave remote sensing system designed to obtain global maps of the surface salinity field of the oceans from space. It will be flown on the Aquarius/SAC-D mission, a partnership between the USA (NASA) and Argentina (CONAE) with launch scheduled for late in 2008. The objective of Aquarius is to monitor the seasonal and interannual variation of the large scale features of the surface salinity field in the open ocean. This will provide data to address scientific questions associated with ocean circulation and its impact on climate. For example, salinity is needed to understand the large scale thermohaline circulation, driven by buoyancy, which moves large masses of water and heat around the globe. Of the two variables that determine buoyancy (salinity and temperature), temperature is already being monitored. Salinity is the missing variable needed to understand this circulation. Salinity also has an important role in energy exchange between the ocean and atmosphere, for example in the development of fresh water lenses (buoyant water that forms stable layers and insulates water below from the atmosphere) which alter the air-sea coupling. Aquarius is a combination radiometer and scatterometer (radar) operating at L-band (1.413 GHz for the radiometer and 1.26 GHz for the scatterometer). The primary instrument,for measuring salinity is the radiometer which is able to detect salinity because of the modulation salinity produces on the thermal emission from sea water. This change is detectable at the long wavelength end of the microwave spectrum. The scatterometer will provide a correction for surface roughness (waves) which is one of the greatest unknowns in the retrieval. The sensor will be in a sun-synchronous orbit at about 650 km with equatorial crossings of 6am/6pm. The antenna for these two instruments is a 3 meter offset fed reflector with three feeds arranged in pushbroom fashion looking away from the sun toward the shadow side of the orbit to minimize sunglint. The mission goal is to produce maps of the salinity field globally once each month with an accuracy of 0.2 psu and a spatial resolution of 100 km. This will be adequate to address l&ge scale features of the salinity field of the open ocean. The temporal resolution is sufficient to address seasonal changes and a three year mission is planned to-collect sufficient data to look for interannual variation. Aquarius is being developed by NASA as part of the Earth System Science Pathfinder (ESSP) program. The SAC-D mission is being developed by CONAE and will include the space craft and several additional instruments, including visible and infrared cameras and a microwave radiometer to monitor rain and wind velocity over the oceans, and sea ice.

Lagerloef, G. S. E.↗

NASA NTRS2004

Evaluation of the Harmful Algal Bloom Mapping System (HABMapS) and Bulletin

The National Oceanic and Atmospheric Administration (NOAA) Harmful Algal Bloom (HAB) Mapping System and Bulletin provide a Web-based geographic information system (GIS) and an e-mail alert system that allow the detection, monitoring, and tracking of HABs in the Gulf of Mexico. NASA Earth Science data that potentially support HABMapS/Bulletin requirements include ocean color, sea surface temperature (SST), salinity, wind fields, precipitation, water surface elevation, and ocean currents. Modeling contributions include ocean circulation, wave/currents, along-shore current regimes, and chlorophyll modeling (coupled to imagery). The most immediately useful NASA contributions appear to be the 1-km Moderate Resolution Imaging Spectrometer (MODIS) chlorophyll and SST products and the (presently used) SeaWinds wind vector data. MODIS pigment concentration and SST data are sufficiently mature to replace imagery currently used in NOAA HAB applications. The large file size of MODIS data is an impediment to NOAA use and modified processing schemes would aid in NOAA adoption of these products for operational HAB forecasting.

Hall, Callie↗

NASA NTRSDate not supplied

MISR Level 1B2 Ellipsoid Data (MI1B2E_V1)

The MISR instrument consists of nine pushbroom cameras which measure radiance in four spectral bands. Global coverage is achieved in nine days. The cameras are arranged with one camera pointing toward the nadir, four cameras pointing forward and four cameras pointing aftward. It takes 7 minutes for all nine cameras to view the same surface location. The view angles relative to the surface reference ellipsoid, are 0, 26.1, 45.6, 60.0, and 70.5 degrees. The spectral band shapes are nominally gaussian, centered at 443, 555, 670, and 865 nm. The Ellipsoid product is referenced to the World Geodetic System 1984 (WGS84) ellipsoid, which approximates the Earth's shape at sea level. In this product, the radiances and associated altitudes are projected to the ellipsoid, so that higher elevation data appear displaced from their true location for non-nadir camera views, much as they are seen by the instrument. (A cloud at location F, or a mountain top at location T in the image below appears as if it is at location E.) The more oblique the camera view, or the higher in altitude the feature, the more displaced the elevated data will appear. This displacement is used to advantage in MISR stereo retrievals, and this product is the primary input to Level 2 top-of-atmosphere/cloud processing. [Location=GLOBAL] [Temporal_Coverage: Start_Date=2000-02-24; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=563.2 km (cross-track); Longitude_Resolution=140.8 km (along-track); Horizontal_Resolution_Range=250 meters - < 500 meters; Temporal_Resolution=about 15 orbits/day; Temporal_Resolution_Range=about 15 orbits/day].

EOSDIS↗

NASA NTRS2018

Fire Analysis of the Thomas Fire in California Using NASA Data in a GIS

NASA's Earth Observing System Data Information System (EOSDIS) manages Earth Observation satellites and the Distributed Active Archive Centers (DAACs), where the data is stored and processed. This poster presents the use of satellite data from NASA's inventory that have the potential for use in identification and analysis of forest fire risk and subsequent after-effects within a geographic information system (GIS). The challenge is that Earth Observation data is complicated. There is plenty of data available, however, the science teams have had a "top-down" approach: define what it is you are trying to study -select a set of satellite(s) and sensor(s), and drill down for the data.

Worldview↗

NASA NTRS2018

Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse Use Cases in NASA's EOSDIS

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Driven by rapidly rising data volumes, EOSDIS is migrating to a cloud computing based archive over the next few years. Although this simplifies data management somewhat, the main aim is to provide the data in an environment where end users can bring their analysis to the data rather than attempting to download and manage ever-increasing volumes. To that end, a cloud-based analysis platform is being constructed to enable data transformations, analyses and visualization without egressing the data from the cloud. In this endeavor, we expect a wide variety of users, algorithms and use cases. Consequently, the architecture of this cloud analytics platform is expressly designed to be based on open services, thus fostering an ecosystem that enables the efficient combination of common components with data-specific or analysis-specific components. Reviewed and approved by Andrew Mitchell, ESDIS project manager.

Cloud computing↗

NASA NTRS2019

Smart Handoffs: Preserving User Context Between Tools and Services Related to NASA's EOSDIS Data Archive

Analytics↗

NASA NTRS2012

Building a Massive Volcano Archive and the Development of a Tool for the Science Community

The Jet Propulsion Laboratory has traditionally housed one of the world's largest databases of volcanic satellite imagery, the ASTER Volcano Archive (~10Tb), making these data accessible online for public and scientific use. However, a series of changes in how satellite imagery is housed by the Earth Observing System (EOS) Data Information System has meant that JPL has been unable to systematically maintain its database for the last several years. We have provided a fast, transparent, machine-to-machine client that has updated JPL's database and will keep it current in near real-time. The development of this client has also given us the capability to retrieve any data provided by NASA's Earth Observing System Clearinghouse (ECHO) that covers a volcanic event reported by U.S. Air Force Weather Agency (AFWA). We will also provide a publicly available tool that interfaces with ECHO that can provide functionality not available in any of ECHO's Earth science discovery tools.

remote sensing↗

NASA NTRS2003

TRMM Data from the Goddard Earth Sciences (GES) DISC DAAC: Tropical Rainfall Measuring Mission (TRMM)

Tropical rainfall affects the lives and economies of a majority of the Earth's population. Tropical rain systems, such as hurricanes, typhoons, and monsoons, are crucial to sustaining the livelihoods of those living in the tropics. Excess rainfall can cause floods and great property and crop damage, whereas too little rainfall can cause drought and crop failure. The latent heat release during the process of precipitation is a major source of energy that drives the atmospheric circulation. This latent heat can intensify weather systems, affecting weather thousands of kilometers away, thus making tropical rainfall an important indicator of atmospheric circulation and short-term climate change. The Tropical Rainfall Measuring Mission (TRMM), jointly sponsored by the National Aeronautics and Space Administration (NASA) of the United States and the National Space Development Agency (NASDA) of Japan, provides visible, infrared, and microwave observations of tropical and subtropical rain systems. The satellite observations are complemented by ground radar and rain gauge measurements to validate satellite rain estimation techniques. Goddard Space Flight Center's involvement includes the observatory, four instruments, integration and testing of the observatory, data processing and distribution, and satellite operations. TRMM has a design lifetime of three years. It is currently in its fifth year of operation. Data generated from TRMM and archived at the GES DAAC are useful not only for hydrologists, atmospheric scientists, and climatologists, but also for the health community studying infectious diseases, the ocean research community, and the agricultural community.

Source record↗

NASA NTRSDate not supplied

Modeling Atmospheric Science Knowledge from Research Publications

NASA Earth Science Data Centers contain enormous amounts of remote sensing digital data. It is often a significant challenge for users to find data suitable for their research topic in these vast archives. One of the approaches is the usage-driven dataset discovery, where users seek publications on projects similar to their intended study. For this approach to be effective, users need a clear connection between the underlying data in the publications and the study objectives; this is not often apparent to non-expert users. Tools and methodologies that can help facilitate and organize these connections are therefore valuable for creating improved knowledge mappings, which can be further used by search engines to suggest data or publications best tailored to a user’s specific research goal. As an illustration of these challenges, in this work we focus on the atmospheric chemistry processes related to Earth environmental impacts such as ozone depletion, aerosols, smog formation, acid rain, and radiative forcing. We further limit our study to publications that use data from the Microwave Limb Sounder (MLS) instrument flown on the Aura Earth Observing System. To create knowledge representations of science carried out in these publications, we use existing ontologies such as the Global Change Master Directory (GCMD) and Semantic Web for Earth and Environmental Terminology (SWEET). These ontologies together encompass term dictionaries that include measured variables, names of molecules or radicals, mission and instrument names, locations, action words, among many others. Based on these terms acknowledge graph database was populated with the terms retrieved from scientific publications that study atmospheric chemistry. These databases can be used to further enhance the automation of knowledge discovery and facilitate machine learning and artificial intelligence algorithms or applications. These tools and methods can also be extended to apply to content from other related Earth science domains.

Irina Gerasimov↗

NASA NTRS2006

Aerocapture Technology Development for Planetary Science - Update

Within NASA's Science Mission Directorate is a technological program dedicated to improving the cost, mass, and trip time of future scientific missions throughout the Solar System. The In-Space Propulsion Technology (ISPT) Program, established in 2001, is charged with advancing propulsion systems used in space from Technology Readiness Level (TRL) 3 to TRL6, and with planning activities leading to flight readiness. The program's content has changed considerably since inception, as the program has refocused its priorities. One of the technologies that has remained in the ISPT portfolio through these changes is Aerocapture. Aerocapture is the use of a planetary body's atmosphere to slow a vehicle from hyperbolic velocity to a low-energy orbit suitable for science. Prospective use of this technology has repeatedly shown huge mass savings for missions of interest in planetary exploration, at Titan, Neptune, Venus, and Mars. With launch vehicle costs rising, these savings could be the key to mission viability. This paper provides an update on the current state of the Aerocapture technology development effort, summarizes some recent key findings, and highlights hardware developments that are ready for application to Aerocapture vehicles and entry probes alike. Description of Investments: The Aerocapture technology area within the ISPT program has utilized the expertise around NASA to perform Phase A-level studies of future missions, to identify technology gaps that need to be filled to achieve flight readiness. A 2002 study of the Titan Explorer mission concept showed that the combination of Aerocapture and a Solar Electric Propulsion system could deliver a lander and orbiter to Titan in half the time and on a smaller, less expensive launch vehicle, compared to a mission using chemical propulsion for the interplanetary injection and orbit insertion. The study also identified no component technology breakthroughs necessary to implement Aerocapture on such a mission. Similar studies of Aerocapture applications at Neptune, Venus, and Mars were studied in 2003 through 2005. All showed significant performance improvements for the missions studied. Findings from these studies were used to guide the technology development tasks originally solicited in a 2002 NASA ROSS Research Announcement. The tasks are now in their final year and have provided numerous improvements in modeling and hardware, for use in proposals or new mission starts. Major Accomplishments: Since validation of the Aerocapture maneuver requires a space flight, ground developments have focused on modeling and environment prediction, materials, and sensors. Lockheed Martin has designed and built a 2-meter Carbon-Carbon aeroshell "hot structure." The article utilizes co-cured stiffening ribs and advanced insulation to achieve large scale, and up to a 40% reduction in areal density over the Genesis probe construction. This concept would be an efficient solution for probes that experience heat rates near 800-1000 W/cm(exp 2), such as at Venus and Earth. Applied Research Associates has extensively tested a family of efficient ablative TPS materials that provide solutions for a range of heating conditions. These materials are being applied to high-temperature structures built by ATK Space Systems, led by Langley Research Center. One-meter aeroshells will be thermally tested to validate construction and demonstrate higher bondline temperatures, which can lead to mass savings of up to 30% over traditional heatshields. Ames Research Center has developed aeroshell instrumentation that could measure environmental conditions and material performance during atmospheric entry. Instruments to measure TPS recession, heat flux, and catalycity could be combined with traditional sensors to provide a "plug-and-play" system for minimal mass and power, that would acquire flight data for model improvement and risk reduction on future missions. Improved atmospheric and aerothermodynamic models ha also been a major focus of the program. Next Steps: Aerocapture is one of five technologies in competition for a flight validation opportunity through the New Millennium Program. If selected, a fully autonomous vehicle will perform an Aerocapture at Earth in 2010, and flight data will be used to validate the guidance system and the TPS material for science mission infusion.

Munk, Michelle M.↗

NASA NTRS1999

Technology Assessment of High Capacity Data Storage Systems: Can We Avoid a Data Survivability Crisis?

In a recent address at the California Science Center in Los Angeles, Vice President Al Gore articulated a Digital Earth Vision. That vision spoke to developing a multi-resolution, three-dimensional visual representation of the planet into which we can roam and zoom into vast quantities of embedded geo-referenced data. The vision was not limited to moving through space, but also allowing travel over a time-line, which can be set for days, years, centuries, or even geological epochs. A working group of Federal Agencies, developing a coordinated program to implement the Vice President's vision, developed the definition of the Digital Earth as a visual representation of our planet that enables a person to explore and interact with the vast amounts of natural and cultural geo-referenced information gathered about the Earth. One of the challenges identified by the agencies was whether the technology existed that would be available to permanently store and deliver all the digital data that enterprises might want to save for decades and centuries. Satellite digital data is growing by Moore's Law as is the growth of computer generated data. Similarly, the density of digital storage media in our information-intensive society is also increasing by a factor of four every three years. The technological bottleneck is that the bandwidth for transferring data is only growing at a factor of four every nine years. This implies that the migration of data to viable long-term storage is growing more slowly. The implication is that older data stored on increasingly obsolete media are at considerable risk if they cannot be continuously migrated to media with longer life times. Another problem occurs when the software and hardware systems for which the media were designed are no longer serviced by their manufacturers. Many instances exist where support for these systems are phased out after mergers or even in going out of business. In addition, survivability of older media can suffer from physical breakdown of components (e.g. tapes simply lose their magnetic properties after a long time in storage). As a result, a potential data survivability crisis is emerging. The scale of the crisis is comparable to that facing the Social Security System. Sometime in one or two decades, the exponential growth of data will become so great that many enterprises will not be able to migrate through their data to more permanent media during the lifetime of the media on which it resides. This will result in significant losses of data and their resultant impacts. To avoid this crisis, we need to plan and devote greater financial and intellectual resources are needed for the development and refinement of new storage media and migration technologies in order to preserve all data any organization determines worth saving permanently. This talk will explore technological solutions and suggested recommendations to address this technological data crisis.

Halem, Milton↗

NASA NTRS1977

Attitude determination system for Seasat-A mission

The attitude determination (AD) system hardware of Seasat-A consists of two sun aspect sensors, two infrared (IR) horizon scanners, related electronics, telemetry, and science sensors. The earth horizon pulse signal sensing is accomplished by a variable thresholding method intended to self-adjust for earth radiance variations. The use of sun sensors rather than star trackers for yaw determination calls for estimation rather than measurement of yaw attitude about 50% of mission time. Features of the yaw interpolation algorithm include adaptability and automatic incorporation of time-varying confidence levels of various data sources. The performance accuracy of the AD system was assessed, where it is found that the most important error source is the variation in earth atmospheric radiance. Analysis of science instrument sensitivity to AD uncertainties was performed, and the most sensitive instrument, the scatterometer system, shows only gradual degradation of performance in the vicinity of the AD accuracy requirement.

Ohtakay, H.↗

NASA NTRS2004

Initial Processing of Infrared Spectral Data

The Atmospheric Infrared Spectrometer (AIRS) Science Processing System is a collection of computer programs, denoted product generation executives (PGEs), for processing the readings of the AIRS suite of infrared and microwave instruments orbiting the Earth aboard NASA's Aqua spacecraft. Following from level 0 (representing raw AIRS data), the PGEs and their data products are denoted by alphanumeric labels (1A, 1B, and 2) that signify the successive stages of processing. Once level-0 data have been received, the level-1A PGEs begin processing, performing such basic housekeeping tasks as ensuring that all the Level-0 data are present and ordering the data according to observation times. The level-1A PGEs then perform geolocation-refinement calculations and conversions of raw data numbers to engineering units. Finally, the level-1A data are grouped into packages, denoted granules, each of which contain the data from a six-minute observation period. The granules are forwarded, along with calibration data, to the Level-1B PGEs for processing into calibrated, geolocated radiance products. The Level-2 PGEs, which are not yet operational, are intended to process the level-1B data into temperature and humidity profiles, and other geophysical properties.

De Picciotto, Solomon↗

DOE OSTI2021

The International Soil Moisture Network: serving Earth system science for over a decade

In 2009, the International Soil Moisture Network (ISMN) was initiated as a community effort, funded by the European Space Agency, to serve as a centralised data hosting facility for globally available in situ soil moisture measurements (Dorigo et al., 2011b, a). The ISMN brings together in situ soil moisture measurements collected and freely shared by a multitude of organisations, harmonises them in terms of units and sampling rates, applies advanced quality control, and stores them in a database. Users can freely retrieve the data from this database through an online web portal (https://ismn.earth/en/, last access: 28 October 2021). Meanwhile, the ISMN has evolved into the primary in situ soil moisture reference database worldwide, as evidenced by more than 3000 active users and over 1000 scientific publications referencing the data sets provided by the network. As of July 2021, the ISMN now contains the data of 71 networks and 2842 stations located all over the globe, with a time period spanning from 1952 to the present. The number of networks and stations covered by the ISMN is still growing, and approximately 70% of the data sets contained in the database continue to be updated on a regular or irregular basis. The main scope of this paper is to inform readers about the evolution of the ISMN over the past decade, including a description of network and data set updates and quality control procedures. A comprehensive review of the existing literature making use of ISMN data is also provided in order to identify current limitations in functionality and data usage and to shape priorities for the next decade of operations of this unique community-based data repository.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2002

Satellites as Sentinels for Environment & Health

Satellites as Sentinels for Environment & Health Remotely-sensed data and observations are providing powerful new tools for addressing human and ecosystem health by enabling improved understanding of the relationships and linkages between health-related environmental parameters and society as well as techniques for early warning of potential health problems. NASA Office of Earth Science Applications Program has established a new initiative to utilize its data, expertise, and observations of the Earth for public health applications. In this initiative, lead by Goddard Space Flight Center, remote sensing, geographic information systems, improved computational capabilities, and interdisciplinary research between the Earth and health science communities are being combined in rich collaborative efforts resulting in more rapid problem-solving, early warning, and prevention in global health issues. This presentation provides a number of recent examples of applications of advanced remote sensing and other technologies to health.and security issues related to the following: infectious and vector-borne diseases; urban, regional and global air pollution; African and Asian airborne dust; heat stress; UV radiation; water-borne disease; extreme weather; contaminant pathways (ocean, atmosphere, ice)

Maynard, Nancy G.↗

NASA NTRS2011

Improvements in Space Geodesy Data Discovery at the CDDIS

The Crustal Dynamics Data Information System (CDDIS) supports data archiving and distribution activities for the space geodesy and geodynamics community. The main objectives of the system are to store space geodesy and geodynamics related data products in a central data bank. to maintain information about the archival of these data, and to disseminate these data and information in a timely manner to a global scientific research community. The archive consists of GNSS, laser ranging, VLBI, and DORIS data sets and products derived from these data. The CDDIS is one of NASA's Earth Observing System Data and Information System (EOSDIS) distributed data centers; EOSDIS data centers serve a diverse user community and arc tasked to provide facilities to search and access science data and products. Several activities are currently under development at the CDDIS to aid users in data discovery, both within the current community and beyond. The CDDIS is cooperating in the development of Geodetic Seamless Archive Centers (GSAC) with colleagues at UNAVCO and SIO. TIle activity will provide web services to facilitate data discovery within and across participating archives. In addition, the CDDIS is currently implementing modifications to the metadata extracted from incoming data and product files pushed to its archive. These enhancements will permit information about COOlS archive holdings to be made available through other data portals such as Earth Observing System (EOS) Clearinghouse (ECHO) and integration into the Global Geodetic Observing System (GGOS) portal.

Noll, C.↗

NASA NTRS2012

An Information NEXUS: The NASA Global Hawk Link Module

The Link Module described in this paper was first developed for the NASA Global Hawk Pacific Mission (GloPAC), four flights of 30 hour duration, supporting the Aura Validation Experiment (AVE). Its second use was during the Genesis and Rapid Intensification Processes (GRIP) experiment, a NASA Earth Science field experiment to better understand how tropical storms form and develop into major hurricanes. In these missions, the Link module negotiated all communication over the high bandwidth Ku satellite link, archived al the science data from onboard experiments in a spatially enable database, routed command and control of the instruments from the Global Hawk Operations Center, and retransmitted select data sets directly to experimenters control and analysis systems. The availability of aggregated information from collections of sensors, and remote control capabilities, in real-time, is revolutionizing the way Airborne Science is being conducted. Also described is the next generation Link Module now being designed and tested to support the NASA Earth Venture missions, the Hurricane and Severe Storm Sentinel (HS3) mission, and Airborne Tropical Tropopause Experiment (ATTREX) mission. Advanced data fusion technologies being developed will further advance the Scientific productivity, flexibility and robustness of these systems. Historically, the Link module evolved from the instrument and communication interface controller used by NASA's Pathfinder and Pathfinder plus solar powered UAS's in the late 1990's. It later was expanded for use in the AIRDAS four channel scanner flown on the NASA Altus UAS, and then again to a module in the AMS twelve channel multispectral scanner flying on the NASA (Predator-b) Ikhana UAS. The current system is the next step in the evolution, a multi board system packaged in a Curtiss Wright MIL-spec, flight qualified enclosure.

Remote Sensing↗

DOE OSTI2019

SDN-NGenIA, a Software Defined Next Generation Integrated Architecture for HEP and Data Intensive Science

This project aims to enable the LHC and other leading programs in high energy physics and other global science domains funded by the DOE to operate with a new level of efficiency and control, through the development of a Next Generation Integrated Architecture (NGenIA) based on intelligent software defined network (SDN)-driven network systems coupled to high throughput applications. While the initial focus is on the challenging LHC use case, the systems and products being developed are general, and apply to many fields of data intensive science ranging from astrophysical sky surveys to bioinformatics and earth observation, as well as other organizations facing the challenges of extracting knowledge from distributed multi-petabyte data stores. A central concept in this development program is a new paradigm ‘consistent network operations’ among widely distributed computing and storage facilities, where stable high throughput flows, at set rates, cross load-balanced network paths, up to flexible high water marks that are adjusted in real time to accommodate other network traffic. The large smooth flows are launched and managed by SDN services that act in concert with the experiments’ site-resident data distribution and management systems, to meet the expanding needs of the science programs. The technical goals include the construction of autonomous, intelligent site-resident services that interact dynamically with network-resident services, and with the science programs’ principal data distribution and management tools, to request or command network resources in support of high throughput petascale to exascale workflow Specific work items include: 1. Developing compact Data Transfer Nodes (DTNs) with auto-tuning functions that support data transfer rates in the 100 Gbps to the 1 Tbps range when used with high throughput data transfer applications 2. Deep site orchestration among virtualized clusters, storage subsystems and subnets to successfully coschedule CPU, storage and network resources 3. Science-program designed site architectures, operational modes, and policy and resource usage priorities, adjudicated across multiple network domains and virtual organizations, using the orchestration functions and methods 4. Seamlessly extending end-to-end operation across both extra-site and intra-site boundaries through the use of Open vSwitch (OVS), FireQoS + next generation Science DMZs 5. Novel methods of system integration that enable granular control of extreme scale long distance transfers through flow matching of scattered source-destination address pairs to multi-domain dynamic circuits 6. Funneling massive sets of streams to DTNs at the site edge hosting petascale buffer pools configured for flows of 100 Gbps and up, exploiting state of the art data transfers 7. Adaptive scheduling based on pervasive end-to-end monitoring, including DTN or compute-node resident agents providing comprehensive end-system profiling 8. Developing unsupervised and supervised machine learning and modeling methods to optimize the workflow involving terabyte to multi-petabyte datasets.

47 OTHER INSTRUMENTATION↗