January 2020 Meteorological Support Interface Control Working Group (MSICWG) Instrumentation, Data Format, and Networks Document
Explore the source record for details and available documents.
James C Brenton↗
SEARCH · Engineering Papers
Results for “network data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
ASDC’s Python-Based Metadata Extraction Pipeline for Suborbital Campaigns
The FAIRness of data products, especially findability and accessibility depend on rich metadata which, when extracted, can allow for proper curation. Over the past few years, the Atmospheric Science Data Center (ASDC) suborbital science support team has developed a metadata extraction pipeline to ensure the required metadata can be retrieved systematically, effectively, and efficiently to ensure the data can be used by a broad community. The development of a pipeline has presented many, but necessary, challenges to support archival and distribution of ASDC’s 30+ suborbital missions. Though sufficient metadata is provided by instrument scientists, the metadata may not be readily machine actionable due to different formats and templates. Further complicating metadata extraction, our team has found that the nature of metadata can be quite diverse given the difference in measurement types, instruments, and measurement platforms. A metadata extraction pipeline has been developed to provide an efficient, plugin-in based, method for adding new parsers, a configuration system that lets non-developers customize how files are processed, and a system for identifying and logging metadata quality issues to ensure they are readily found and addressed. The metadata extraction pipeline identifies critical pieces of metadata that are needed to promote data FAIRness, including location, file revision, measurement start/end datetime and can be easily modified to extract further information (such as variables). Given the wide-ranging datasets, the pipeline has been modified to accommodate multiple file formats, including multiple versions of ICARTT (International Consortium for Atmospheric Research on Transport and Transformation), HDF (Hierarchical Data Format), netCDF (network Common Data Form), and multiple versions of the Ames File Format. The pipeline also supports building metadata for file formats that cannot have metadata easily extracted from them, such as PDF (Portable Document Format) and GIF (Graphics Interchange Format). The pipeline has allowed our team to maintain a consistent flow of data and metadata to archival and distribution services, ensuring the ASDC meets the needs of the suborbital science community. This presentation will highlight the ASDC’s suborbital metadata extraction pipeline, its development, how it’s been modified to support data FAIRness, and plans for maintaining the pipeline and adding new features.
Abraham Porter↗
Enabling Nonlinear Manifold Projection Reduced-Order Models by Extending Convolutional Neural Networks to Unstructured Data.
Abstract not provided.
Tencer, John↗
Analysis methods to build contact networks from mobility data.
Abstract not provided.
Klise, Katherine A.↗
ncompare: A Python Package for Comparing netCDF Structures
Earth science researchers and data engineers have a common problem: they often need to compare data files to see what is different between them. A lot of time is spent developing code to test differences. When it comes to comparing multidimensional data file formats like netCDFs (Network Common Data Form), this is particularly challenging and time-consuming, since there is frequently a need to evaluate the differences between dimension sizes, variable structures, and variable attributes, especially for regression testing. Since netCDFs are widely used in Earth science — with climate models, oceanographic or atmospheric reanalyses, and observational data — improved means of evaluating netCDF files can help enable a wide range of applications. We have developed a reusable open source approach through `ncompare`, which is a Python package for comparing netCDF structures [[https://github.com/nasa/ncompare]]. The `ncompare` tool compares the structure of two Network Common Data Form (NetCDF) files at the command line. It facilitates rapid comparisons by generating a formatted display of the matching and non-matching groups, variables, and associated metadata between two NetCDF datasets. The user has the option to colorize the terminal output for ease of viewing, and `ncompare` can optionally save comparison reports in text, comma-separated value (CSV), and/or Microsoft Excel formats. Despite the availability of tools (such as ncmpidiff or nccmp) that compare the values of variables, there was not previously a readily available, Python-based tool for rapid visual comparisons of group and variable structures, attributes, and chunking. `ncompare` was developed at NASA’s Atmospheric Science Data Center (ASDC) and is a collaboration with NASA Openscapes [[https://nasa-openscapes.github.io]] mentors across 11 of NASA’s data centers. Openscapes’ overarching vision is to support scientific researchers using NASA Earthdata as they migrate their workflows to the cloud. Relevant links: - https://github.com/nasa/ncompare - https://github.com/pyOpenSci/software-submission/issues/146 - https://nasa-openscapes.github.io
Daniel Kaufman↗
Reducing Wind Tunnel Data Requirements Using Neural Networks
The use of neural networks to minimize the amount of data required to completely define the aerodynamic performance of a wind tunnel model is examined. The accuracy requirements for commercial wind tunnel test data are very severe and are difficult to reproduce using neural networks. For the current work, multiple input, single output networks were trained using a Levenberg-Marquardt algorithm for each of the aerodynamic coefficients. When applied to the aerodynamics of a 55% scale model of a U.S. Air Force/ NASA generic fighter configuration, this scheme provided accurate models of the lift, drag, and pitching-moment coefficients. Using only 50% of the data acquired during, the wind tunnel test, the trained neural network had a predictive accuracy equal to or better than the accuracy of the experimental measurements.
Ross, James C.↗
Top-of-the-Atmosphere Shortwave Flux Estimation from Satellite Observations: An Empirical Neural Network Approach Applied with Data from the A-Train Constellation
Estimates of top-of-the-atmosphere (TOA) radiative flux are essential for the understanding of Earth's energy budget and climate system. Clouds, aerosols, water vapor, and ozone (O3) are among the most important atmospheric agents impacting the Earth's shortwave (SW) radiation budget. There are several sensors in orbit that provide independent information related to these parameters. Having coincident information from these sensors is important for understanding their potential contributions. The A-train constellation of satellites provides a unique opportunity to analyze data from several of these sensors. In this paper, retrievals of cloud/aerosol parameters and total column ozone (TCO) from the Aura Ozone Monitoring Instrument (OMI) have been collocated with the Aqua Clouds and Earth's Radiant Energy System (CERES) estimates of total reflected TOA outgoing SW flux (SWF). We use these data to develop a variety of neural networks that estimate TOA SWF globally over ocean and land using only OMI data and other ancillary information as inputs and CERES TOA SWF as the output for training purposes. OMI-estimated TOA SWF from the trained neural networks reproduces independent CERES data with high fidelity. The global mean daily TOA SWF calculated from OMI is consistently within 1% of CERES throughout the year 2007. Application of our neural network method to other sensors that provide similar retrieved parameters, both past and future, can produce similar estimates TOA SWF. For example, the well-calibrated Total Ozone Mapping Spectrometer (TOMS) series could provide estimates of TOA SWF dating back to late 1978.
CERES data with high fidelity↗
Network Modeling of Complex Data Sets
We demonstrate a selection of network and machine learning techniques useful in the analysis of complex datasets, including 2-way similarity networks, Markov clustering, enrichment statistical networks, FCROS differential analysis, and random forests. We demonstrate each of these techniques on the Populus trichocarpa gene expression atlas.
Jones, Piet C.↗
Application of Neural Networks to Wind tunnel Data Response Surface Methods
The integration of nonlinear neural network methods with conventional linear regression techniques is demonstrated for representative wind tunnel force balance data modeling. This work was motivated by a desire to formulate precision intervals for response surfaces produced by neural networks. Applications are demonstrated for representative wind tunnel data acquired at NASA Langley Research Center and the Arnold Engineering Development Center in Tullahoma, TN.
Lo, Ching F.↗
Services, architectures, and protocols for space data systems
The author presents a comprehensive discussion of three major aspects of the work of the Consultative Committee for Space Data Systems (CCSDS), a worldwide cooperative effort of national space agencies. The author examines the CCSDS space data communications network concept on which the data communications facilities of future advanced orbiting systems will be based. He derives the specifications of an open communications architecture as a reference model for the development of services and protocols that support the transfer of information over space data communications networks. Detailed specifications of the communication services and information transfer protocols that have reached a high degree of maturity and stability are offered. The author also includes a complete list of currently available CCSDS standards and supporting documentation.
Helgert, Hermann J.↗
Madrid space station
The Madrid space station, operated under bilateral agreements between the governments of the United States and Spain, is described in both Spanish and English. The space station utilizes two tracking and data acquisition networks: the Deep Space Network (DSN) of the National Aeronautics and Space Administration and the Spaceflight Tracking and Data Network (STDN) operated under the direction of the Goddard Space Flight Center. The station, which is staffed by Spanish employees, comprises four facilities: Robledo 1, Cebreros, and Fresnedillas-Navalagamella, all with 26-meter-diameter antennas, and Robledo 2, with a 64-meter antenna.
Fahnestock, R. J.↗
NSSDC provides network access to key data via NDADS
The National Space Science Data Center (NSSDC) is making a growing fraction of its most customer-desirable data electronically accessible via both the local and wide area networks. NSSDC is witnessing a great increase in its data dissemination owing to this network accessibility. To provide its customers the best data accessibility, the NSSDC makes data available from a nearline, mass storage system, the NSSDC Data Archive and Dissemination Service (NDADS). The NDADS, the initial version was made available in January 1992, is a customized system of hardware and software that provides users access to the nearline data via ANONYMOUS FTP, an e-mail interface (ARMS), and a C-based software library. In January 1992, the NDADS registered 416 requests for 1,957 files. By December of 1994, NDADS had been populated with 800 gigabytes of electronically accessible data and had registered 1458 requests for 20,887 files. In this report we describe the NDADS system, both hardware and software. Later in the report, we discuss some of the lessons that were learned as a result of operating NDADS, particularly in the area of ingest and dissemination.
Behnke, Jeanne↗
Influence functions for the Delaware Oil and Gas Basin Tower Network
This data consists of influence functions created for the Delaware tower network for the purpose of calculating methane enhancements in the region. Two different footprint models were used. The Lagrangian Particle Dispersion model (LPDM, Uliasz 1993) was run from March 2020-May 2023. The model uses in-house WRF simulations for the meteorological transport as described in Barkley et al., (2023). Nine hundred (900) particles were released during each hourly period and are traced back 72 hours backwards in time. A static binning height of 110 m AGL was used to calculate surface influence. After May 2023, the LPDM simulations were phased out and replaced with influence functions created using HYSPLIT under the STILT configuration (Lin et al., 2003). The STILT influence functions use HRRR for the meteorological transport (Dowell et al., 2022). Nine hundred (900) particles were released during each hourly period and are traced back 72 hours backwards in time. A binning height of half the boundary layer height was used to calculate surface influence. Footprints are currently available from Sept 2020-December 2024 and will be updated periodically as more are generated in time. No STILT influence functions were created for tower site Maljimar, which was decommissioned in March 2022. Lat/Lon grid information for the LPDM influence functions is contained in lpdm-lat-lon-grid.nc. STILT grid information is contained within the influence functions themselves. Tower coordinates and heights can be found in tower-info.csv
Barkley, Z. R.↗
CyberGAN: Generating High-fidelity Cybersecurity Data With Generative Adversarial Networks
Machine learning for cyber defense offers the promise of detecting adversarial activity against the ground data systems managing critical space assets. A fundamental challenge facing machine learning research in cybersecurity is the lack of high-fidelity, shareable datasets for robust evaluation and testing of machine learning-based solutions. High-fidelity, real-world datasets are necessary for reliable benchmarking of nominal system behavior and malicious activity. Unfortunately, such realistic datasets of both nominal and adversarial activity are rarely shared publicly by data owners due to security and privacy concerns. Besides, the available adversarial data is sparse, which makes training models on malicious activity much harder. This situation has impeded and continues to impede the research and successful adoption of machine learning methods for cyber defense. Researchers have dealt with this problem by generating data within a low-fidelity lab environment, using classified and thus unshareable datasets, or downloading low-fidelity public datasets made available by others. We propose an innovative solution to the problem by employing machine learning methods to generate high-fidelity data. Specifically, we propose the use of Generative Adversarial Networks (GANs) to generate high-fidelity data for cybersecurity purposes. GANs have found successful image processing and natural language applications, but have not yet been investigated for cyber data generation. Our proposed approach first involves training the `discriminator' network of the GAN with a sample of real-world data consisting of malicious and nominal samples. We then use the `generator' network to generate new high-fidelity data samples consisting of an appropriate mix of malicious and nominal activity. We demonstrate applications of our architecture by generating high-fidelity cybersecurity data containing both malicious and nominal samples. We thoroughly evaluate the fidelity of our generated data using heuristics and evaluate its usefulness for machine learning applications using three different datasets. Overall, our approach results in high-fidelity, shareable datasets.
Zhang, Yuening↗
International Network of Nuclear Reaction Data Centres
This report summarizes the IAEA Technical Meeting on the International Network of Nuclear Reaction Data Centres held as a video meeting from 4 to 7 May 2021. The meeting was attended by 29 participants representing 13 cooperative Centres from eight Member States (China, Hungary, India, Japan, Korea, Russia, Ukraine and USA) and two International Organisations (NEA, IAEA) as well as a participant from Kazakhstan. A summary of the meeting is given in this report along with the conclusions and actions.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
The nasa satellite instrumentation network.
Tracking, data acquisition, command and support facility requirements for NASA worldwide Satellite Instrumentation Network
SATELLITE GROUND SUPPORT NETWORK↗
Using the Global GPS Network and Other Satellite Data to Monitor Ionospheric Total Electron Content
A globally distributed network of dual-frequency global positioning system (GPS) receivers is the primary source of data used to measure ionospheric total electron content (TEC) on global scales. Maps of TEC useful for calibrating propagation delays, or monitoring the solar-terrestrial environment, can be produced using this continuously operating network. The maps can also form the basis of a TEC calibration service for users around the world. Potential users may include single-frequency satellite altimetry missions, satellite tracking stations, and astronomical observatories.
satellite altimetry↗