SEARCH · Engineering Papers

Results for “web archives”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

NASA NTRS2004

Sensor Web for Spatio-Temporal Monitoring of a Hydrological Environment

The Sensor Web is a macroinstrument concept that allows for the spatio-temporal understanding of an environment through coordinated efforts between multiple numbers and types of sensing platforms, including, in its most general form, both orbital and terrestrial and both fixed and mobile. Each of these platforms, or pods, communicates within its local neighborhood and thus distributes information to the instrument as a whole. The result of sharing and continual processing of this information among all the Sensor Web elements will result in an information flow and a global perception of and reactive capability to the environment. As illustrated, the Sensor Web concept also allows for the recursive notion of a web of webs with individual distributed instruments possibly playing the role of a single node point on a larger Sensor Web instrument. In particular, the fusion of inexpensive, yet sophisticated, commercial technology from both the computation and telecommunication revolutions has enabled the development of practical, fielded, and embedded in situ systems that have been the focus of the NASA/JPL Sensor Webs Project (http://sensorwebs.jpl.nasa.gov/). These Sensor Webs are complete systems consisting of not only the pod elements that wirelessly communicate among themselves, but also interfacing and archiving software that allows for easy use by the end-user. Previous successful deployments have included environments as diverse as coastal regions, Antarctica, and desert areas. The Sensor Web has broad implications for Earth and planetary science and will revolutionize the way experiments and missions are conceived and performed. As part of our current efforts to develop a macrointelligence within the system, we have deployed a Sensor Web at the Central Avra Valley Storage and Recovery Project (CAVSARP) facility located west of Tucson, AZ. This particular site was selected because it is ideal for studying spatio-temporal phenomena and for providing a test site for more sophisticated hydrological studies in the future.

Delin, K. A.↗

NASA NTRS2003

Enhancing and Archiving the APS Catalog of the POSS I

We have worked on two different projects: 1) Archiving the APS Catalog of the POSS I for distribution to NASA's NED at IPAC, SIMBAD in France, and individual astronomers and 2) The automated morphological classification of galaxies. We have completed archiving the Catalog into easily readable binary files. The database together with the software to read it has been distributed on DVD's to the national and international data centers and to individual astronomers. The archived Catalog contains more than 89 million objects in 632 fields in the first epoch Palomar Observatory Sky Survey. Additional image parameters not available in the original on-line version are also included in the archived version. The archived Catalog is also available and can be queried at the APS web site (URL: http://aps.umn.edu) which has been improved with a much faster and more efficient querying system. The Catalog can be downloaded as binary datafiles with the source code for reading it. It is also being integrated into the SkyQuery system which includes the Sloan Digital Sky Survey, 2MASS, and the FIRST radio sky survey. We experimented with different classification algorithms to automate the morphological classification of galaxies. This is an especially difficult problem because there are not only a large number of attributes or parameters and measurement uncertainties, but also the added complication of human disagreement about the adopted types. To solve this problem we used 837 galaxy images from nine POSS I fields at the North Galactic Pole classified by two independent astronomers for which they agree on the morphological types. The initial goal was to separate the galaxies into the three broad classes relevant to issues of large scale structure and galaxy formation and evolution: early (ellipticals and lenticulars), spirals, and late (irregulars) with an accuracy or success rate that rivals the best astronomer classifiers. We also needed to identify a set of parameters derived from the digitized images that separate the galaxies by type. The human eye can easily recognize complicated patterns in images such as spiral arms which can be spotty, blotchy affairs that are difficult for automated techniques. A galaxy image can potentially be described by hundreds of parameters, all of which may have some relation to the morphological type. In the set of initial experiments we used 624 such parameters, in two colors, blue and red. These parameters include the surface brightness and color measured at different radii, ratios of these parameters at different radii, concentration indices, Fourier transforms and wavelet decomposition coefficients. We experimented with three different classes of classification algorithms; decision trees, k-nearest neighbors, and support vector machines (SVM). A range of experiments were conducted and we eventually narrowed the parameters to 23 selected parameters. SVM consistently outperformed the other algorithms with both sets of features. By combining the results from the different algorithms in a weighted scheme we achieved an overall classification success of 86%.

Humphreys, Roberta M.↗

NASA NTRS2009

Approach to Managing MeaSURES Data at the GSFC Earth Science Data and Information Services Center (GES DISC)

A major need stated by the NASA Earth science research strategy is to develop long-term, consistent, and calibrated data and products that are valid across multiple missions and satellite sensors. (NASA Solicitation for Making Earth System data records for Use in Research Environments (MEaSUREs) 2006-2010) Selected projects create long term records of a given parameter, called Earth Science Data Records (ESDRs), based on mature algorithms that bring together continuous multi-sensor data. ESDRs, associated algorithms, vetted by the appropriate community, are archived at a NASA affiliated data center for archive, stewardship, and distribution. See http://measures-projects.gsfc.nasa.gov/ for more details. This presentation describes the NASA GSFC Earth Science Data and Information Services Center (GES DISC) approach to managing the MEaSUREs ESDR datasets assigned to GES DISC. (Energy/water cycle related and atmospheric composition ESDRs) GES DISC will utilize its experience to integrate existing and proven reusable data management components to accommodate the new ESDRs. Components include a data archive system (S4PA), a data discovery and access system (Mirador), and various web services for data access. In addition, if determined to be useful to the user community, the Giovanni data exploration tool will be made available to ESDRs. The GES DISC data integration methodology to be used for the MEaSUREs datasets is presented. The goals of this presentation are to share an approach to ESDR integration, and initiate discussions amongst the data centers, data managers and data providers for the purpose of gaining efficiencies in data management for MEaSUREs projects.

Vollmer, Bruce↗

NASA NTRS2005

NASA Records Database

The NASA Records Database, comprising a Web-based application program and a database, is used to administer an archive of paper records at Stennis Space Center. The system begins with an electronic form, into which a user enters information about records that the user is sending to the archive. The form is smart : it provides instructions for entering information correctly and prompts the user to enter all required information. Once complete, the form is digitally signed and submitted to the database. The system determines which storage locations are not in use, assigns the user s boxes of records to some of them, and enters these assignments in the database. Thereafter, the software tracks the boxes and can be used to locate them. By use of search capabilities of the software, specific records can be sought by box storage locations, accession numbers, record dates, submitting organizations, or details of the records themselves. Boxes can be marked with such statuses as checked out, lost, transferred, and destroyed. The system can generate reports showing boxes awaiting destruction or transfer. When boxes are transferred to the National Archives and Records Administration (NARA), the system can automatically fill out NARA records-transfer forms. Currently, several other NASA Centers are considering deploying the NASA Records Database to help automate their records archives.

Callac, Christopher↗

NASA NTRS2019

Collecting and Processing Earth Science Data Metrics at NASA ESDIS

Since the launch of Terra satellite in 1999, the number of Earth Science remote sensing data products created and distributed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) has increased from a few hundred to nearly ten thousand. NASA's Earth Science Data and Information System (ESDIS) Metrics System (EMS) collects metrics on data ingest, archive, and distribution by its Distributed Active Archive Centers (DAACs) and the Science Investigator-led Systems (SIPS), known as Data Providers. These metrics are critical in helping NASA management as well as data producers in resource planning and gaining a wide range of knowledge of data users and data usage.EMS receives flat files, or log files of data archive, ingest, and distribution either in their raw format, such as Apache web logs, or text files of log records formatted by the Data Providers. Tens of millions of records are processed each day to extract metrics on data products, user information, distribution protocols and services, and so on. The metrics are then made available to designated parties.This presentation provides an overview of the EMS processing workflow and improvement efforts made in recent years to handle ever-increasing number of data records and new metrics requirements, discusses several key steps including mapping log records to data products and identifying user communities along with geo-distribution, and demonstrates typical metrics capabilities produced by the EMS system. Challenges and potential approaches to improve the system are also discussed.

Pan, Jianfu↗

NASA NTRSDate not supplied

New developments in space radiation research at NASA: Annotating data using a novel radiation biology ontology

Like many interdisciplinary sciences, data producers and consumers in the field of radiation biology often use a wide variety of terminology to describe their experiments and data. Furthermore, space systems and technologies are rapidly evolving, and a shared understanding and common terminology for these is also lacking. The efficiency of research organizations can be enhanced by standardizing metadata through the use of knowledge resources like ontologies. Employing a sophisticated model such as a formal ontology to standardize metadata enables automated data acquisition processes and supports more complete, accurate meta-analysis through more efficient and complete data discovery and retrieval, particularly when using multiple data sources. Thus, we developed the Radiation Biology Ontology (RBO) in order to improved radiation biology metadata uniformity and transparency. We used open-source software (the Ontology Development Kit, Protégé and WebProtégé) and worked within the OBO Foundry framework, which includes a set of ontology development principles and practices for ontology consistency, uniformity, and accountability. The RBO has now been incorporated into two radiation research data repositories, NASA’s GeneLab omics database (https://genelab.nasa.gov), and the European Commission STORE database (https://www.storedb.org/). Continuous build integration tools allowed our international RBO collaboration to be more efficient and focus its efforts on semantic model design. Currently, the RBO contains over 300 annotated classes and individuals specific to the study of radiation on biological systems, as well as imports of many additional classes from other OBO Foundry ontologies that relate to and/or provide context for these RBO entities. We publish the RBO through the OBO Foundry, so that it is available for browsing, download, and querying through NCBI Bioportal web site and application programming interface. The NASA Ames Life Science Data Archive (ALSDA) is also in the process of adopting use of the RBO, taking NASA one step closer to a knowledge-based system for space biology data. It is our hope that the global communities of radiation research Investigators, data curators and data analysts can similarly leverage the RBO and will contribute to its further development.

radiation↗

NASA NTRSDate not supplied

Increasing Accessibility of the Runs-on-Request Metadata, Data, and Services at the Community Coordinated Modeling Center

Space weather models are essential to our ability to understand and predict space weather events. For over 20 years, the Community Coordinated Modeling Center (CCMC, https://ccmc.gsfc.nasa.gov) has been providing transformative tools and platforms for hosting space weather models and associated services, free and open to anyone interested in studying space weather. Runs-on-Request system (ROR) is one of the popular services at CCMC that permits researchers and other end-users to exercise cutting-edge hosted heliophysics and space weather models using a simple web interface, as well as collaborate on an extensive and continuously growing archive of over 28,000 model run results. Similar to other projects at CCMC, ROR has grown as a community project that strives to be open and transparent to its users. In this poster, we discuss some of our recent efforts to further expose ROR data, metadata, and services to the end users through both custom and community-developed access protocols. We also discuss how in-house science support provided by the CCMC team plays a paramount role in making ROR data and services truly accessible by the community.

Maksym Petrenko↗

NASA NTRS2009

Scientific Benefits of Space Science Models Archiving at Community Coordinated Modeling Center

The Community Coordinated Modeling Center (CCMC) hosts a set of state-of-the-art space science models ranging from the solar atmosphere to the Earth's upper atmosphere. CCMC provides a web-based Run-on-Request system, by which the interested scientist can request simulations for a broad range of space science problems. To allow the models to be driven by data relevant to particular events CCMC developed a tool that automatically downloads data from data archives and transform them to required formats. CCMC also provides a tailored web-based visualization interface for the model output, as well as the capability to download the simulation output in portable format. CCMC offers a variety of visualization and output analysis tools to aid scientists in interpretation of simulation results. During eight years since the Run-on-request system became available the CCMC archived the results of almost 3000 runs that are covering significant space weather events and time intervals of interest identified by the community. The simulation results archived at CCMC also include a library of general purpose runs with modeled conditions that are used for education and research. Archiving results of simulations performed in support of several Modeling Challenges helps to evaluate the progress in space weather modeling over time. We will highlight the scientific benefits of CCMC space science model archive and discuss plans for further development of advanced methods to interact with simulation results.

Kuznetsova, Maria M.↗

DOE OSTI2021

Archival, anonymization and presentation of HTCondor logs with GlideinMonitor

GlideinWMS is a pilot framework to provide uniform and reliable HTCondor clusters using heterogeneous and unreliable resources. The Glideins are pilot jobs that are sent to the selected nodes, test them, set them up as desired by the user jobs, and ultimately start an HTCondor schedd to join an elastic pool. These Glideins collect information that is very useful to evaluate the health and efficiency of the worker nodes and invaluable to troubleshoot when something goes wrong. This data, including local stats, the results of all the tests, and the HTCondor log files, is packed and sent to the GlideinWMS Factory. To access this information, developers and troubleshooters must exchange emails with Factory operators and dig manually into files. Furthermore, these files contain also information like email and IP addresses, and user IDs, that we want to protect and limit access to. GlideinMonitor is a Web application to make these logs more accessible and useful: it organizes the logs in an efficient compressed archive; it allows to search, unpack, and inspect them, all in a convenient and secure Web interface; via plugins like the log anonymizer, it can redact protected information preserving the parts useful for troubleshooting.

Mambelli, Marco↗

NASA NTRS2001

Ontolology Negotiation Between Scientific Archives

This paper describes an approach to ontology negotiation between information agents. Ontologies are declarative (data driven) expressions of an agent's "world": the objects, operations, facts, and rules that constitute the logical space within which an agent performs. Ontology negotiation enables agents to cooperate in performing a task, even if they are based on different ontologies. 'Me process allows agents to discover ontology conflicts and then, though incremental interpretation, clarification, and explanation, establish a common basis for communicating with each other. The need for ontology negotiation stems from the proliferation of information sources and of agents with widely varying specialty expertise. The unmanageability of massive amounts of web-based information is already becoming apparent. It is starting to have an impact on professions that rely on distributed archived information. If the expansion continues at its present rate without an ontology negotiation process being introduced, there will soon be no way to ensure the accuracy and completeness of information that scientists obtain from sources other than their own experiments. Ontology negotiation is becoming increasingly recognized as a crucial element of scalable agent technology. This is because agents, by their very nature, are supposed to operate with a fair amount of autonomy and independence from their end-users. Part of this independence is the ability to enlist other agents for help in performing a task (such as locating information on the web). The agents enlisted for help may be "owned" by a different end-user or organization (such as a document archive), and there is no guarantee that they will use the same terminology or understand the same concepts (objects, operators, theorems, rules) as the recruiting agent. For NASA, the need for ontology negotiation arises at the boundaries between scientific disciplines. For example: modeling the effects of global warming might involve knowledge about imaging, climate analysis, ecology, demographics, industrial economics, and biology. The need for ontology negotiation also arises at the boundaries between scientific programs. For example, a Principal Investigator may want to use information from a previous mission to complement downloads from the instruments currently deployed.

Bailin, Sidney C.↗

NASA NTRS2007

The Sensor Management for Applied Research Technologies (SMART) Project

NASA seeks on-demand data processing and analysis of Earth science observations to facilitate timely decision-making that can lead to the realization of the practical benefits of satellite instruments, airborne and surface remote sensing systems. However, a significant challenge exists in accessing and integrating data from multiple sensors or platforms to address Earth science problems because of the large data volumes, varying sensor scan characteristics, unique orbital coverage, and the steep "learning curve" associated with each sensor, data type, and associated products. The development of sensor web capabilities to autonomously process these data streams (whether real-time or archived) provides an opportunity to overcome these obstacles and facilitate the integration and synthesis of Earth science data and weather model output.

Goodman, Michael↗

NASA NTRS2006

Cassini Archive Tracking System

The Cassini Archive Tracking System (CATS) is a computer program that enables tracking of scientific data transfers from originators to the Planetary Data System (PDS) archives. Without CATS, there is no systematic means of locating products in the archive process or ensuring their completeness. By keeping a database of transfer communications and status, CATS enables the Cassini Project and the PDS to efficiently and accurately report on archive status. More importantly, problem areas are easily identified through customized reports that can be generated on the fly from any Web-enabled computer. A Web-browser interface and clearly defined authorization scheme provide safe distributed access to the system, where users can perform functions such as create customized reports, record a transfer, and respond to a transfer. CATS ensures that Cassini provides complete science archives to the PDS on schedule and that those archives are available to the science community by the PDS. The three-tier architecture is loosely coupled and designed for simple adaptation to multimission use. Written in the Java programming language, it is portable and can be run on any Java-enabled Web server.

Conner, Diane↗

NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly.

Lynnes, Christopher↗

NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

Lynnes, Christopher↗

NASA NTRS2015

Application of Data Cubes for Improving Detection of Water Cycle Extreme Events

As part of an ongoing NASA-funded project to remove a longstanding barrier to accessing NASA data (i.e., accessing archived time-step array data as point-time series), for the hydrology and other point-time series-oriented communities, "data cubes" are created from which time series files (aka "data rods") are generated on-the-fly and made available as Web services from the Goddard Earth Sciences Data and Information Services Center (GES DISC). Data cubes are data as archived rearranged into spatio-temporal matrices, which allow for easy access to the data, both spatially and temporally. A data cube is a specific case of the general optimal strategy of reorganizing data to match the desired means of access. The gain from such reorganization is greater the larger the data set. As a use case of our project, we are leveraging existing software to explore the application of the data cubes concept to machine learning, for the purpose of detecting water cycle extreme events, a specific case of anomaly detection, requiring time series data. We investigate the use of support vector machines (SVM) for anomaly classification. We show an example of detection of water cycle extreme events, using data from the Tropical Rainfall Measuring Mission (TRMM).

water cycle extreme events↗

DOE OSTI2020

Data, Photographs, Videos, and Information for the Niwot Ridge Subalpine Forest (US-NR1) AmeriFlux site

This data package contains data and information about the operation of the Niwot Ridge Subalpine Forest AmeriFlux site (US-NR1) between Nov 1998 to the present (2020). This data archive supplements the primary 30-min data storage for the US-NR1 data (i.e., https://doi.org/10.17190/AMF/1246088) by providing the following: (i) five-minute statistics (means, variances, covariances) of all data measured by the data system between Nov 1998 and September 2020 in netCDF format, (ii) CSV data files saved within the memory of the CR23X data loggers (as well as an archive of the data logger programs), (iii) an archive of previous 30-min ASCII data versions of the US-NR1 AmeriFlux data and information related to each data release (a replica of what can be found at http://urquell.colorado.edu/data_ameriflux/), (iv) a web calendar (in HTML format) documenting activity at the site (a replica of http://urquell.colorado.edu/calendar/), (v) photos (over 15,000) and video taken at the site between years 2001 and present day (2020), and (vi) several auxiliary datasets, primary related to trees near the site, soil moisture and soil temperature, and subcanopy radiation data. The data package is setup so that the web calendar, photos, and electronic logbook can be easily accessed on a local computer using a web browser. The provided data files are in either netCDF, CSV, ASCII, or MATLAB format. To obtain a better understanding about the archive, please start by reading the PDF: README_ESS_DIVE_USNR1_readme_first.pdf.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2015

The Open Data Repositorys Data Publisher

Data management and data publication are becoming increasingly important components of researcher's workflows. The complexity of managing data, publishing data online, and archiving data has not decreased significantly even as computing access and power has greatly increased. The Open Data Repository's Data Publisher software strives to make data archiving, management, and publication a standard part of a researcher's workflow using simple, web-based tools and commodity server hardware. The publication engine allows for uploading, searching, and display of data with graphing capabilities and downloadable files. Access is controlled through a robust permissions system that can control publication at the field level and can be granted to the general public or protected so that only registered users at various permission levels receive access. Data Publisher also allows researchers to subscribe to meta-data standards through a plugin system, embargo data publication at their discretion, and collaborate with other researchers through various levels of data sharing. As the software matures, semantic data standards will be implemented to facilitate machine reading of data and each database will provide a REST application programming interface for programmatic access. Additionally, a citation system will allow snapshots of any data set to be archived and cited for publication while the data itself can remain living and continuously evolve beyond the snapshot date. The software runs on a traditional LAMP (Linux, Apache, MySQL, PHP) server and is available on GitHub (http://github.com/opendatarepository) under a GPLv2 open source license. The goal of the Open Data Repository is to lower the cost and training barrier to entry so that any researcher can easily publish their data and ensure it is archived for posterity.

Astrobiology data↗

NASA NTRS2010

Life Sciences Data Archives (LSDA) in the Post-Shuttle Era

Now, more than ever before, NASA is realizing the value and importance of their intellectual assets. Principles of knowledge management-the systematic use and reuse of information, experience, and expertise to achieve a specific goal-are being applied throughout the agency. LSDA is also applying these solutions, which rely on a combination of content and collaboration technologies, to enable research teams to create, capture, share, and harness knowledge to do the things they do well, even better. In the early days of spaceflight, space life sciences data were collected and stored in numerous databases, formats, media-types and geographical locations. These data were largely unknown/unavailable to the research community. The Biomedical Informatics and Health Care Systems Branch of the Space Life Sciences Directorate at JSC and the Data Archive Project at ARC, with funding from the Human Research Program through the Exploration Medical Capability Element, are fulfilling these requirements through the systematic population of the Life Sciences Data Archive. This project constitutes a formal system for the acquisition, archival and distribution of data for HRP-related experiments and investigations. The general goal of the archive is to acquire, preserve, and distribute these data and be responsive to inquiries for the science communities. Information about experiments and data, as well as non-attributable human data and data from other species' are available on our public Web site http://lsda.jsc.nasa.gov. The Web site also includes a repository for biospecimens, and a utilization process. NASA has undertaken an initiative to develop a Shuttle Data Archive repository. The Shuttle program is nearing its end in 2010 and it is critical that the medical and research data related to the Shuttle program be captured, retained, and usable for research, lessons learned, and future mission planning. Communities of practice are groups of people who share a concern or a passion for something they do, and learn how to do it better as they interact regularly. LSDA works with the HRP community of practice to ensure that we are preserving the relevant research and data they need in the LSDA repository. An evidence-based approach to risk management is required in space life sciences. Evidence changes over time. LSDA has a pilot project with Collexis, a new type of Web-based search engine. Collexis differentiates itself from full-text search engines by making use of thesauri for information retrieval. The high-quality search is based on semantics that have been defined in a life sciences ontology. Additionally, Collexis' matching technology is unique, allowing discovery of partially matching dicuments. Users do not have to construct a complicated (Boolean) search query, but can simply enter a free text search without the risk of getting "no results". Collexis may address these issues by virtue of its retrieval and discovery capabilities across multiple repositories.

Fitts, Mary A.↗