SEARCH · Engineering Papers

Results for “Data Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43

All sources NASA NTRS DOE OSTI

DOE OSTI2022

DevOps and Data: Faster-Time-to-Knowledge through SageOps, MLOps, and DataOps

This report describes the approach, investigation, and prototyping efforts to develop an efficient, reusable methodology and reference framework for applying DevOps to disparate data for data science and data analytics at scale, based on focused application of this methodology and reusable reference framework within Sandia National Laboratories’ Pulsed Power community. Additionally, this report reviews: engineered instantiation of the reference framework used for development and production solutions, our experiences and results in using the reference framework, and future plans regarding research and development.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2017

S5: Information Technology for Science Missions

NASA Missions and Programs create a wealth of science data and information that are essential to understanding our earth, our solar system and the universe. Advancements in information technology will allow many people within and beyond the Agency to more effectively analyze and apply these data and information to create knowledge. The desired end result is to see that NASA data and science information are used to generate the maximum possible impact to the nation: to advance scientific knowledge and technological capabilities, to inspire and motivate the nation's students and teachers, and to engage and educate the public.

SBIR↗

DOE OSTI2021

An Indicator-based Approach to Sustainable Management of Natural Resources (Chapter 12)

Assessing the sustainability of natural resource management choices for agricultural and forest lands requires quantification of potential changes to a set of environmental and socioeconomic indicators selected to characterize reference scenarios relative to projected future scenarios. Correctly framing the questions with local stakeholders is a critical first step in the sustainability assessment, and the questions that can be addressed are often limited by data availability. Selecting and prioritizing indicators with stakeholders to address their needs and concerns improves the likelihood of investment in monitoring and evaluation of those indicators over time. Computational techniques for analyzing interactions between the selected indicators are inherently affected by the scales and formats of the assembled indicator datasets. Data analytics have the potential to improve understanding of the potential synergies and tradeoffs involved with meeting multiple environmental and socioeconomic goals simultaneously, but timely and appropriate indicator datasets are not always available—even in this new era of “big data.” Continued improvements in data science and data analytics are needed to broaden understanding and acceptance of problems and to provide valuable information for natural resource management. Advances in these areas will enable society to design future landscapes that meet multiple objectives, including the provisioning of agricultural and forest resources along with a variety of ecosystem services (e.g., clean water and healthy soils).

Parish, Esther↗

NASA NTRS2002

Implementation of CCSDS Lossless Data Compression in HDF

The Earth Science Data and Information System (ESDIS) handles over one terabyte (10(exp 12) bytes) of data daily and is using the Hierarchical Data Format (EDF) for data archiving and distribution. This report provides the progress and status of our effort to alleviate bandwidth and storage burdens by first performing compression studies on various science data products and later integrating the selected compression scheme into HDF.

Pen-Shu Yeh↗

NASA NTRS1991

Data compression for data archival, browse or quick-look

Soon after space and Earth science data is collected, it is stored in one or more archival facilities for later retrieval and analysis. Since the purpose of the archival process is to keep an accurate and complete record of data, any data compression used in an archival system must be lossless, and protect against propagation of error in the storage media. A browse capability for space and Earth science data is needed to enable scientists to check the appropriateness and quality of particular data sets before obtaining the full data set(s) for detailed analysis. Browse data produced for these purposes could be used to facilitate the retrieval of data from an archival facility. Quick-look data is data obtained directly from the sensor for either previewing the data or for an application that requires very timely analysis of the space or Earth science data. Two main differences between data compression techniques appropriate to browse and quick-look cases, are that quick-look can be more specifically tailored, and it must be limited in complexity by the relatively limited computational power available on space platforms.

Dozier, Jeff↗

DOE OSTI2019

Data analytics for designing Fe-9Cr steels (tensile strength)

Extended abstract/paper for 'Data analytics for designing Fe-9Cr steels (tensile strength)' - oral presentation at 2019 AIChE Annual Meeting, Orlando, FL, November 2019; Paper 411c as published in the conference proceedings, section 'Innovations in Methods of Data Science' sponsored by AIChE Applications of Data Science to Molecules and Materials, Computing Systems and Technology Division, and Data and Information Systems.

data science, steel, tensile strength OSTI Subject↗

NASA NTRSDate not supplied

Introduction to the JPSS-2 Advanced Technology Microwave Sounder (ATMS) Government Calibration Data Book (GCDB)

The third Advanced Technology Microwave Sounder (ATMS) is an instrument onboard the Joint Polar Satellite System (JPSS), JPSS-2 (renamed NOAA-21 in orbit) mission. This report is to introduce the JPSS-2 Government Calibration Data Book (J2 GCDB) for ATMS, SN 304. This J2 GCDB document contains key information generated during the calibration testing campaign that is driving parameters for radiometric performance. This document also contains supporting data that augments the calibration results. The values in this document are utilized by ATMS’s calibration packet which is, in turn, an integral component in the interpretation of science data. The calibration data in this report was collected from tests such as shelf-level testing, antenna testing, instrument thermal vacuum (TVAC) testing; satellite TVAC testing; and JPSS-2 post-launch tests. JPSS-2 was launched on November 10, 2022. In the subsequent years, the Government will release an ATMS GCDB for each JPSS mission. We expect that all public users can download these ATMS GCDBs from the NOAA operational Integrated Calibration and Validation System (ICVS) website, see more discussions below. The goal of this GCDB is to demonstrate how to characterize ATMS measurements using JPSS-2 ATMS on-orbit operational data and to provide relevant explanations. This document serves as a primary public domain reference for calibrating operational ATMS Raw Data Records (RDR) science data, as used in the current operational Interface Data Processing Segment (IDPS) system. This same RDR science data is distributed through direct broadcast (DB) to DB users for use in their ground processing systems. This J2 GCDB provides the results of the ATMS system radiometric calibration, the antenna flat reflector emissivity [1], the antenna pattern measurements, the antenna pattern corrected brightness temperature [2], the brightness temperature of the lunar disk [3], Lunar Intrusion (LI) correction algorithm [4], receiver spectral parameters, and mechanical alignment on-orbit pointing results, and the striping effect appeared significantly in S-NPP on-orbit radiance data when the data are compared to the Radiative Transfer Model (RTM) simulation in numerical weather prediction (NWP) system [5]. It also provides the parameters required for conversion of telemetry counts to engineering units, for radiometric calibration, and for antenna beam geo-location. Moreover, it provides JPSS-2 ATMS Spectral Response Functions data, some additional information related to ATMS on-orbit performance, on-orbit lunar intrusion correction parameters and Earth contamination bias, and on how to derive ATMS RDR, antenna Temperature Data Records (TDR), and Sensor Data Records (SDR). Furthermore, an introduction of NOAA operational Integrated Calibration and Validation System (ICVS) website and services is added in this J2 GCDB. This ICVS hosts a long-term monitoring system which allows to visualization and comparison of data from JPSS missions, NOAA legacy Polar Operational Environmental Satellites (POES), and Geostationary Operational Environmental Satellites (GOES). From NOAA Comprehensive Large Array-data Stewardship System (CLASS), the public users can download all JPSS ATMS data products for all JPSS missions.

Microwave Sounder↗

NASA NTRS1967

Data management study. Appendix O - Contractor data requirements science integration /SI/ Final report

Contractor data requirements for integration of scientific experiments aboard Voyager spacecraft

EXPERIMENT DESIGN↗

NASA NTRSDate not supplied

Experience in Visualizing Satellite Data With AR/VR Technologies at NASA's GES-DISC

The Goddard Earth Sciences Data and Information Services Center (GES-DISC) at NASA plays a crucial role in archiving and disseminating vital Earth science data. In this project, we leverage immersive visualization technologies to reshape engagement with and comprehension of the intricate datasets of our planet. Through the adoption of immersive technologies, we are enhancing established methods, offering novel, intuitive experiences in visualizing and interacting with Earth science data. Our goal is to contribute to the ongoing evolution of data interaction, pushing the boundaries of what's possible in data interpretation and education.

Visualization↗

NASA NTRS2009

The 2nd Generation Real Time Mission Monitor (RTMM) Development

The NASA Real Time Mission Monitor (RTMM) is a visualization and information system that fuses multiple Earth science data sources, to enable real time decisionmaking for airborne and ground validation experiments. Developed at the National Aeronautics and Space Administration (NASA) Marshall Space Flight Center, RTMM is a situational awareness, decision-support system that integrates satellite imagery and orbit data, radar and other surface observations (e.g., lightning location network data), airborne navigation and instrument data sets, model output parameters, and other applicable Earth science data sets. The integration and delivery of this information is made possible using data acquisition systems, network communication links, network server resources, and visualizations through the Google Earth virtual globe application. In order to improve the usefulness and efficiency of the RTMM system, capabilities are being developed to allow the end-user to easily configure RTMM applications based on their mission-specific requirements and objectives. This second generation RTMM is being redesigned to take advantage of the Google plug-in capabilities to run multiple applications in a web browser rather than the original single application Google Earth approach. Currently RTMM employs a limited Service Oriented Architecture approach to enable discovery of mission specific resources. We are expanding the RTMM architecture such that it will more effectively utilize the Open Geospatial Consortium Sensor Web Enablement services and other new technology software tools and components. These modifications and extensions will result in a robust, versatile RTMM system that will greatly increase flexibility of the user to choose which science data sets and support applications to view and/or use. The improvements brought about by RTMM 2nd generation system will provide mission planners and airborne scientists with enhanced decision-making tools and capabilities to more efficiently plan, prepare and execute missions, as well as to playback and review past mission data. To paraphrase the old television commercial RTMM doesn t make the airborne science, it makes the airborne science easier.

Blakeslee, Richard↗

NASA NTRS2013

Datacasting V3.0

Datacasting V3.0 provides an RSSbased feed mechanism for publishing the availability of Earth science data records in real time. It also provides a utility for subscribing to these feeds and sifting through all the items in an automatic manner to identify and download the data records that are required for a specific application. Datacasting is a method by which multiple data providers can publish the availability of new Earth science data and users download those files that meet a predefined need; for example, to only download data files related to a specific earthquake or region on the globe. Datacasting is a server-client architecture. The server-side software is used by data providers to create and publish the metadata about recently available data according to the Datacasting RSS (Really Simple Syndication) specification. The client software subscribes to the Datacasting RSS and other RSS-based feeds. By configuring filters associated with feeds, data consumers can use the client to identify and automatically download files that meet a specific need. On the client side, a Datacasting feed reader monitors the server for new feeds. The feed reader will be tuned by the user, via a graphical user interface (GUI), to examine the content of the feeds and initiate a data pull after some criteria are satisfied. The criteria might be, for example, to download sea surface temperature data for a particular region that has cloud cover less than 50% and during daylight hours. After the granule is downloaded to the client, the user will have the ability to visualize the data in the GUI. Based on the popular concept of podcasting, which gives listeners the capability to download only those MP3 files that match their preference, Earth science Datacasting will give users a method to download only the Earth science data files that are required for a particular application.

Bingham, Andrew W.↗

NASA NTRS2017

Evolution of NASA's Earth Science Digital Object Identifier Registration System

NASA's Earth Science Data and Information System (ESDIS) Project has implemented a fully automated system for assigning Digital Object Identifiers (DOIs) to Earth Science data products being managed by its network of 12 distributed active archive centers (DAACs). A key factor in the successful evolution of the DOI registration system over last 7 years has been the incorporation of community input from three focus groups under the NASA's Earth Science Data System Working Group (ESDSWG). These groups were largely composed of DOI submitters and data curators from the 12 data centers serving the user communities of various science disciplines. The suggestions from these groups were formulated into recommendations for ESDIS consideration and implementation. The ESDIS DOI registration system has evolved to be fully functional with over 5,000 publicly accessible DOIs and over 200 DOIs being held in reserve status until the information required for registration is obtained. The goal is to assign DOIs to the entire 8000+ data collections under ESDIS management via its network of discipline-oriented data centers. DOIs make it easier for researchers to discover and use earth science data and they enable users to provide valid citations for the data they use in research. Also for the researcher wishing to reproduce the results presented in science publications, the DOI can be used to locate the exact data or data products being cited.

DOI; digital object identifier; mappin↗

DOE OSTI2023

Tutorial: Lessons Learned for Behavior Analysts from Data Scientists

Big data is a computing term used to refer to large and complex data sets, typically consisting of terabytes or more of diverse data that is produced rapidly. The analysis of such complex data sets requires advanced analysis techniques with the capacity to identify patterns and abstract meanings from the vast data. The field of data science combines computer science with mathematics/statistics and leverages artificial intelligence, in particular machine learning, to analyze big data. This field holds great promise for behavior analysis, where both clinical and research studies produce large volumes of diverse data at a rapid pace (i.e., big data). This article presents basic lessons for the behavior analytic researchers and clinicians regarding integration of data science into the field of behavior analysis. We provide guidance on how to collect, protect, and process the data, while highlighting the importance of collaborating with data scientists to select a proper machine learning model that aligns with the project goals and develop models with input from human experts. Here, we hope this serves as a guide to support the behavior analysts interested in the field of data science to advance their practice or research, and helps them avoid some common pitfalls.

42 ENGINEERING↗

NASA NTRSDate not supplied

Impact of Canadian Wildfires 2023 on North Atlantic's Region Air Quality: An Analysis Using ASDC Data

Wildfires pose a growing concern in North America due to their harmful impacts on air quality and public health, with increased wildfire activity in recent years leading to widespread smoke plumes that can transcend borders. The exposure of New York City (NYC), the most populous city in North America, to Canadian wildfire smoke highlights the substantial implications for public health and urban environments. To better understand the impact of Canadian wildfires on air quality in NYC, satellite data from the NASA Atmospheric Science Data Center (ASDC) at Langley Research Center, along with ground-based measurements and atmospheric modeling results, are analyzed. NASA's Atmospheric Science Data Center (ASDC) is in the Science Directorate located at NASA'S Langley Research Center in Hampton, Virginia. The Science Directorate's Climate Science Branch, Atmospheric Composition Branch, and Chemistry and Dynamics Branch work with ASDC to study changes in the Earth and its atmosphere. ASDC projects focus on the Earth science disciplines: Radiation Budget, Clouds, Aerosols, and Tropospheric Composition. All the products chosen for this analysis are products hosted by ASDC and available for users to obtain via our services [1].

Hazem Mahmoud↗

NASA NTRS2018

Cumulus: NASA Archives in the Cloud

NASA's Earth Observing System Data and Information System (EOSDIS) houses nearly 30PBs of critical Earth Science data and with upcoming missions is expected to balloon to between 200PBs-300PBs over the next seven years. The magnitude of data collected makes it infeasible to download data and process it locally, forcing us to re-think how we store and work with earth science data. NASA has looked to the cloud to address this, building its open source Cumulus software to manage the ingest of diverse data in a wide variety of formats into the cloud and provide services to manage and access the data. In this talk, we will describe how Cumulus provides common features needed to manage a cloud archive in the realm of ingest, data stewardship, and cost-controlled distribution of science data to users and services.

EOSDIS↗

NASA NTRSDate not supplied

Development of an Improved Spatial Metadata Simplification Algorithm

The National Aeronautics and Space Administration's (NASA) Atmospheric Science Data Center (ASDC) at NASA Langley Research Center in Hampton, VA provides atmospheric science data products and services to the science community, including enhanced search and subsetting capabilities for numerous Earth Science datasets. The ASDC is the official Distributed Active Archive Center (DAAC) of record for the Tropospheric Emissions: Monitoring of Pollution (TEMPO) instrument. TEMPO is situated on a geostationary satellite positioned at a longitude near the center of the conterminous United States and focused on North America, making hourly swaths of its field of regard from east to west. Spatial metadata is an essential component for the discovery and distribution of Earth Science data. The simplified polygonal boundaries representing the archived data files ensure that any granule can be identified quickly and accurately by a geospatial query. Historically the Douglas-Peucker algorithm has been used for polygon simplification; however, due to the nature of the algorithm, a buffer must be added to the polygon before simplification to ensure pivotal points are not removed by the algorithm. This adds in additional error to the polygon simplification. ASDC’s goal is to test other methods of polyline simplification, such as Visvalingan-Whyatt and Opheim simplification alongside of Douglas-Peucker and different buffering methods, to produce less error during polygon simplification of TEMPO data swaths, and special spatial query geometries such as EPA non-attainment regions, and geopolitical boundaries.

Spatial Metadata↗

NASA NTRSDate not supplied

Use of Spatial Metadata Simplification for TEMPO

Spatial Metadata↗

DOE OSTI2017

The medical science DMZ: a network design pattern for data-intensive medical science

Abstract Objective We describe a detailed solution for maintaining high-capacity, data-intensive network flows (eg, 10, 40, 100 Gbps+) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet-filter firewalls, network intrusion-detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive datasets between research institutions over national research networks, and give 3 detailed descriptions of implemented Medical Science DMZs. Discussion The exponentially increasing amounts of “omics” data, high-quality imaging, and other rapidly growing clinical datasets have resulted in the rise of biomedical research “Big Data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large datasets. Maintaining data-intensive flows that comply with the Health Insurance Portability and Accountability Act (HIPAA) and other regulations presents a new challenge for biomedical research. We describe a strategy that marries performance and security by borrowing from and redefining the concept of a Science DMZ, a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

Peisert, Sean↗