SEARCH · Engineering Papers

Results for “Data science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

How GES DISC is Opening Doors to Open Science

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of 12 NASA Earth Observing System data centers that document, process, archive, and distribute data from Earth science missions and related projects. GES DISC hosts numerous Earth Science products through various data formats and supports their access with several programming and website tools. The User Needs team at GES DISC is responsible for producing tutorials and resources to teach these tools, collecting and engaging with user feedback, and collaborating with other NASA data centers to enable science creation for everyone. This poster presentation outlines how GES DISC, and the User Needs team, are committed to enabling open science.

Christopher Battisto↗

NASA NTRSDate not supplied

How GES DISC is Opening Doors to Open Science

Christopher Battisto↗

NASA NTRS2014

Use of Semantic Technology to Create Curated Data Albums

One of the continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available online. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the data sets they need can obtain the specific files using these systems. However, in cases where researchers are interested in studying an event of research interest, they must manually assemble a variety of relevant data sets by searching the different distributed data systems. Consequently, there is a need to design and build specialized search and discover tools in Earth science that can filter through large volumes of distributed online data and information and only aggregate the relevant resources needed to support climatology and case studies. This paper presents a specialized search and discovery tool that automatically creates curated Data Albums. The tool was designed to enable key elements of the search process such as dynamic interaction and sense-making. The tool supports dynamic interaction via different modes of interactivity and visual presentation of information. The compilation of information and data into a Data Album is analogous to a shoebox within the sense-making framework. This tool automates most of the tedious information/data gathering tasks for researchers. Data curation by the tool is achieved via an ontology-based, relevancy ranking algorithm that filters out nonrelevant information and data. The curation enables better search results as compared to the simple keyword searches provided by existing data systems in Earth science.

Ramachandran, Rahul↗

NASA NTRS2014

Use of Semantic Technology to Create Curated Data Albums

One of the continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available online. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the data sets they need can obtain the specific files using these systems. However, in cases where researchers are interested in studying an event of research interest, they must manually assemble a variety of relevant data sets by searching the different distributed data systems. Consequently, there is a need to design and build specialized search and discovery tools in Earth science that can filter through large volumes of distributed online data and information and only aggregate the relevant resources needed to support climatology and case studies. This paper presents a specialized search and discovery tool that automatically creates curated Data Albums. The tool was designed to enable key elements of the search process such as dynamic interaction and sense-making. The tool supports dynamic interaction via different modes of interactivity and visual presentation of information. The compilation of information and data into a Data Album is analogous to a shoebox within the sense-making framework. This tool automates most of the tedious information/data gathering tasks for researchers. Data curation by the tool is achieved via an ontology-based, relevancy ranking algorithm that filters out non-relevant information and data. The curation enables better search results as compared to the simple keyword searches provided by existing data systems in Earth science.

Ramachandran, Rahul↗

NASA NTRS2004

Data processing pipeline with transaction-oriented data sharing

This paper makes three contributions to the area of modern science data processing systems. First, the paper describes the science data processing pipeline, developed at the Multi-mission Image Processing Lab of JPL, for transforming raw space data into high quality image data and automating the distribution of data using a high-performance file transaction service. File Exchange Interface is the file transaction service developed MIPL. Second, it presents the FEI component architecture in the are of file transaction management, security,a nd file integrity verfication. Finally, the paper presents the federated model for the FEI service to demonstrate how to create a pool of file trasaction services to support load balancing and service fallover, and simplify service management.

science data processing↗

NASA NTRS2004

Global Change Data Center: Mission, Organization, Major Activities, and 2003 Highlights

Rapid, efficient access to Earth sciences data from satellites and ground validation stations is fundamental to the nation's efforts to understand the effects of global environmental changes and their implications for public policy. It becomes a bigger challenge in the future when data volumes increase from current levels to terabytes per day. Demands on data storage, data access, network throughput, processing power, and database and information management are increased by orders of magnitude, while budgets remain constant and even shrink.The Global Change Data Center's (GCDC) mission is to develop and operate data systems, generate science products, and provide archival and distribution services for Earth science data in support of the U.S. Global Change Program and NASA's Earth Sciences Enterprise. The ultimate product of the GCDC activities is access to data to support research, education, and public policy.

FROM↗

NASA NTRS1994

Ion drift meter research

The final activity period for the DE project has been particularly productive. This period has seen the final delivery of geophysical data sets to the National Space Science Data Center, the granting of three Ph.D. degrees from cumulative work on the project, the operation of automatic data access and display routines for the data, and an increased effort in research and publication of the data. As before the research activities, largely devoted to studies involving the dynamics of the ionosphere, utilize data from the IDM and the RPA and thus the work is not easily attributable to one or the other of these separately funded efforts. In this final report we provide brief descriptions of the work accomplished in the final phase of the program. The Dynamics Explorer program has provided a significant opportunity for much of the community to participate in the data analysis and interpretation. The data, now residing in the national space science data center, are a great legacy that should continue to yield important results for many years.

Heelis, Roderick A.↗

NASA NTRS2009

HRP Data Accessibility Current Status

Overview of talk: a) Content of Human Life Science data; b) Data archive structure; c) Applicable legal documents and policies; and d) Methods for data access. Life Science Data Archive (LSDA) contains research data from NASA-funded experiments, primarily data from flight experiments and ground analog data collected at NASA facilities. Longitudinal Study of Astronaut Health (LSAH) contains electronic health records (medical data) of all astronauts, including mission data. Data are collected for clinical purposes. Clinical data are analyzed by LSAH epidemiologists to identify trends in crew health and implement changes in pre-, in-, or post-flight medical care.

Sams, Clarence↗

NASA NTRS1997

Case Study of an Object-Oriented System: EOSDIS

Mission to Planet Earth (MTPE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of MTPE. The EOS Data and Information System (EOSDIS) is another component of MTPE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at six Distributed Active Archive Centers (DAACS) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. While there are many segments in EOSDIS (e.g., flight operations, network) this case study discusses the development of the science data processing segment (SDPS). We briefly review the architecture of the system, the goals of the SDPS, and the development progress to date. This study highlights key software development challenges, experiences integrating COTS, and the difficulties of managing a complex system development effort.

Behnke, Jeanne↗

DOE OSTI2016

Cosmic shear measurements with Dark Energy Survey Science Verification data

Here, we present measurements of weak gravitational lensing cosmic shear two-point statistics using Dark Energy Survey Science Verification data. We also demonstrate that our results are robust to the choice of shear measurement pipeline, either ngmix or im3shape, and robust to the choice of two-point statistic, including both real and Fourier-space statistics. Our results pass a suite of null tests including tests for B-mode contamination and direct tests for any dependence of the two-point functions on a set of 16 observing conditions and galaxy properties, such as seeing, airmass, galaxy color, galaxy magnitude, etc. We furthermore use a large suite of simulations to compute the covariance matrix of the cosmic shear measurements and assign statistical significance to our null tests. We find that our covariance matrix is consistent with the halo model prediction, indicating that it has the appropriate level of halo sample variance. We also compare the same jackknife procedure applied to the data and the simulations in order to search for additional sources of noise not captured by the simulations. We find no statistically significant extra sources of noise in the data. We determined that the overall detection significance with tomography for our highest source density catalog is 9.7 σ. Cosmological constraints from the measurements in this work are presented in a companion paper [DES et al., Phys. Rev. D 94, 022001 (2016).].

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2018

What is Machine Learning?

Have you ever been unsure what people mean when they talk about 'machine learning'? What machines? And what are they learning? This video is for you! For more information about Machine Learning at LLNL and other Data Science projects, visit the LLNL Data Science Institute website at data-science.llnl.gov

3-D PRINTING↗

NASA NTRS2019

NASA ESDIS Standards Office

The ESDIS Standards Office (ESO) assists NASA’s Earth Science Data and Information System (ESDIS) project in formulating standards policy for NASA Earth Science Data Systems, coordinates standards activities within ESDIS, and provides technical expertise and assistance with standards related tasks within the NASA Earth Science Data System Working Groups (ESDSWG).

Enloe, Yonsook↗

DOE OSTI2019

SDN-NGenIA, a Software Defined Next Generation Integrated Architecture for HEP and Data Intensive Science

This project aims to enable the LHC and other leading programs in high energy physics and other global science domains funded by the DOE to operate with a new level of efficiency and control, through the development of a Next Generation Integrated Architecture (NGenIA) based on intelligent software defined network (SDN)-driven network systems coupled to high throughput applications. While the initial focus is on the challenging LHC use case, the systems and products being developed are general, and apply to many fields of data intensive science ranging from astrophysical sky surveys to bioinformatics and earth observation, as well as other organizations facing the challenges of extracting knowledge from distributed multi-petabyte data stores. A central concept in this development program is a new paradigm ‘consistent network operations’ among widely distributed computing and storage facilities, where stable high throughput flows, at set rates, cross load-balanced network paths, up to flexible high water marks that are adjusted in real time to accommodate other network traffic. The large smooth flows are launched and managed by SDN services that act in concert with the experiments’ site-resident data distribution and management systems, to meet the expanding needs of the science programs. The technical goals include the construction of autonomous, intelligent site-resident services that interact dynamically with network-resident services, and with the science programs’ principal data distribution and management tools, to request or command network resources in support of high throughput petascale to exascale workflow Specific work items include: 1. Developing compact Data Transfer Nodes (DTNs) with auto-tuning functions that support data transfer rates in the 100 Gbps to the 1 Tbps range when used with high throughput data transfer applications 2. Deep site orchestration among virtualized clusters, storage subsystems and subnets to successfully coschedule CPU, storage and network resources 3. Science-program designed site architectures, operational modes, and policy and resource usage priorities, adjudicated across multiple network domains and virtual organizations, using the orchestration functions and methods 4. Seamlessly extending end-to-end operation across both extra-site and intra-site boundaries through the use of Open vSwitch (OVS), FireQoS + next generation Science DMZs 5. Novel methods of system integration that enable granular control of extreme scale long distance transfers through flow matching of scattered source-destination address pairs to multi-domain dynamic circuits 6. Funneling massive sets of streams to DTNs at the site edge hosting petascale buffer pools configured for flows of 100 Gbps and up, exploiting state of the art data transfers 7. Adaptive scheduling based on pervasive end-to-end monitoring, including DTN or compute-node resident agents providing comprehensive end-system profiling 8. Developing unsupervised and supervised machine learning and modeling methods to optimize the workflow involving terabyte to multi-petabyte datasets.

47 OTHER INSTRUMENTATION↗

NASA NTRS1994

Magellan Project: Evolving enhanced operations efficiency to maximize science value

Magellan has been one of NASA's most successful spacecraft, returning more science data than all planetary spacecraft combined. The Magellan Spacecraft Team (SCT) has maximized the science return with innovative operational techniques to overcome anomalies and to perform activities for which the spacecraft was not designed. Commanding the spacecraft was originally time consuming because the standard development process was envisioned as manual tasks. The Program understood that reducing mission operations costs were essential for an extended mission. Management created an environment which encouraged automation of routine tasks, allowing staff reduction while maximizing the science data returned. Data analysis and trending, command preparation, and command reviews are some of the tasks that were automated. The SCT has accommodated personnel reductions by improving operations efficiency while returning the maximum science data possible.

Cheuvront, Allan R.↗

NASA NTRS2019

NASA EOSDIS Data Usage Metrics- Insight and Assessment

NASA's Earth Science Data and Information System (ESDIS) Project collects Earth science data usage metrics on a daily basis through the ESDIS Metrics System (EMS). This includes metrics on distribution of data products, users, data volumes, and number of files, which are key parameters in evaluating system-level performance of any of the Distributed Active Archive Centers (DAACs) encompassed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS). EOSDIS data usage metrics illustrate the benefits of making NASA data openly available to the public and show a rapid growth in data distribution to a worldwide user community. In fact, each year since 2014 the EOSDIS has distributed over one billion data files of products from EOS satellite, airborne, and in situ observations. An assessment of the long-term trends of data usage metrics and user characterization provides insights into data usability.This study will focus on describing the EMS as a metrics collection tool and will provide a comprehensive analysis of EOSDIS data usage metrics over the last 10 years. This study will also characterize the product distribution metrics by various tools and services, such as Giovanni, the Open-source Project for a Network Data Access Protocol (OPeNDAP), and subsets, to address how these tools/services have extended the usage of data in the EOSDIS collection. Data usage patterns based on discipline and study area will further assist in understanding how EOSDIS data user needs have evolved over time. Results from this study will provide useful information for the DAACs that can help them improve the functionality of their tools and services as well as more efficiently allocate the resources necessary for enhanced access and availability of their data products. Knowledge of these metrics may also benefit user discovery of data in the EOSDIS collection, promote research collaboration, and stimulate new ideas from work and research conducted using specific datasets and data collections.

Kafle, Durga N.↗

DOE OSTI2017

Weak-lensing mass calibration of redMaPPer galaxy clusters in Dark Energy Survey Science Verification data

We use weak-lensing shear measurements to determine the mean mass of optically selected galaxy clusters in Dark Energy Survey Science Verification data. In a blinded analysis, we split the sample of more than 8,000 redMaPPer clusters into 15 subsets, spanning ranges in the richness parameter $5 \leq \lambda \leq 180$ and redshift $0.2 \leq z \leq 0.8$, and fit the averaged mass density contrast profiles with a model that accounts for seven distinct sources of systematic uncertainty: shear measurement and photometric redshift errors; cluster-member contamination; miscentering; deviations from the NFW halo profile; halo triaxiality; and line-of-sight projections. We combine the inferred cluster masses to estimate the joint scaling relation between mass, richness and redshift, $\mathcal{M}(\lambda,z) \varpropto M_0 \lambda^{F} (1+z)^{G}$. We find $M_0 \equiv \langle M_{200\mathrm{m}}\,|\,\lambda=30,z=0.5\rangle=\left[ 2.35 \pm 0.22\ \rm{(stat)} \pm 0.12\ \rm{(sys)} \right] \cdot 10^{14}\ M_\odot$, with $F = 1.12\,\pm\,0.20\ \rm{(stat)}\, \pm\, 0.06\ \rm{(sys)}$ and $G = 0.18\,\pm\, 0.75\ \rm{(stat)}\, \pm\, 0.24\ \rm{(sys)}$. The amplitude of the mass-richness relation is in excellent agreement with the weak-lensing calibration of redMaPPer clusters in SDSS by Simet et al. (2016) and with the Saro et al. (2015) calibration based on abundance matching of SPT-detected clusters. Our results extend the redshift range over which the mass-richness relation of redMaPPer clusters has been calibrated with weak lensing from $z\leq 0.3$ to $z\leq0.8$. Calibration uncertainties of shear measurements and photometric redshift estimates dominate our systematic error budget and require substantial improvements for forthcoming studies.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

Wide-Field Lensing Mass Maps from Dark Energy Survey Science Verification Data

We present a mass map reconstructed from weak gravitational lensing shear measurements over 139 deg 2 from the Dark Energy Survey science verification data. The mass map probes both luminous and dark matter, thus providing a tool for studying cosmology. We also find good agreement between the mass map and the distribution of massive galaxy clusters identified using a red-sequence cluster finder. Potential candidates for superclusters and voids are identified using these maps. We measure the cross-correlation between the mass map and a magnitude-limited foreground galaxy sample and find a detection at the 6.8σ level with 20 arc min smoothing. These measurements are consistent with simulated galaxy catalogs based on N-body simulations from a cold dark matter model with a cosmological constant. This suggests low systematics uncertainties in the map. Finally, we summarize our key findings in this Letter; the detailed methodology and tests for systematics are presented in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2004

Harnessing Systems Engineering Methodology in Using Earth Science Research Data for Real Applications

For the last three decades, Earth science remote sensing technologies have been providing an enormous amount of useful data and information serving to broaden our understanding of the home planet as a system. NASA's Earth science program has deployed about 18 complex satellites and is in the process of defining and launching multiple observing systems in this decade. At the same time, the European Community and many other countries such as Russia, France, India, Japan, and China have also significantly contributed to Earth science research. To date, the majority of such efforts have concentrated on expanding our scientific understanding of the multiple nonlinear and chaotic processes of Earth's behavior. In recent years, legislators and stakeholders have put serious pressure on the science community to devote more attention to making use of scientific results for societal benefit. For instance, there are a number of areas such as energy forecasting, aviation safety, agricultural efficiency, disaster management, air quality and public health that can directly take advantage of Earth science results to analyze and predict large scale problems and conditions. This is becoming even more important now that we live in a global economy interconnected via the internet and transportation systems; regional environmental conditions can have far reaching impact across continental boundaries. These factors dictate requirements for global data that can help us assess and control the devastating problems of famine, water resources, wildfires, human health and more. To do this requires a serious, organized, and systematic approach that transfers fundamental research products to the applied sciences domain. This paper presents a systems engineering and management process that can effectively make such transfer of data to the user community. Examples are presented on how the above decision making framework can help in solving critical problems such as the spread of vector borne diseases, forecasts of harmful algal blooms as well as forest fires and wildfires, and the intercontinental transport of dust storms and pollution.

Habib, Shahid↗