Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data discovery”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

Astrometry and orbits of Nix, Kerberos, AND Hydra

We present new Hubble Space Telescope observations of three of Pluto's outer moons, Nix, Kerberos, and Hydra. This work revises previously published astrometry of Nix and Hydra from 2002 to 2003. New data from a four-month span during 2007 include observations designed to better measure the positions of Nix and Hydra. A third data set from 2010 also includes data on Nix and Hydra as well as some pre-discovery observations of Kerberos. The data were fitted using numerical point-spread function (PSF) fitting techniques to get accurate positions but also to remove the extended wings of the Pluto and Charon PSFs when working on these faint satellites. The resulting astrometric data were fitted with two-body Keplerian orbits that are useful for short-term predictions of the future positions of these satellites for stellar occultation and for guiding encounter planning for the upcoming New Horizons flyby of the Pluto system. The mutual inclinations of the satellites are all within 0.°2 of the plane of Charon's orbit. The periods for all continue to show that their orbits are near but distinct from integer period ratios relative to Charon. Based on our results, the period ratios are Hydra:Charon = 5.98094 ± 0.00001, Kerberos:Charon = 5.0392 ± 0.0003, and Nix:Charon = 3.89135 ± 0.00001. Based on period ratios alone, there is a trend of increased distance from an integer period ratio with decreasing distance from Charon. Our analysis shows that orbital uncertainties for Nix and Hydra are now low enough to permit useful stellar occultation predictions and for New Horizons encounter planning. In 2015 July, our orbits predict a position error of 60 km for Nix and 38 km for Hydra, well below other limiting errors that affect targeting. The orbit for Kerberos, however, still needs a lot of work as its uncertainty in 2015 is quite large at 22,000 km based on these data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Scaling Irregular Applications through Data Aggregation and Software Multithreading

Bioinformatics, data analytics, semantic databases, knowledge discovery are emerging high performance application areas that exploit dynamic, linked data structures such as graphs, unbalanced trees or unstructured grids. These data structures usually are very large, requiring significantly more memory than available on single shared memory systems. Additionally, these data structures are difficult to partition on distributed memory systems. They also present poor spatial and temporal locality, thus generating unpredictable memory and network accesses. The Partitioned Global Address Space (PGAS) programming model seems suitable for these applications, because it allows using a shared memory abstraction across distributed-memory clusters. However, current PGAS languages and libraries are built to target regular remote data accesses and block transfers. Furthermore, they usually rely on the Single Program Multiple Data (SPMD) parallel control model, which is not well suited to the fine grained, dynamic and unbalanced parallelism of irregular applications. In this paper we present {\bf GMT} (Global Memory and Threading library), a custom runtime library that enables efficient execution of irregular applications on commodity clusters. GMT integrates a PGAS data substrate with simple fork/join parallelism and provides automatic load balancing on a per node basis. It implements multi-level aggregation and lightweight multithreading to maximize memory and network bandwidth with fine-grained data accesses and tolerate long data access latencies. A key innovation in the GMT runtime is its thread specialization (workers, helpers and communication threads) that realize the overall functionality. We compare our approach with other PGAS models, such as UPC running using GASNet, and hand-optimized MPI code on a set of typical large-scale irregular applications, demonstrating speedups of an order of magnitude.

Aggregation↗

Building a Community-Focused Data Platform

The National Data Service (NDS) presents a broad-reaching vision of integrating discovery, publishing, and sharing of data. Metcalfe's Law states that the value of a network increases with the number of connections in the network, not the number of nodes. The NDS must develop and steward connections not only between data, as is in its vision statement, but between individuals as well. We propose that this be accomplished by explicitly developing it as a platform on which data exploration can be constructed by individual researchers.

97 MATHEMATICS AND COMPUTING↗

Challenges and Vision for Standardization of Biopolymer Data Sets for Machine Learning

Machine learning (ML) is transforming materials research, yet potential for biopolymer discovery remains constrained by fragmented data and nonstandardized reporting. Biopolymers differ significantly from synthetic polymers, requiring specialized approaches to represent their biosynthetic origins, hierarchical structures, and application-specific metrics. In this Perspective, we identify three core challenges limiting biopolymer representation: information encoding, data quality, and data sharing. We describe the most pressing issues and propose commensurate approaches to address each key challenge. Recommendations include the design and adoption of biopolymer-specific fingerprinting and representation frameworks, development of hybrid human-large language model (LLM) data extraction strategies, and expanding Findable, Accessible, Interoperable, Reusable (FAIR)-compliant repositories. We propose a robust foundation to define interoperable, high-quality data sets that capture the full context of biopolymer materials. Standardized metadata, shared ontologies, and community-driven infrastructure would enable scalable, reproducible workflows and accelerate the ML-driven development of biopolymers.

36 MATERIALS SCIENCE↗

Open-Source Science-led Development of the Atmosphere Observing System (AOS) Mission Science Data System (SDS)

The Earth System Observatory (ESO) Atmosphere Observing System (AOS) mission will provide space-based and suborbital observations of collocated cloud, dynamic, precipitation and aerosol processing leading to improved weather, air quality, and climate predictions. The AOS Science Data System (SDS) will be a system of systems developed within the Cloud to manage the research and operational processing of AOS mission orbital and suborbital sensors and curate these data for reprocessing (e.g., in near real-time or by collection) and transfer them to a NASA Distributed Active Archive Center (DAAC) for long-term storage. Further, AOS SDS will follow guidelines provided by NASA Earth Science Data Systems (ESDS) program including standard conventions for data file formats, naming, and metadata to improve data interoperability, interpretability, usability, discovery, provenance, and spatiotemporal representativeness. The AOS mission follows NASA’s lead in making a commitment to Open-Source Science (OSS) including the sharing of data, software, and knowledge in an open and timely manner. Each of the AOS SDS system components will be developed with open-source concepts including components of SDS itself as well as AOS mission algorithms. Further, the AOS SDS assumes the role to lead and facilitate OSS activities for the AOS mission. This presentation describes the framework of the AOS SDS and its integral part in facilitating OSS within the AOS mission.

David Giles↗

Open-Source Science-led Development of the AOS Mission Science Data System (SDS)

The Earth System Observatory (ESO) Atmosphere Observing System (AOS) mission will provide space-based and suborbital observations of collocated cloud, dynamic, precipitation and aerosol processing leading to improved weather, air quality, and climate predictions. The AOS Science Data System (SDS) will be a system of systems developed within the Cloud to manage the research and operational processing of AOS mission orbital and suborbital sensors and curate these data for reprocessing (e.g., in near real-time or by collection) and transfer them to a NASA Distributed Active Archive Center (DAAC) for long-term storage. Further, AOS SDS will follow guidelines provided by NASA Earth Science Data Systems (ESDS) program including standard conventions for data file formats, naming, and metadata to improve data interoperability, interpretability, usability, discovery, provenance, and spatiotemporal representativeness. The AOS mission follows NASA’s lead in making a commitment to Open-Source Science (OSS) including the sharing of data, software, and knowledge in an open and timely manner. Each of the AOS SDS system components will be developed with open-source concepts including components of SDS itself as well as AOS mission algorithms. Further, the AOS SDS assumes the role to lead and facilitate OSS activities for the AOS mission. This presentation describes the framework of the AOS SDS and its integral part in facilitating OSS within the AOS mission.

David M. Giles↗

Stewardship Best Practices for Improved Discovery and Reuse of Heterogeneous and Cross-Disciplinary Earth System Data

Some of the Earth system data products such as those from NASA airborne and field investigations (a.k.a. campaigns), are highly heterogeneous and cross-disciplinary, making the data extremely challenging to manage. For example, airborne and field campaign measurements tend to be sporadic over a period of time, with large gaps. Data products generated are of various processing levels and utilized for a wide range of inter- and cross-disciplinary research and applications. Data and derived products have been historically stored in a variety of domain-specific standard (and some non-standard) formats and in various locations such as NASA Distributed Active Archive Centers (DAACs), NASA airborne science facilities, field archives, or even individual scientists’ computer hard drives. As a result, airborne and field campaign data products have often been managed and represented differently, making it onerous for data users to find, access, and utilize campaign data. Some difficulties in discovering and accessing the campaign data originate from the incomplete data product and contextual metadata that may contain details relevant to the campaign (e.g. campaign acronym and instrument deployment locations), but tend to lack other significant information needed to understand conditions surrounding the data. Such details can be burdensome to locate after the conclusion of a campaign. Utilizing consistent terminology, essential for improved discovery and reuse, is also challenging due to the variety of involved disciplines. To help address the aforementioned challenges faced by many repositories and data managers handling airborne and field data, this presentation will describe stewardship practices developed by the Airborne Data Management Group (ADMG) within the Interagency Implementation and Advanced Concepts Team (IMPACT) under the NASA’s Earth Science Data systems (ESDS) Program.

best practices↗

The allwise motion survey and the quest for cold subdwarfs

The AllWISE processing pipeline has measured motions for all objects detected on Wide-field Infrared Survey Explorer (WISE) images taken between 2010 January and 2011 February. In this paper, we discuss new capabilities made to the software pipeline in order to make motion measurements possible, and we characterize the resulting data products for use by future researchers. Using a stringent set of selection criteria, we find 22,445 objects that have significant AllWISE motions, of which 3525 have motions that can be independently confirmed from earlier Two Micron All Sky Survey (2MASS) images, yet lack any published motions in SIMBAD. Another 58 sources lack 2MASS counterparts and are presented as motion candidates only. Limited spectroscopic follow-up of this list has already revealed eight new L subdwarfs. These may provide the first hints of a 'subdwarf gap' at mid-L types that would indicate the break between the stellar and substellar populations at low metallicities (i.e., old ages). Another object in the motion list—WISEA J154045.67–510139.3—is a bright (J ≈ 9 mag) object of type M6; both the spectrophotometric distance and a crude preliminary parallax place it ∼6 pc from the Sun. We also compare our list of motion objects to the recently published list of 762 WISE motion objects from Luhman. While these first large motion studies with WISE data have been very successful in revealing previously overlooked nearby dwarfs, both studies missed objects that the other found, demonstrating that many other nearby objects likely await discovery in the AllWISE data products.

79 ASTRONOMY AND ASTROPHYSICS↗

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗

Representation of Serendipitous Scientific Data

A computer program defines and implements an innovative kind of data structure than can be used for representing information derived from serendipitous discoveries made via collection of scientific data on long exploratory spacecraft missions. Data structures capable of collecting any kind of data can easily be implemented in advance, but the task of designing a fixed and efficient data structure suitable for processing raw data into useful information and taking advantage of serendipitous scientific discovery is becoming increasingly difficult as missions go deeper into space. The present software eases the task by enabling definition of arbitrarily complex data structures that can adapt at run time as raw data are transformed into other types of information. This software runs on a variety of computers, and can be distributed in either source code or binary code form. It must be run in conjunction with any one of a number of Lisp compilers that are available commercially or as shareware. It has no specific memory requirements and depends upon the other software with which it is used. This program is implemented as a library that is called by, and becomes folded into, the other software with which it is used.

James, Mark↗

GeoThermalCloud: Cloud Fusion of Big Data and Multi-Physics Models using Machine Learning for Discovery, Exploration and Development of Hidden Geothermal Resources

Geothermal exploration and production are challenging, expensive and risky. The GeoThermalCloud uses Machine Learning to predict the location of hidden geothermal resources. This submission includes a training dataset for the GeoThermalCloud neural network. Machine Learning for Discovery, Exploration, and Development of Hidden Geothermal Resources.

15 GEOTHERMAL ENERGY↗

Reduction and analysis of photometric data on Comet Halley

The discovery that periodic variations in the brightness of Comet Halley were characterized by two unrelated frequencies implies that the nucleus is in a complex state of rotation. It either nutates as a result of the random addition of small torque perturbations accumulated over many perihelion passages, or the jet activity torques are so strong that it precesses wildly at each perihelion passage. To diagnose the state of nuclear rotation, researchers began a program to acquire photometric time series of the comet as it recedes from the sun. The intention is to observe the decay of the comet's atmosphere and then, when it is unemcumbered by the light of the coma, follow the light variation of the nucleus itself. The latter will be compared with preperihelion time series and the orientation of the nucleus at the time of Vega and Giotto flybys and an accurate rotational ephemeris constructed. Halley was observed on 38 nights during 1987 and approximately 21 nights in 1988. The comet moved from 5 AU to 8.5 AU during this time. The brightness of the coma was found to rapidly decrease in 1988 as the coma and cometary activity collapses. The magnitude in April 1988 was 19 mag (visual) and it is predicted that the nucleus itself will be the major contributor to the brightness in the 1988 and 1989 season.

Belton, Michael J. S.↗

Machine-Learning of Nonlocal Kernels for Anomalous Subsurface Transport from Breakthrough Curves

Anomalous behavior is ubiquitous in subsurface solute transport due to the presence of high degrees of heterogeneity at different scales in the media. Although fractional models have been extensively used to describe the anomalous transport in various subsurface applications, their application is hindered by computational challenges. Simpler nonlocal models characterized by integrable kernels and finite interaction length represent a computationally feasible alternative to fractional models; yet, the informed choice of their kernel functions still remains an open problem. We propose a general data-driven framework for the discovery of optimal kernels on the basis of very small and sparse data sets in the context of anomalous subsurface transport. Using spatially sparse breakthrough curves recovered from fine-scale particle-density simulations, we learn the best coarse-scale nonlocal model using a nonlocal operator regression technique. Predictions of the breakthrough curves obtained using the optimal nonlocal model show good agreement with fine-scale simulation results even at locations and time intervals different from the ones used to train the kernel, confirming the excellent generalization properties of the proposed algorithm. A comparison with trained classical models and with black-box deep neural networks confirms the superiority of the predictive capability of the proposed model.

97 MATHEMATICS AND COMPUTING↗

NASA Environmental Justice Data Search Interface Overview

NASA’s Earth Science Division (ESD) is committed to empower Environmental Justice (EJ) communities by expanding awareness, accessibility, and use of Earth science data to enable contributions to Earth science research and applications. To that end, the NASA Earth Science Data Systems (ESDS) Program developed an EJ Data Catalog, a simple guide to NASA datasets and socioeconomic datasets that may be useful in EJ research. The EJ Data Catalog is divided by topics—such as disasters, urban flooding, extreme heat, food availability, water availability, climate, and health and air quality—and possible use cases for each dataset. The new version of the EJ Data Catalog is now integrated into NASA’s Science Discovery Engine (SDE), an open-source science infrastructure to enable collaborative and interdisciplinary science. In this workshop you will learn about NASA’s Equity and Environmental Justice (EEJ) activities and opportunities as well as participate on an interactive live demo of the new Science Discovery Engine for Environmental Justice.

environmental justice↗

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

Estimating Species Distributions Across Space Through Time and with Features of the Environment

Complete guidance for mastering the tools and techniques of the digital revolution With the digital revolution opening up tremendous opportunities in many fields, there is a growing need for skilled professionals who can develop data-intensive systems and extract information and knowledge from them. This book frames for the first time a new systematic approach for tackling the challenges of data-intensive computing, providing decision makers and technical experts alike with practical tools for dealing with our exploding data collections. Emphasizing data-intensive thinking and interdisciplinary collaboration, The Data Bonanza: Improving Knowledge Discovery in Science, Engineering, and Business examines the essential components of knowledge discovery, surveys many of the current research efforts worldwide, and points to new areas for innovation. Complete with a wealth of examples and DISPEL-based methods demonstrating how to gain more from data in real-world systems, the book: Outlines the concepts and rationale for implementing data-intensive computing in organizations Covers from the ground up problem-solving strategies for data analysis in a data-rich world Introduces techniques for data-intensive engineering using the Data-Intensive Systems Process Engineering Language DISPEL Features in-depth case studies in customer relations, environmental hazards, seismology, and more Showcases successful applications in areas ranging from astronomy and the humanities to transport engineering Includes sample program snippets throughout the text as well as additional materials on a companion website The Data Bonanza is a must-have guide for information strategists, data analysts, and engineers in business, research, and government, and for anyone wishing to be on the cutting edge of data mining, machine learning, databases, distributed systems, or large-scale computing.

Kelling, S.↗

The green bank northern celestial cap pulsar survey. I. Survey description, data analysis, and initial results

We describe an ongoing search for pulsars and dispersed pulses of radio emission, such as those from rotating radio transients (RRATs) and fast radio bursts, at 350 MHz using the Green Bank Telescope. With the Green Bank Ultimate Pulsar Processing Instrument, we record 100 MHz of bandwidth divided into 4096 channels every 81.92 μs. This survey will cover the entire sky visible to the Green Bank Telescope (δ > –40°, or 82% of the sky) and outside of the Galactic Plane will be sensitive enough to detect slow pulsars and low dispersion measure (<30 pc cm{sup –3}) millisecond pulsars (MSPs) with a 0.08 duty cycle down to 1.1 mJy. For pulsars with a spectral index of –1.6, we will be 2.5 times more sensitive than previous and ongoing surveys over much of our survey region. Here we describe the survey, the data analysis pipeline, initial discovery parameters for 62 pulsars, and timing solutions for 5 new pulsars. PSR J0214+5222 is an MSP in a long-period (512 days) orbit and has an optical counterpart identified in archival data. PSR J0636+5129 is an MSP in a very short-period (96 minutes) orbit with a very low mass companion (8 M{sub J}). PSR J0645+5158 is an isolated MSP with a timing residual RMS of 500 ns and has been added to pulsar timing array experiments. PSR J1434+7257 is an isolated, intermediate-period pulsar that has been partially recycled. PSR J1816+4510 is an eclipsing MSP in a short-period orbit (8.7 hr) and may have recently completed its spin-up phase.

79 ASTRONOMY AND ASTROPHYSICS↗

PESSTO: survey description and products from the first data release by the Public ESO Spectroscopic Survey of Transient Objects

Context. The Public European Southern Observatory Spectroscopic Survey of Transient Objects (PESSTO) began as a public spectroscopic survey in April 2012. PESSTO classifies transients from publicly available sources and wide-field surveys, and selects science targets for detailed spectroscopic and photometric follow-up. PESSTO runs for nine months of the year, January - April and August - December inclusive, and typically has allocations of 10 nights per month. Aims. We describe the data reduction strategy and data products that are publicly available through the ESO archive as the Spectroscopic Survey data release 1 (SSDR1). Methods. PESSTO uses the New Technology Telescope with the instruments EFOSC2 and SOFI to provide optical and NIR spectroscopy and imaging. We target supernovae and optical transients brighter than 20.5m for classification. Science targets are selected for follow-up based on the PESSTO science goal of extending knowledge of the extremes of the supernova population. We use standard EFOSC2 set-ups providing spectra with resolutions of 13-18 Å between 3345-9995 Å. A subset of the brighter science targets are selected for SOFI spectroscopy with the blue and red grisms (0.935-2.53 μm and resolutions 23-33 Å) and imaging with broadband JHKs filters. Results. This first data release (SSDR1) contains flux calibrated spectra from the first year (April 2012-2013). A total of 221 confirmed supernovae were classified, and we released calibrated optical spectra and classifications publicly within 24 h of the data being taken (via WISeREP). The data in SSDR1 replace those released spectra. They have more reliable and quantifiable flux calibrations, correction for telluric absorption, and are made available in standard ESO Phase 3 formats. We estimate the absolute accuracy of the flux calibrations for EFOSC2 across the whole survey in SSDR1 to be typically ~15%, although a number of spectra will have less reliable absolute flux calibration because of weather and slit losses. Acquisition images for each spectrum are available which, in principle, can allow the user to refine the absolute flux calibration. The standard NIR reduction process does not produce high accuracy absolute spectrophotometry but synthetic photometry with accompanying JHKs imaging can improve this. Whenever possible, reduced SOFI images are provided to allow this. Conclusions. Future data releases will focus on improving the automated flux calibration of the data products. The rapid turnaround between discovery and classification and access to reliable pipeline processed data products has allowed early science papers in the first few months of the survey.

79 ASTRONOMY AND ASTROPHYSICS↗