Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Fractals and Spatial Methods for Mining Remote Sensing Imagery

The rapid increase in digital remote sensing and GIS data raises a critical problem -- how can such an enormous amount of data be handled and analyzed so that useful information can be derived quickly? Efficient handling and analysis of large spatial data sets is central to environmental research, particularly in global change studies that employ time series. Advances in large-scale environmental monitoring and modeling require not only high-quality data, but also reliable tools to analyze the various types of data. A major difficulty facing geographers and environmental scientists in environmental assessment and monitoring is that spatial analytical tools are not easily accessible. Although many spatial techniques have been described recently in the literature, they are typically presented in an analytical form and are difficult to transform to a numerical algorithm. Moreover, these spatial techniques are not necessarily designed for remote sensing and GIS applications, and research must be conducted to examine their applicability and effectiveness in different types of environmental applications. This poses a chicken-and-egg problem: on one hand we need more research to examine the usability of the newer techniques and tools, yet on the other hand, this type of research is difficult to conduct if the tools to be explored are not accessible. Another problem that is fundamental to environmental research are issues related to spatial scale. The scale issue is especially acute in the context of global change studies because of the need to integrate remote-sensing and other spatial data that are collected at different scales and resolutions. Extrapolation of results across broad spatial scales remains the most difficult problem in global environmental research. There is a need for basic characterization of the effects of scale on image data, and the techniques used to measure these effects must be developed and implemented to allow for a multiple scale assessment of the data before any useful process-oriented modeling involving scale-dependent data can be conducted. Through the support of research grants from NASA, we have developed a software module called ICAMS (Image Characterization And Modeling System) to address the need to develop innovative spatial techniques and make them available to the broader scientific communities. ICAMS provides new spatial techniques, such as fractal analysis, geostatistical functions, and multiscale analysis that are not easily available in commercial GIS/image processing software. By bundling newer spatial methods in a user-friendly software module, researchers can begin to test and experiment with the new spatial analysis methods and they can gauge scale effects using a variety of remote sensing imagery. In the following, we describe briefly the development of ICAMS and present application examples.

Lam, Nina↗
NASA NTRS2023

Addition of First-Person Visualization Capabilities to the Glenn Research Center Communication Analysis Suite (GCAS)

The ability to visualize large sets of data is important to giving an understanding of abstract ideas to large groups. With the ability to visualize the terrain of lunar surfaces in three dimensions from both an orbital and first-person view, the capabilities of the Glenn Research Center Communication Analysis Suite (GCAS) will have expanded to allow specific landing sites to be seen more realistically. GCAS will utilize GeoTIFF-based data to visualize the lunar terrain.

ECMAScript↗
DOE OSTI2023

Transferring a Molecular Foundation Model for Polymer Property Predictions

Transformer-based large language models have remarkable potential to accelerate design optimization for applications such as drug development and material discovery. Self-supervised pretraining of transformer models requires large-scale data sets, which are often sparsely populated in topical areas such as polymer science. Further, state-of-the-art approaches for polymers conduct data augmentation to generate additional samples but unavoidably incur extra computational costs. In contrast, large-scale open-source data sets are available for small molecules and provide a potential solution to data scarcity through transfer learning. In this work, we show that using transformers pretrained on small molecules and fine-tuned on polymer properties achieves comparable accuracy to those trained on augmented polymer data sets for a series of benchmark prediction tasks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1992

Data indexing techniques for the EUVE all-sky survey

This poster describes techniques developed for manipulating large full-sky data sets for the Extreme Ultraviolet Explorer project. The authors have adapted the quatrilateralized cubic sphere indexing algorithm to allow us to efficiently store and process several types of large data sets, such as full-sky maps of photon counts, exposure time, and count rates. A variation of this scheme is used to index sparser data such as individual photon events and viewing times for selected areas of the sky, which are eventually used to create EUVE source catalogs.

Lewis, J.↗
NASA NTRS2005

Comparison of Monthly Mean Cloud Fraction and Cloud Optical depth Determined from Surface Cloud Radar, TOVS, AVHRR, and MODIS over Barrow, Alaska

A one year comparison is made of mean monthly values of cloud fraction and cloud optical depth over Barrow, Alaska (71 deg 19.378 min North, 156 deg 36.934 min West) between 35 GHz radar-based retrievals, the TOVS Pathfinder Path-P product, the AVHRR APP-X product, and a MODIS based cloud retrieval product from the CERES-Team. The data sets represent largely disparate spatial and temporal scales, however, in this paper, the focus is to provide a preliminary analysis of how the mean monthly values derived from these different data sets compare, and determine how they can best be used separately, and in combination to provide reliable estimates of long-term trends of changing cloud properties. The radar and satellite data sets described here incorporate Arctic specific modifications that account for cloud detection challenges specific to the Arctic environment. The year 2000 was chosen for this initial comparison because the cloud radar data was particularly continuous and reliable that year, and all of the satellite retrievals of interest were also available for the year 2000. Cloud fraction was chosen as a comparison variable as accurate detection of cloud is the primary product that is necessary for any other cloud property retrievals. Cloud optical depth was additionally selected as it is likely the single cloud property that is most closely correlated to cloud influences on surface radiation budgets.

Uttal, Taneil↗
NASA NTRS1987

Problems in merging Earth sensing satellite data sets

Satellite remote sensing systems provide a tremendous source of data flow to the Earth science community. These systems provide scientists with data of types and on a scale previously unattainable. Looking forward to the capabilities of Space Station and the Earth Observing System (EOS), the full realization of the potential of satellite remote sensing will be handicapped by inadequate information systems. There is a growing emphasis in Earth science research to ask questions which are multidisciplinary in nature and global in scale. Many of these research projects emphasize the interactions of the land surface, the atmosphere, and the oceans through various physical mechanisms. Conducting this research requires large and complex data sets and teams of multidisciplinary scientists, often working at remote locations. A review of the problems of merging these large volumes of data into spatially referenced and manageable data sets is presented.

Smith, Paul H.↗
DOE OSTI2025

Extracting Material Property Measurements from Scientific Literature with Limited Annotations

Extracting material property data from scientific text is pivotal for advancing data-driven research in chemistry and materials science; however, the extensive annotation effort required to produce training data for named entity recognition (NER) models for this task often makes it a barrier to extracting specialized data sets. Here, in this work, we present a comparative study of the conventional, supervised NER methodology to alternative few-shot learning architectures and large language model (LLM)-based approaches that mitigate the need to label large training data sets. We find that the best-performing LLM (GPT-4o) not only excels in directly extracting relevant material properties based on limited examples but also enhances supervised learning through data augmentation. We supplement our findings with error and data quality assessments to provide a nuanced understanding of factors that impact property measurement extraction.

36 MATERIALS SCIENCE↗
NASA NTRS1985

Data-link alternatives for the NASA pilot data systems

There are several preliminary, or pilot, efforts for developing data systems for supporting NASA science research. The Pilot Data Systems consist of the Pilot Land Data System (PLDS), the Pilot Ocean Data System (PODS), the Pilot Climate Data System (PCDS), and the Pilot Planetary Data System (PPDS). The PLDS is likely to have the greatest communication needs of the various pilots. Communications issues and requirements are examined in the context of the mature PLDS as it may exist by 1990. The PLDS is seen as a distributed processing system linking resources at a number of NASA research centers and outside universities. Large image data sets, including LANDSAT scenes, are a major data type to be moved along the PLDS communications network. The unusually large size of these data files requires examining new technologies that may allow efficient and affordable communication at rates of up to 60 megabits per second. Potentially useful developments include portable satellite ground stations, and time division multiple access for sharing high-speed satellite links. Further pooling of communications for the various data systems into a single network would reduce costs and may make economic justification of access to the required data rates possible.

Likens, W. C.↗
NASA NTRS2010

Visual Data Comm: A Tool for Visualizing Data Communication in the Multi Sector Planner Study

Data comm is a new technology proposed in future air transport system as a potential tool to provide comprehensive data connectivity. It is a key enabler to manage 4D trajectory digitally, potentially resulting in improved flight times and increased throughput. Future concepts with data comm integration have been tested in a number of human-in-the-loop studies but analyzing the results has proven to be particularly challenging because future traffic environment in which data comm is fully enabled has assumed high traffic density, resulting in data set with large amount of information. This paper describes the motivation, design, current and potential future application of Visual Data Comm (VDC), a tool for visualizing data developed in Java using Processing library which is a tool package designed for interactive visualization programming. This paper includes an example of an application of VDC on data pertaining to the most recent Multi Sector Planner study, conducted at NASA s Airspace Operations Laboratory in 2009, in which VDC was used to visualize and interpret data comm activities

Lee, Hwasoo Eric↗
DOE OSTI2022

The Data Synergy Effects of Time-Series Deep Learning Models in Hydrology

When fitting statistical models to variables in geoscientific disciplines such as hydrology, it is a customary practice to stratify a large domain into multiple regions (or regimes) and study each region separately. Traditional wisdom suggests that models built for each region separately will have higher performance because of homogeneity within each region. However, each stratified model has access to fewer and less diverse data points. Here, through two hydrologic examples (soil moisture and streamflow), we show that conventional wisdom may no longer hold in the era of big data and deep learning (DL). We systematically examined an effect we call data synergy, where the results of the DL models improved when data were pooled together from characteristically different regions. The performance of the DL models benefited from modest diversity in the training data compared to a homogeneous training set, even with similar data quantity. Moreover, allowing heterogeneous training data makes eligible much larger training datasets, which is an inherent advantage of DL. A large, diverse data set is advantageous in terms of representing extreme events and future scenarios, which has strong implications for climate change impact assessment. The results here suggest the research community should place greater emphasis on data sharing.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2001

Simultaneous Retrieval of Temperature, Water Vapor and Ozone Atmospheric Profiles from IASI: Compression, De-noising, First Guess Retrieval and Inversion Algorithms

A fast temperature water vapor and ozone atmospheric profile retrieval algorithm is developed for the high spectral resolution Infrared Atmospheric Sounding Interferometer (IASI) space-borne instrument. Compression and de-noising of IASI observations are performed using Principal Component Analysis. This preprocessing methodology also allows, for a fast pattern recognition in a climatological data set to obtain a first guess. Then, a neural network using first guess information is developed to retrieve simultaneously temperature, water vapor and ozone atmospheric profiles. The performance of the resulting fast and accurate inverse model is evaluated with a large diversified data set of radiosondes atmospheres including rare events.

Aires, F.↗
DOE OSTI2020

PPINN: Parareal physics-informed neural network for time-dependent PDEs

Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time–space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speed-up for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion–reaction equation. Furthermore, our results demonstrate that PPINNs converge in a few iterations with significant speed-ups proportional to the number of time-subdomains employed.

42 ENGINEERING↗
DOE OSTI2020

Building Large-Scale U.S. Synthetic Electric Distribution System Models

Rapid increases in distributed energy resources on distribution systems are prompting research efforts to improve and evaluate electric power distribution algorithms; however, there is a shortage of realistic, large-scale, U.S.-style test systems for the evaluation of such advanced algorithms. Some available tools to build large-scale test systems are of European style, and their application to studies focused on the United States might not be desirable given very different characteristics between the European and U.S. distribution designs. Motivated by this need, this paper develops detailed algorithms to build large-scale U.S. distribution systems and incorporates them in a new Reference Network Model, RNM-US. The approach starts with information from street maps and a catalog with electric equipment that includes power lines, transformers, voltage regulators, capacitors, and switching devices. The paper presents the algorithms through an illustrative case study of the different steps that comprise the process of building a synthetic distribution grid. Finally, the paper presents a medium- and a large-scale data set covering 10 million electrical nodes and 120,000 km of power lines, demonstrating the applicability of the proposed method to build very large-scale synthetic distribution systems.

27 ARPA - Advanced Research Projects Agency-Energy↗
NASA NTRS1991

Autoclass: An automatic classification system

The task of inferring a set of classes and class descriptions most likely to explain a given data set can be placed on a firm theoretical foundation using Bayesian statistics. Within this framework, and using various mathematical and algorithmic approximations, the AutoClass System searches for the most probable classifications, automatically choosing the number of classes and complexity of class descriptions. A simpler version of AutoClass has been applied to many large real data sets, has discovered new independently-verified phenomena, and has been released as a robust software package. Recent extensions allow attributes to be selectively correlated within particular classes, and allow classes to inherit, or share, model parameters through a class hierarchy. The mathematical foundations of AutoClass are summarized.

Stutz, John↗
NASA NTRS1991

Bayesian classification theory

The task of inferring a set of classes and class descriptions most likely to explain a given data set can be placed on a firm theoretical foundation using Bayesian statistics. Within this framework and using various mathematical and algorithmic approximations, the AutoClass system searches for the most probable classifications, automatically choosing the number of classes and complexity of class descriptions. A simpler version of AutoClass has been applied to many large real data sets, has discovered new independently-verified phenomena, and has been released as a robust software package. Recent extensions allow attributes to be selectively correlated within particular classes, and allow classes to inherit or share model parameters though a class hierarchy. We summarize the mathematical foundations of AutoClass.

Hanson, Robin↗
NASA NTRS2003

Software for Managing Parametric Studies

The Information Power Grid Virtual Laboratory (ILab) is a Practical Extraction and Reporting Language (PERL) graphical-user-interface computer program that generates shell scripts to facilitate parametric studies performed on the Grid. (The Grid denotes a worldwide network of supercomputers used for scientific and engineering computations involving data sets too large to fit on desktop computers.) Heretofore, parametric studies on the Grid have been impeded by the need to create control language scripts and edit input data files painstaking tasks that are necessary for managing multiple jobs on multiple computers. ILab reflects an object-oriented approach to automation of these tasks: All data and operations are organized into packages in order to accelerate development and debugging. A container or document object in ILab, called an experiment, contains all the information (data and file paths) necessary to define a complex series of repeated, sequenced, and/or branching processes. For convenience and to enable reuse, this object is serialized to and from disk storage. At run time, the current ILab experiment is used to generate required input files and shell scripts, create directories, copy data files, and then both initiate and monitor the execution of all computational processes.

Yarrow, Maurice↗
NASA NTRSDate not supplied

A Gaussian Process Enhancement to Linear Parameter Varying Models

Simulation and analysis for modern engineering systems now routinely requires the merging of multiple disciplines, physical-domains, time-scales, and data sets — all at ever increasing levels. These capabilities are especially needed in the domain of Advanced Air Mobility, where rapidly emerging vehicle designs are significantly more complex, while having to be both cost-effective and safe. To meet these engineering challenges, machine learning methods are an attractive option for merging models and data across multiple areas while providing uncertainty quantification and maintaining computational efficiency. This paper examines the use of Gaussian process machine learning to generalize and enhance the commonly used class of quasi-Linear Parameter Varying models for fast full-envelope simulation while also supporting control system design and analysis with model uncertainty. Gaussian process machine learning is selected because it: can fuse multiple data sets, enables an easy trade-off between data fitting and smoothing, provides model uncertainty quantification, scales well with increasing complexity, and does not generally require starting from a large training data set. To demonstrate the benefits of the approach, a robust stability analysis with Gaussian process uncertainty is shown for a NASA reference design of an electric quad-rotor air-taxi concept vehicle with motor parameter uncertainty.

Gaussian Process↗
NASA NTRS1992

SAVS: A Space Analysis and Visualization System

We propose to develop, test, demonstrate, and deliver to NASA a powerful and versatile data acquisition, manipulation, analysis and visualization system which will enhance scientific capabilities in the display and interpretation of diverse and distributed data within an integrated user-friendly environment. Our approach exploits existing technologies and combines three major elements into an easy-to-use interactive package: (1) innovative visualization software; (2) advanced database techniques; and (3) a rich set of mathematical and image processing tools. Visualization capabilities will include one-, two-, and three-dimensional displays, along with animation, compression, warping and slicing functions. Analysis tools will include generic mathematical and statistical techniques along with the ability to use large scale models for interactive interpretation of large volume data sets. Our system will be implemented on Sun and DEC UNIX workstations and on the Stardent Graphics Super computer. Our final deliverable will include complete documentation and a NASA/NSF-CDAW/SUNDIAL campaign demonstration.

Szuszczewicz, Edward P.↗