Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

OCI Geolocation Evaluation and Refinement Using Landsat Control Points

The Plankton, Aerosol, Cloud, and ocean Ecosystem (PACE) mission is NASA’s next investment in ocean biology, clouds, and aerosol data records. A key feature of PACE is the inclusion of an advanced satellite radiometer, Ocean Color Instrument (OCI), a global mapping radiometer that combines multispectral and hyperspectral remote sensing. The geolocation processing is performed for OCI using spacecraft navigation data and an instrument geometry model. To evaluate the geolocation accuracy for OCI and develop refinements to the processing methods, control point matching using Landsat data has been implemented as a step in the operational processing of OCI data at the Science Data Segment. This processing provides between 200 and 300 high-quality matchups per day with good global and geometric distribution, allowing rapid evaluation of the OCI geolocation accuracy. The results provided an early indication of the overall quality of the geolocation processing and of specific aspects needing improvement. A standard set of granules was identified to support rapid implementation and testing of geolocation refinements, and this approach has been highly successful in improving the geolocation processing accuracy to meet the science requirements. The evaluation will continue throughout the mission to ensure the ongoing accuracy of geolocation. This paper describes the control point matching methodology, the approach to development of the geolocation processing refinements, and the recent results.

PACE↗
DOE OSTI2022

Science of the Van Allen Probes Science Operations Centers

The Van Allen Probes mission operations materialized through a distributed model in which operational responsibility was divided between the Mission Operations Center (MOC) and separate instrument specific SOCs. The sole MOC handled all aspects of telemetering and receiving tasks as well as certain scientifically relevant ancillary tasks. Each instrument science team developed individual instrument specific SOCs proficient in unique capabilities in support of science data acquisition, data processing, instrument performance, and tools for the instrument team scientists. In parallel activities, project scientists took on the task of providing a significant modeling tool base usable by the instrument science teams and the larger scientific community. With a mission as complex as Van Allen Probes, scientific inquiry occurred due to constant and significant collaboration between the SOCs and in concert with the project science team. Planned cross-instrument coordinated observations resulted in critical discoveries during the seven-year mission. Instrument cross-calibration activities elucidated a more seamless set of data products. Specific topics include post-launch changes and enhancements to the SOCs, discussion of coordination activities between the SOCs, SOC specific analysis software, modeling software provided by the Van Allen Probes project, and a section on lessons learned. One of the most significant lessons learned was the importance of the original decision to implement individual team SOCs providing timely and well-documented instrument data for the NASA Van Allen Probes Mission scientists and the larger magnetospheric and radiation belt scientific community.

47 OTHER INSTRUMENTATION↗
NASA NTRS2014

Integrated Science Investigation of the Sun (ISIS): Design of the Energetic Particle Investigation

The Integrated Science Investigation of the Sun (ISIS) is a complete science investigation on the Solar Probe Plus (SPP) mission, which flies to within nine solar radii of the Sun's surface. ISIS comprises a two-instrument suite to measure energetic particles over a very broad energy range, as well as coordinated management, science operations, data processing, and scientific analysis. Together, ISIS observations allow us to explore the mechanisms of energetic particles dynamics, including their: (1) Origins-defining the seed populations and physical conditions necessary for energetic particle acceleration; (2) Acceleration-determining the roles of shocks, reconnection, waves, and turbulence in accelerating energetic particles; and (3) Transport-revealing how energetic particles propagate from the corona out into the heliosphere. The two ISIS Energetic Particle Instruments measure lower (EPI-Lo) and higher (EPI-Hi) energy particles. EPI-Lo measures ions and ion composition from approx. 20 keV/nucleon-15 MeV total energy and electrons from approx.25-1000 keV. EPI-Hi measures ions from approx. 1-200 MeV/nucleon and electrons from approx. 0.5-6 MeV. EPI-Lo comprises 80 tiny apertures with fields-of-view (FOVs) that sample over nearly a complete hemisphere, while EPI-Hi combines three telescopes that together provide five large-FOV apertures. ISIS observes continuously inside of 0.25 AU with a high data collection rate and burst data (EPI-Lo) coordinated with the rest of the SPP payload; outside of 0.25 AU, ISIS runs in low-rate science mode whenever feasible to capture as complete a record as possible of the solar energetic particle environment and provide calibration and continuity for measurements closer in to the Sun. The ISIS Science Operations Center plans and executes commanding, receives and analyzes all ISIS data, and coordinates science observations and analyses with the rest of the SPP science investigations. Together, ISIS' unique observations on SPP will enable the discovery, untangling, and understanding of the important physical processes that govern energetic particles in the innermost regions of our heliosphere, for the first time. This paper summarizes the ISIS investigation at the time of the SPP mission Preliminary Design Review in January 2014.

Coronal Mass Ejections↗
NASA NTRS2000

Integrated tools and techniques applied to the TES ground data system

The author of this paper will dicuss the selection of CASE tools, a decision making process, requirements tracking and a review mechanism that leads to a highly integrated approach to software development that must deal with the constant pressure to change software requirements and design that is associated with research and development.

science data processing TES↗
NASA NTRS1974

Apollo experience report: Apollo lunar surface experiments package data processing system

Apollo Program experience in the processing of scientific data from the Apollo lunar surface experiments package, in which computers and associated hardware and software were used, is summarized. The facility developed for the preprocessing of the lunar science data is described, as are several computer facilities and programs used by the Principal Investigators. The handling, processing, and analyzing of lunar science data and the interface with the Principal Investigators are discussed. Pertinent problems that arose in the development of the data processing schemes are discussed so that future programs may benefit from the solutions to the problems. The evolution of the data processing techniques for lunar science data related to recommendations for future programs of this type.

Eason, R. L.↗
NASA NTRS2008

Aquarius's Instrument Science Data System (ISDS) Automated to Acquire, Process, Trend Data and Produce Radiometric System Assessment Reports

The Aquarius Radiometer, a subsystem of the Aquarius Instrument required a data acquisition ground system to support calibration and radiometer performance assessment. To support calibration and compose performance assessments, we developed an automated system which uploaded raw data to a ftp server and saved raw and processed data to a database. This paper details the overall functionalities of the Aquarius Instrument Science Data System (ISDS) and the individual electrical ground support equipment (EGSE) which produced data files that were infused into the ISDS. Real time EGSEs include an ICDS Simulator, Calibration GSE, Labview controlled power supply, and a chamber data acquisition system. ICDS Simulator serves as a test conductor primary workstation, collecting radiometer housekeeping (HK) and science data and passing commands and HK telemetry collection request to the radiometer. Calibration GSE (Radiometer Active Test Source) provides source choice from multiple targets for the radiometer external calibration. Power Supply GSE, controlled by labview, provides real time voltage and current monitoring of the radiometer. And finally the chamber data acquisition system produces data reflecting chamber vacuum pressure, thermistor temperatures, AVG and watts. Each GSE system produce text based data files every two to six minutes and automatically copies the data files to the Central Archiver PC. The Archiver PC stores the data files, schedules automated uploads of these files to an external FTP server, and accepts request to copy all data files to the ISDS for offline data processing and analysis. Aquarius Radiometer ISDS contains PHP and MATLab programs to parse, process and save all data to a MySQL database. Analysis tools (MATLab programs) in the ISDS system are capable of displaying radiometer science, telemetry and auxiliary data in near real time as well as performing data analysis and producing automated performance assessment reports of the Aquarius Radiometer.

Source record↗
NASA NTRS2016

Agile Multi-Scale Decompositions for Automatic Image Registration

In recent works, the first and third authors developed an automatic image registration algorithm based on a multiscale hybrid image decomposition with anisotropic shearlets and isotropic wavelets. This prototype showed strong performance, improving robustness over registration with wavelets alone. However, this method imposed a strict hierarchy on the order in which shearlet and wavelet features were used in the registration process, and also involved an unintegrated mixture of MATLAB and C code. In this paper, we introduce a more agile model for generating features, in which a flexible and user-guided mix of shearlet and wavelet features are computed. Compared to the previous prototype, this method introduces a flexibility to the order in which shearlet and wavelet features are used in the registration process. Moreover, the present algorithm is now fully coded in C, making it more efficient and portable than the MATLAB and C prototype. We demonstrate the versatility and computational efficiency of this approach by performing registration experiments with the fully-integrated C algorithm. In particular, meaningful timing studies can now be performed, to give a concrete analysis of the computational costs of the flexible feature extraction. Examples of synthetically warped and real multi-modal images are analyzed.

Science Data Processing↗
NASA NTRS2016

Agile Multi-Scale Decompositions for Automatic Image Registration

In recent works, the first and third authors developed an automatic image registration algorithm based on a multiscale hybrid image decomposition with anisotropic shearlets and isotropic wavelets. This prototype showed strong performance, improving robustness over registration with wavelets alone. However, this method imposed a strict hierarchy on the order in which shearlet and wavelet features were used in the registration process, and also involved an unintegrated mixture of MATLAB and C code. In this paper, we introduce a more agile model for generating features, in which a flexible and user-guided mix of shearlet and wavelet features are computed. Compared to the previous prototype, this method introduces a flexibility to the order in which shearlet and wavelet features are used in the registration process. Moreover, the present algorithm is now fully coded in C, making it more efficient and portable than the MATLAB and C prototype. We demonstrate the versatility and computational efficiency of this approach by performing registration experiments with the fully-integrated C algorithm. In particular, meaningful timing studies can now be performed, to give a concrete analysis of the computational costs of the flexible feature extraction. Examples of synthetically warped and real multi-modal images are analyzed.

Science Data Processing↗
DOE OSTI2021

An Extreme-Preserving Long-Term Gridded Daily Precipitation Data Set for the Conterminous United States

Extreme daily precipitation contributes to flooding that can cause significant economic damages, and so is important to properly capture in gridded meteorological data sets. This work examines precipitation extremes, the mean precipitation on wet days, and fraction of wet days in two widely used gridded data sets over the conterminous U.S. (CONUS). When compared to the underlying station observations, the gridded data show a 27% reduction in annual 1-day maximum precipitation, 25% increase in wet day fraction, 1.5 to 2.5 day increase in mean wet spell length, 30% low bias in 20-year return values of daily precipitation, and 25% decrease in mean precipitation on wet days. It is shown these changes arise primarily from the time-adjustment applied to put the precipitation gauge observations into a uniform time frame, with the gridding process playing a lesser role. A new daily precipitation data set is developed that omits the time-adjustment (as well as extending the gridded data by 7 years) and is shown to perform significantly better in reproducing extreme precipitation metrics. When the new data set is used to force a land surface model, annually averaged 1-day maximum runoff increases 38% compared to the original data, annual mean runoff increases 17%, evapotranspiration drops 2.3%, and fewer wet days leads to a 3.3% increase in estimated solar insolation. Consequently, these changes are large enough to affect portrayals of flood risk and water balance components important for ecological and climate-change applications across the CONUS.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

PeakDecoder enables machine learning-based metabolite annotation and accurate profiling in multidimensional mass spectrometry measurements

Multidimensional measurements using state-of-the-art separations and mass spectrometry provide advantages in untargeted metabolomics analyses for studying biological and environmental bio-chemical processes. However, the lack of rapid analytical methods and robust algorithms for these heterogeneous data has limited its application. Here, we develop and evaluate a sensitive and high-throughput analytical and computational workflow to enable accurate metabolite profiling. Our workflow combines liquid chromatography, ion mobility spectrometry and data-independent acquisition mass spectrometry with PeakDecoder, a machine learning-based algorithm that learns to distinguish true co-elution and co-mobility from raw data and calculates metabolite identification error rates. We apply PeakDecoder for metabolite profiling of various engineered strains of Aspergillus pseudoterreus, Aspergillus niger, Pseudomonas putida and Rhodosporidium toruloides. Results, validated manually and against selected reaction monitoring and gas-chromatography platforms, show that 2683 features could be confidently annotated and quantified across 116 microbial sample runs using a library built from 64 standards.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1997

Case Study of an Object-Oriented System: EOSDIS

Mission to Planet Earth (MTPE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of MTPE. The EOS Data and Information System (EOSDIS) is another component of MTPE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at six Distributed Active Archive Centers (DAACS) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. While there are many segments in EOSDIS (e.g., flight operations, network) this case study discusses the development of the science data processing segment (SDPS). We briefly review the architecture of the system, the goals of the SDPS, and the development progress to date. This study highlights key software development challenges, experiences integrating COTS, and the difficulties of managing a complex system development effort.

Behnke, Jeanne↗
NASA NTRS2016

A Quantum Annealing Computer Team Addresses Climate Change Predictability

The near confluence of the successful launch of the Orbiting Carbon Observatory2 on July 2, 2014 and the acceptance on August 20, 2015 by Google, NASA Ames Research Center and USRA of a 1152 qubit D-Wave 2X Quantum Annealing Computer (QAC), offered an exceptional opportunity to explore the potential of this technology to address the scientific prediction of global annual carbon uptake by land surface processes. At UMBC,we have collected and processed 20 months of global Level 2 light CO2 data as well as fluorescence data. In addition we have collected ARM data at 2sites in the US and Ameriflux data at more than 20 stations. J. Dorband has developed and implemented a multi-hidden layer Boltzmann Machine (BM) algorithm on the QAC. Employing the BM, we are calculating CO2 fluxes by training collocated OCO-2 level 2 CO2 data with ARM ground station tower data to infer to infer measured CO2 flux data. We generate CO2 fluxes with a regression analysis using these BM derived weights on the level 2 CO2 data for three Ameriflux sites distinct from the ARM stations. P. Gentine has negotiated for the access of K34 Ameriflux data in the Amazon and is applying a neural net to infer the CO2 fluxes. N. Talik validated the accuracy of the BM performance on the QAC against a restricted BM implementation on the IBM Softlayer Cloud with the Nvidia co-processors utilizing the same data sets. G. Nearing and K. Harrison have extended the GSFC LIS model with the NCAR Noah photosynthetic parameterization and have run a 10 year global prediction of the net ecosystem exchange. C. Pellisier is preparing a BM implementation of the Kalman filter data assimilation of CO2 fluxes. At UMBC, R. Prouty is conducting OSSE experiments with the LISNoah model on the IBM iDataPlex to simulate the impact of CO2 fluxes to improve the prediction of global annual carbon uptake. J. LeMoigne and D. Simpson have developed a neural net image registration system that will be used for MODIS ENVI and will be converted to a BM algorithm implementation on the QAC. The first integer adder has been implemented on the D-Wave 2X by A. Shehab that will perform HAAR wavelets for image compression of MODIS scenes. Finally, based on the next generations of QACs, we are preparing a 5-year performance road map on the scalability of the current QAC algorithms.

Science Data Processing↗
DOE OSTI2023

Molecular Simulations of the Chain Length Dependent Adsorption of C7‐C14 n ‐Alkanes in ZIF‐8

Recent experiments show that the diffusivities of C8 and C10 n-alkanes in ZIF-8 are higher than those of C7 and C9, respectively. We investigated this unusual ‘odd-even' effect by simulating the adsorption of C7-C14 n-alkanes in ZIF-8 using hybrid Monte Carlo molecular simulations. The resultant adsorption isotherms, guest-host energies, isosteric heats, and chain length distributions are analyzed for trends among the n-alkanes. ZIF-8 cages filled with n-alkanes are characterized using a combination of image processing and data science to quantify the differences in packing that occur with chain length. In conclusion, results indicate that packing changes drastically from C7 to C8 and from C9 to C10, which is consistent with the diffusion trends previously reported in experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

ContScout: sensitive detection and removal of contamination from annotated genomes

Contamination of genomes is an increasingly recognized problem affecting several downstream applications, from comparative evolutionary genomics to metagenomics. Here we introduce ContScout, a precise tool for eliminating foreign sequences from annotated genomes. It achieves high specificity and sensitivity on synthetic benchmark data even when the contaminant is a closely related species, outperforms competing tools, and can distinguish horizontal gene transfer from contamination. A screen of 844 eukaryotic genomes for contamination identified bacteria as the most common source, followed by fungi and plants. Furthermore, we show that contaminants in ancestral genome reconstructions lead to erroneous early origins of genes and inflate gene loss rates, leading to a false notion of complex ancestral genomes. Taken together, we offer here a tool for sensitive removal of foreign proteins, identify and remove contaminants from diverse eukaryotic genomes and evaluate their impact on phylogenomic analyses.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Improvement of eukaryotic protein predictions from soil metagenomes

During the last decades, metagenomics has highlighted the diversity of microorganisms from environmental or host-associated samples. Most metagenomics public repositories use annotation pipelines tailored for prokaryotes regardless of the taxonomic origin of contigs. Consequently, eukaryotic contigs with intrinsically different gene features, are not optimally annotated. Using a bioinformatics pipeline, we have filtered 7.9 billion contigs from 6,872 soil metagenomes in the JGI’s IMG/M database to identify eukaryotic contigs. We have re-annotated genes using eukaryote-tailored methods, yielding 8 million eukaryotic proteins and over 300,000 orphan proteins lacking homology in public databases. Comparing the gene predictions we made with initial JGI ones on the same contigs, we confirmed our pipeline improves eukaryotic proteins completeness and contiguity in soil metagenomes. The improved quality of eukaryotic proteins combined with a more comprehensive assignment method yielded more reliable taxonomic annotation. This dataset of eukaryotic soil proteins with improved completeness, quality and taxonomic annotation reliability is of interest for any scientist aiming at studying the composition, biological functions and gene flux in soil communities involving eukaryotes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Coassembly and binning of a twenty-year metagenomic time-series from Lake Mendota

Abstract The North Temperate Lakes Long-Term Ecological Research (NTL-LTER) program has been extensively used to improve understanding of how aquatic ecosystems respond to environmental stressors, climate fluctuations, and human activities. Here, we report on the metagenomes of samples collected between 2000 and 2019 from Lake Mendota, a freshwater eutrophic lake within the NTL-LTER site. We utilized the distributed metagenome assembler MetaHipMer to coassemble over 10 terabases (Tbp) of data from 471 individual Illumina-sequenced metagenomes. A total of 95,523,664 contigs were assembled and binned to generate 1,894 non-redundant metagenome-assembled genomes (MAGs) with ≥50% completeness and ≤10% contamination. Phylogenomic analysis revealed that the MAGs were nearly exclusively bacterial, dominated by Pseudomonadota (Proteobacteria, N = 623) and Bacteroidota (N = 321). Nine eukaryotic MAGs were identified by eukCC with six assigned to the phylum Chlorophyta. Additionally, 6,350 high-quality viral sequences were identified by geNomad with the majority classified in the phylum Uroviricota. This expansive coassembled metagenomic dataset provides an unprecedented foundation to advance understanding of microbial communities in freshwater ecosystems and explore temporal ecosystem dynamics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Standardized phylogenetic and molecular evolutionary analysis applied to species across the microbial tree of life

There is growing interest in reconstructing phylogenies from the copious amounts of genome sequencing projects that target related viral, bacterial or eukaryotic organisms. To facilitate the construction of standardized and robust phylogenies for disparate types of projects, we have developed a complete bioinformatic workflow, with a web-based component to perform phylogenetic and molecular evolutionary (PhaME) analysis from sequencing reads, draft assemblies or completed genomes of closely related organisms. Furthermore, the ability to incorporate raw data, including some metagenomic samples containing a target organism (e.g. from clinical samples with suspected infectious agents), shows promise for the rapid phylogenetic characterization of organisms within complex samples without the need for prior assembly.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Multimodal X-ray nano-spectromicroscopy analysis of chemically heterogeneous systems

Abstract Understanding the nanoscale chemical speciation of heterogeneous systems in their native environment is critical for several disciplines such as life and environmental sciences, biogeochemistry, and materials science. Synchrotron-based X-ray spectromicroscopy tools are widely used to understand the chemistry and morphology of complex material systems owing to their high penetration depth and sensitivity. The multidimensional (4D+) structure of spectromicroscopy data poses visualization and data-reduction challenges. This paper reports the strategies for the visualization and analysis of spectromicroscopy data. We created a new graphical user interface and data analysis platform named XMIDAS (X-ray multimodal image data analysis software) to visualize spectromicroscopy data from both image and spectrum representations. The interactive data analysis toolkit combined conventional analysis methods with well-established machine learning classification algorithms (e.g. nonnegative matrix factorization) for data reduction. The data visualization and analysis methodologies were then defined and optimized using a model particle aggregate with known chemical composition. Nanoprobe-based X-ray fluorescence (nano-XRF) and X-ray absorption near edge structure (nano-XANES) spectromicroscopy techniques were used to probe elemental and chemical state information of the aggregate sample. We illustrated the complete chemical speciation methodology of the model particle by using XMIDAS. Next, we demonstrated the application of this approach in detecting and characterizing nanoparticles associated with alveolar macrophages. Our multimodal approach combining nano-XRF, nano-XANES, and differential phase-contrast imaging efficiently visualizes the chemistry of localized nanostructure with the morphology. We believe that the optimized data-reduction strategies and tool development will facilitate the analysis of complex biological and environmental samples using X-ray spectromicroscopy techniques.

36 MATERIALS SCIENCE↗