Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Data Quality Challenges for Analysis Ready Data (ARD)

Data quality plays a critical role in research and applications. The Earth Science Information Partners (ESIP) Information Quality Cluster (IQC) defines four aspects of information quality: Science, Product, Stewardship, and Services. The ESIP IQC has become internationally recognized as an authoritative and responsive resource of information and guidance to data producers and distributors on how to implement data quality standards and best practices for their science data systems, datasets, and data/metadata dissemination services. In recent years, cloud computing environments have provided scale-up capabilities such as data archives and services, enabling interdisciplinary science and applications. More value-added products are expected from data service providers, including Analysis Ready Data (ARD). ARD refers to data that has been preprocessed into a form that allows immediate analysis by the end user, processed to a minimum set of requirements and provides interoperability over time and across multiple datasets. Once a dataset has been developed from its original form to produce ARD, what quality characteristics should the derived dataset or ARD possess? Also, is it safe to assume that the quality of the ARD is consistent with the quality of the source data, or are there special attributes to an ARD that would warrant a secondary, independent quality assessment? What provenance (also called “data lineage”) information needs to be included in ARD? It is important to answer these questions, especially given the ease of use of ARD, and the consequent temptation by users to trust ARD without understanding the limitations or possible variations in quality compared to the source data. In this presentation, we will discuss data quality challenges for ARD products and services and introduce IQC for participation.

data quality↗

Lower bound on reliability for Weibull distribution when shape parameter is not estimated accurately

The mathematical relationships between the shape parameter Beta and estimates of reliability and a life limit lower bound for the two parameter Weibull distribution are investigated. It is shown that under rather general conditions, both the reliability lower bound and the allowable life limit lower bound (often called a tolerance limit) have unique global minimums over a range of Beta. Hence lower bound solutions can be obtained without assuming or estimating Beta. The existence and uniqueness of these lower bounds are proven. Some real data examples are given to show how these lower bounds can be easily established and to demonstrate their practicality. The method developed here has proven to be extremely useful when using the Weibull distribution in analysis of no-failure or few-failures data. The results are applicable not only in the aerospace industry but anywhere that system reliabilities are high.

Huang, Zhaofeng↗

Lower bound on reliability for Weibull distribution when shape parameter is not estimated accurately

The mathematical relationships between the shape parameter Beta and estimates of reliability and a life limit lower bound for the two parameter Weibull distribution are investigated. It is shown that under rather general conditions, both the reliability lower bound and the allowable life limit lower bound (often called a tolerance limit) have unique global minimums over a range of Beta. Hence lower bound solutions can be obtained without assuming or estimating Beta. The existence and uniqueness of these lower bounds are proven. Some real data examples are given to show how these lower bounds can be easily established and to demonstrate their practicality. The method developed here has proven to be extremely useful when using the Weibull distribution in analysis of no-failure or few-failures data. The results are applicable not only in the aerospace industry but anywhere that system reliabilities are high.

Huang, Zhaofeng↗

Wave Measurements Using Multi-Frame Processing of Marine Radar Data

Marine radars have proven to be useful for measuring ocean waves, but the accuracy of the measurements is limited by several factors including the look-angle dependence of the radar signals as well as noise in the radar data. The look-angle dependence introduces a systematic error or bias in the measurements, and noise causes a random error. This paper describes a method of combining data from multiple radar frames that is optimal in the sense of minimizing the error for a set of biased measurements with random additive noise. The results are shown experimentally to increase the correlation of the radar estimates with buoy measurements.

47 OTHER INSTRUMENTATION↗

Providing Global Change Information for Decision-Making: Capturing and Presenting Provenance

Global change information demands access to data sources and well-documented provenance to provide evidence needed to build confidence in scientific conclusions and, in specific applications, to ensure the information's suitability for use in decision-making. A new generation of Web technology, the Semantic Web, provides tools for that purpose. The topic of global change covers changes in the global environment (including alterations in climate, land productivity, oceans or other water resources, atmospheric composition and or chemistry, and ecological systems) that may alter the capacity of the Earth to sustain life and support human systems. Data and findings associated with global change research are of great public, government, and academic concern and are used in policy and decision-making, which makes the provenance of global change information especially important. In addition, since different types of decisions benefit from different types of information, understanding how to capture and present the provenance of global change information is becoming more of an imperative in adaptive planning.

provenance tracking↗

The Open Data Repository's Data Publisher

We have quickly gathered a diverse set of databases that have significant activity. Researchers are using them on a daily basis from collection through all phases of research. Focus on: (1) Linked data and semantic web integration are fundamental to our long term plans. (2) Citation system is critical before our first full release. Snapshot-based citation generation for data sets or objects. As we expand our feature set, expect the system will: (1) Provide a very high level of provenance for any data housed in the software. (2) Make data management beneficial to the researcher throughout the research process. (3) Create living archives that allow citable snapshots of data and summaries of the changes since the snapshot was created.

Stone, N.↗

Computer-aided analysis of Skylab scanner data for land use mapping, forestry and water resource applications

Skylab data were obtained over a mountainous test site containing a complex association of cover types and rugged topography. The application of computer-aided analysis techniques to the multispectral scanner data produced a number of significant results. Techniques were developed to digitally overlay topographic data (elevation, slope, and aspect) onto the S-192 MSS data to provide a method for increasing the effectiveness and accuracy of computer-aided analysis techniques for cover type mapping. The S-192 MSS data were analyzed using computer techniques developed at Laboratory for Applications of Remote Sensing (LARS), Purdue University. Land use maps, forest cover type maps, snow cover maps, and area tabulations were obtained and evaluated. These results compared very well with information obtained by conventional techniques. Analysis of the spectral characteristics of Skylab data has conclusively proven the value of the middle infrared portion of the spectrum (about 1.3-3.0 micrometers), a wavelength region not previously available in multispectral satellite data.

Hoffer, R. M.↗

ASTROMETRY.NET: BLIND ASTROMETRIC CALIBRATION OF ARBITRARY ASTRONOMICAL IMAGES

We have built a reliable and robust system that takes as input an astronomical image, and returns as output the pointing, scale, and orientation of that image (the astrometric calibration or World Coordinate System information). The system requires no first guess, and works with the information in the image pixels alone; that is, the problem is a generalization of the 'lost in space' problem in which nothing-not even the image scale-is known. After robust source detection is performed in the input image, asterisms (sets of four or five stars) are geometrically hashed and compared to pre-indexed hashes to generate hypotheses about the astrometric calibration. A hypothesis is only accepted as true if it passes a Bayesian decision theory test against a null hypothesis. With indices built from the USNO-B catalog and designed for uniformity of coverage and redundancy, the success rate is >99.9% for contemporary near-ultraviolet and visual imaging survey data, with no false positives. The failure rate is consistent with the incompleteness of the USNO-B catalog; augmentation with indices built from the Two Micron All Sky Survey catalog brings the completeness to 100% with no false positives. We are using this system to generate consistent and standards-compliant meta-data for digital and digitized imaging from plate repositories, automated observatories, individual scientific investigators, and hobbyists. This is the first step in a program of making it possible to trust calibration meta-data for astronomical data of arbitrary provenance.

07 ISOTOPE AND RADIATION SOURCES↗

The AmeriFlux Data Activity and Data System: An Evolving Collection of Data Management Techniques, Tools, Products and Services

The Carbon Dioxide Information Analysis Center (CDIAC) at Oak Ridge National Laboratory (ORNL), USA has provided scientific data management support for the U.S. Department of Energy and international climate change science since 1982. Over this period, climate change science has expanded from research focusing on basic understanding of geochemical cycles, particularly the carbon cycle, to integrated research addressing climate change impacts, vulnerability, adaptation, and mitigation. Interests in climate change data and information worldwide have grown remarkably and, as a result, so have demands and expectations for CDIAC’s data systems. To meet the growing demands, CDIAC’s strategy has been to design flexible data systems using proven technologies blended with new, evolving technologies and standards. CDIAC development teams are multidisciplinary and include computer science and information technology expertise, but also scientific expertise necessary to address data quality and documentation issues and to identify data products and system capabilities needed by climate change scientists. CDIAC has learned there is rarely a single commercial tool or product readily available to satisfy long-term scientific data system requirements (i.e., one size does not fit all and the breadth and diversity of environmental data are often too complex for easy use with commercial products) and typically deploys a variety of tools and data products in an effort to provide credible data freely to users worldwide. Like many scientific data management applications, CDIAC’s data systems are highly customized to satisfy specific scientific usage requirements (e.g., developing data products specific for model use) but are also designed to be flexible and interoperable to take advantage of new software engineering techniques, standards (e.g., metadata standards) and tools and to support future Earth system data efforts (e.g., ocean acidification). CDIAC has provided data management support for numerous long-term measurement projects crucial to climate change science. One current example is the AmeriFlux measurement network. AmeriFlux provides continuous measurements from forests, grasslands, wetlands, and croplands in North, Central, and South America and offers important insight about carbon cycling in terrestrial ecosystems. We share our approaches in satisfying the challenges of delivering AmeriFlux data worldwide to benefit others with similar challenges handling climate change data, further heighten awareness and use of an outstanding ecological data resource, and highlight expanded software engineering applications being used for climate change measurement data.

Boden, Thomas A.↗

Sun Safe Mode Controller Design for LADEE

This paper presents the development of sun safe controllers which are designed to keep the spacecraft power positive and thermally balanced in the event an anomaly is detected. Employed by NASA's Lunar Atmosphere and Dust Environment Explorer (LADEE), the controllers utilize the measured sun vector and the spacecraft body rates for feedback control. To improve the accuracy of sun vector estimation, the least square minimization approach is applied to process the sensor data, which is proven to be effective and accurate. To validate the controllers, the LADEE spacecraft model engaging the sun safe mode was first simulated and then compared with the actual LADEE orbital fight data. The results demonstrated the applicability of the proposed sun safe controllers.

Sun Safe Control↗

Jet-Surface Interaction Noise from High-Aspect Ratio Nozzles: Test Summary

Noise and flow data have been acquired for a 16:1 aspect ratio rectangular nozzle exhausting near a simple surface at the NASA Glenn Research Center as part of an ongoing effort to understand, model, and predict the noise produced by current and future concept aircraft employing a tightly integrated engine airframe designs. The particular concept under consideration in this experiment is a blended-wing-body airframe powered by a series of electric fans exhausting through slot nozzle over an aft deck. The exhaust Mach number and surface length were parametrically varied during the test. Far-field noise data were acquired for all nozzle surface geometries and exhaust flow conditions. Phased-array noise source localization data and in-flow pressure data were also acquired for a subset of the isolated (no surface) and surface configurations; these measurements provide data that have proven useful for modeling the jet-surface interaction noise source and the surface effect on the jet-mixing noise in round jets. A summary of the nozzle surface geometry, flow conditions tested, and data collected are presented.

Propulsion Noise↗

ERTS-1 applications in hydrology and water resources

After having been in orbit for less than one year, the Earth Resources Technology Satellite (ERTS-1) has shown that it provides very applicable data for more effective monitoring and management of surface water features over the globe. Mapping flooded areas, snowcover, and wetlands and surveying the size, type, and response of glaciers to climate are among the specific areas where ERTS-1 data were applied. In addition the data collection system has proven to be a reliable tool for gathering hydrologic data from remote regions. Turbidity variations in lakes and rivers were also observed and related to shoreline erosion, industrial plant effluent, and overall water quality.

Salomonson, V. V.↗

Application of modern autoradiography to nuclear forensic analysis

Modern autoradiography techniques based on phosphorimaging technology using image plates (IPs) and digital scanning can identify heterogeneities in activity distributions and reveal material properties, serving to inform subsequent analyses. Here, we have adopted these advantages for applications in nuclear forensics, the technical analysis of radioactive or nuclear materials found outside of legal control to provide data related to provenance, production history, and trafficking route for the materials. IP autoradiography is a relatively simple, non-destructive method for sample characterization that records an image reflecting the relative intensity of alpha and beta emissions from a two-dimensional surface. Such data are complementary to information gathered from radiochemical characterization via bulk counting techniques, and can guide the application of other spatially resolved techniques such as scanning electron microscopy (SEM) and secondary ion mass spectrometry (SIMS). IP autoradiography can image large 2-dimenstional areas (up to 20 × 40 cm), with relatively low detection limits for actinides and other radioactive nuclides, and sensitivity to a wide dynamic range (10 5 ) of activity density in a single image. Distributions of radioactivity in nuclear materials can be generated with a spatial resolution of approximately 50 μm using IP autoradiography and digital scanning. While the finest grain silver halide films still provide the best possible resolution (down to ~10 μm), IP autoradiography has distinct practical advantages such as shorter exposure times, no chemical post-processing, reusability, rapid plate scanning, and automated image digitization. Sample preparation requirements are minimal, and the analytical method does not consume or alter the sample. These advantages make IP autoradiography ideal for routine screening of nuclear materials, and for the identification of areas of interest for subsequent micro-characterization methods. Here in this article we present a summary of our setup, as modified for nuclear forensic sample analysis and related research, and provide examples of data from select samples from the nuclear fuel cycle and historical nuclear test debris.

07 ISOTOPE AND RADIATION SOURCES↗

Dynamic Non-Hierarchical File Systems for Exascale Storage

This constitutes the final report for “Dynamic Non-Hierarchical File Systems for Exascale Storage”. The ultimate goal of this project was to improve data management in scientific computing and high-end computing (HEC) applications, and to achieve this goal we proposed: to develop the first, HEC-targeted, file system featuring rich metadata and provenance collection, extreme scalability, and future storage hardware integration as core design goals, and to evaluate and develop a flexible non-hierarchical file system interface suitable for providing more powerful and intuitive data management interfaces to HEC and scientific computing users. Data management is swiftly becoming a serious problem in the scientific community – while copious amounts of data are good for obtaining results, finding the right data is often daunting and sometimes impossible. Scientists participating in a Department of Energy workshop noted that most of their time was spent “...finding, processing, organizing, and moving data and it’s going to get much worse”. Scientists should not be forced to become data mining experts in order to retrieve the data they want, nor should they be expected to remember the naming convention they used several years ago for a set of experiments they now wish to revisit. Ideally, locating the data you need would be as easy as browsing the web. Unfortunately, existing data management approaches are usually based on hierarchical naming, a 40 year-old technology designed to manage thousands of files, not exabytes of data. Today’s systems do not take advantage of the rich array of metadata that current high-end computing (HEC) file systems can gather, including content-based metadata and provenance1 information. As a result, current metadata search approaches are typically ad hoc and often work by providing a parallel management system to the “main” file system, as is done in Linux (the locate utility), personal computers, and enterprise search appliances. These search applications are often optimized for a single file system, making it difficult to move files and their metadata between file systems. Users have tried to solve this problem in several ways, including the use of separate databases to index file properties, the encoding of file properties into file names, and separately gathering and managing provenance data, but none of these approaches has worked well, either due to limited usefulness or scalability, or both. Our research addressed several key issues: High-performance, real-time metadata harvesting: extracting important attributes from files dynamically and immediately updating indexes used to improve search; Transparent, automatic, and secure provenance capture: recording the data inputs and processing steps used in the production of each file in the system; Scalable indexing: indexes that are optimized for integration with the file system; Dynamic file system structure: our approach provides dynamic directories similar to those in semantic file systems, but these are the native organization rather than a feature grafted onto a conventional system. In addition to these goals, our research effort will include evaluating the impact of new storage technologies on the file system design and performance. In particular, the indexing and metadata harvesting functions can potentially benefit from the performance improvements promised by new storage class memories.

42 ENGINEERING↗

SU-E-T-773: Use of a Commercial TPS for Deriving Exit Dose Distribution for Patient Specific QA with An EPID

Purpose: A method to derive exit dose distributions with a commercial treatment planning system for comparison to Epid measured doses. Methods: In contrast to published methods, we present an approach that uses a treatment planning system (Raystation, Raysearch) to calculate exit dose patterns based on a modified patient CT dataset. The EPID is assumed as water equivalent and therefore is represented as water filled ring in the images. This is done by: -Export a renamed copy of the patient CT and plan data set. —Manipulate the CT-data to a field of view of 1000 mm. — Insert the water filled cylinder ring, centered to the target point. — Manipulate plan file with the new target and dose prescription point. — Import and recalculate dose on the manipulated data set. — Export and extract exit field dose matrix from the cylinder (optional entrance dose). — Calibrate matrix to size and rebin data from the cylinder to EPID equivalent data. — Correct Epid measurement for scatter, position and field size. -Evaluate and compare data in Verisoft (PTW). Data manipulation and extraction is done by a simple tool (IDL, Exelis). Results: This method was tested on a Siemens Artiste 6MV for field sizes up to 27×27cm2 limited by used geometry and EPID size. First phantom measurements show good results for fields up to 20×20cm (pass rate > 95% for 3%, 3mm Gamma index) while larger fields have higher discrepancies towards the field edges. This might Result from off axis softening of the beam and the higher sensitivity of the detector to beam scatter. Conclusion: This method might simplify the use of exit dosimetry with the EPID for patient specific QA as it uses the dose calculation of a commercial treatment planning system. The concept was proven by phantom data sets, giving acceptable results.

60 APPLIED LIFE SCIENCES↗

Toward Drilling the Perfect Geothermal Well: An International Research Coordination Network for Geothermal Drilling Optimization Supported by Deep Machine Learning and Cloud Based Data Aggregation

The EDGE project, supported by the U.S. Department of Energy Geothermal Technologies Office under award DE-EE0008793, established a data-driven framework for improving the efficiency, cost-effectiveness, and reliability of geothermal well drilling. The project focused on developing scalable data infrastructure, advanced machine learning and probabilistic models, and integrated analytics tools to support continuous drilling optimization. A central objective was to reduce geothermal drilling costs by up to seventy percent while minimizing the risk of well failure through predictive diagnostics and adaptive planning. Over the project period, a comprehensive data repository was designed and deployed, incorporating records from over one hundred geothermal wells across varied geological settings. This repository supported both structured and unstructured data and adhered to FAIR data principles, enabling provenance tracking, quality control, and standardized metadata. The project introduced automated ingestion pipelines and a cloud-hosted platform that facilitated access to raw, processed, and derived datasets. This infrastructure served as the foundation for model development and analysis. Machine learning workflows were developed to predict key drilling metrics including rate of penetration, non-productive time, and total drilling costs. Self-organizing maps and dimensionality reduction methods were used to uncover operational patterns and outliers, while supervised learning algorithms such as random forests and deep neural networks were applied to forecast performance outcomes. The models were validated on heterogeneous datasets from both U.S. and Icelandic fields, demonstrating variable but significant predictive accuracy. The results indicated that finer temporal resolution, inclusion of lithological data, and consistency in operational annotations could substantially improve model performance. The project also implemented process mining techniques to reconstruct state-transition models from drilling event logs. These models enabled the identification of deviations from optimal workflows and provided insights into recurring failure modes. Analysis of non-productive time highlighted the impact of equipment failures, geological challenges, and human factors, offering opportunities for targeted mitigation strategies. The EDGE Dashboard was developed as a web-based expert system integrating data visualization, model outputs, and user-driven queries. It provided an accessible interface for operators to explore historical data, evaluate predicted outcomes, and compare drilling scenarios. Initial feedback from project partners suggested that the dashboard could serve as a foundation for more advanced advisory and optimization tools. Overall, the EDGE project demonstrated the feasibility and value of applying modern data science techniques to geothermal drilling. It delivered a set of interoperable tools and models that can support more efficient, lower-risk well development. The findings point toward a viable path for transitioning from advisory analytics to semi-autonomous drilling systems, contingent on continued collaboration, expanded datasets, and field validation. The project results have immediate relevance for drilling operations, data management practices, and future geothermal R&D efforts aimed at achieving reliable, cost-competitive geothermal energy at scale.

15 GEOTHERMAL ENERGY↗

Topological Machine Learning Methods for Power System Responses to Contingencies: Preprint

While deep learning tools, coupled with the emerging machinery of topological data analysis, are proven to deliver various performance gains in a broad range of applications, from image classification to biosurveillance to blockchain fraud detection, their utility in areas of high societal importance such as power system modeling and, particularly, resilience quantification in the energy sector yet remain untapped. To provide fast acting synthetic regulation and contingency reserve services to the grid while having minimal disruptions on customer quality of service, we propose a new topologybased system that depends on neural network architecture for impact metrics classification and prediction in power systems. This novel topology-based system allows one to evaluate the impact of three power system contingency types, namely, in conjunction with transmission lines, transformers, and transmission lines combined with transformers. We show that the proposed new neural network architecture equipped with local topological measures facilitates both more accurate classification of unserved load as well as the amount of unserved load. In addition, we are able to learn complex relationships between electrical properties and local topological measurements on the simulated response to contingencies for NREL-SIIP power system.

graph theory↗