SEARCH · Engineering Papers

Results for “Automatic Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36

All sources NASA NTRS DOE OSTI

DOE OSTI2019

Text-mined dataset of inorganic materials synthesis recipes

Abstract Materials discovery has become significantly facilitated and accelerated by high-throughput ab-initio computations. This ability to rapidly design interesting novel compounds has displaced the materials innovation bottleneck to the development of synthesis routes for the desired material. As there is no a fundamental theory for materials synthesis, one might attempt a data-driven approach for predicting inorganic materials synthesis, but this is impeded by the lack of a comprehensive database containing synthesis processes. To overcome this limitation, we have generated a dataset of “codified recipes” for solid-state synthesis automatically extracted from scientific publications. The dataset consists of 19,488 synthesis entries retrieved from 53,538 solid-state synthesis paragraphs by using text mining and natural language processing approaches. Every entry contains information about target material, starting compounds, operations used and their conditions, as well as the balanced chemical equation of the synthesis reaction. The dataset is publicly available and can be used for data mining of various aspects of inorganic materials synthesis.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2016

The Materials Data Facility: Data Services to Advance Materials Science Research

With increasingly strict data management requirements from funding agencies and institutions, expanding focus on the challenges of research replicability, and growing data sizes and heterogeneity, new data needs are emerging in the materials community. The materials data facility (MDF) operates two cloudhosted services, data publication and data discovery, with features to promote open data sharing, self-service data publication and curation, and encourage data reuse, layered with powerful data discovery tools. The data publication service simplifies the process of copying data to a secure storage location, assigning data a citable persistent identifier, and recording custom (e.g., material, technique, or instrument specific)and automatically-extractedmetadata in a registrywhile the data discovery service will provide advanced search capabilities (e.g., faceting, free text range querying, and full text search) against the registered data and metadata. TheMDF services empower individual researchers, research projects, and institutions to (I) publish research datasets, regardless of size, from local storage, institutional data stores, or cloud storage, without involvement of thirdparty publishers; (II) build, share, and enforce extensible domain-specific custom metadata schemas; (III) interact with published data and metadata via representational state transfer (REST) application program interfaces (APIs) to facilitate automation, analysis, and feedback; and (IV) access a data discovery model that allows researchers to search, interrogate, and eventually build on existing published data. We describe MDF’s design, current status, and future plans.

data management↗

DOE OSTI2024

Targeted materials discovery using Bayesian algorithm execution

Rapid discovery and synthesis of future materials requires intelligent data acquisition strategies to navigate large design spaces. A popular strategy is Bayesian optimization, which aims to find candidates that maximize material properties; however, materials design often requires finding specific subsets of the design space which meet more complex or specialized goals. We present a framework that captures experimental goals through straightforward user-defined filtering algorithms. These algorithms are automatically translated into one of three intelligent, parameter-free, sequential data collection strategies (SwitchBAX, InfoBAX, and MeanBAX), bypassing the time-consuming and difficult process of task-specific acquisition function design. Our framework is tailored for typical discrete search spaces involving multiple measured physical properties and short time-horizon decision making. We demonstrate this approach on datasets for TiO 2 nanoparticle synthesis and magnetic materials characterization, and show that our methods are significantly more efficient than state-of-the-art approaches. Overall, our framework provides a practical solution for navigating the complexities of materials design, and helps lay groundwork for the accelerated development of advanced materials.

42 ENGINEERING↗

DOE OSTI2017

Domain Specific Language Support for Exascale. Final Project Report

The project developed a domain specific translator enable legacy MPI source code to tolerate communication delays, which are increasing over time due to technological factors. The translator performs source-to-source translation that incorporates semantic information into the translation process. The output of the translator is a C program runs as a data driven program, and uses an existing run time to overlap communication automatically

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2017

Very High Resolution Tree Cover Mapping for Continental United States using Deep Convolutional Neural Networks

Uncertainties in input land cover estimates contribute to a significant bias in modeled above ground biomass (AGB) and carbon estimates from satellite-derived data. The resolution of most currently used passive remote sensing products is not sufficient to capture tree canopy cover of less than ca. 10-20 percent, limiting their utility to estimate canopy cover and AGB for trees outside of forest land. In our study, we created a first of its kind Continental United States (CONUS) tree cover map at a spatial resolution of 1-m for the 2010-2012 epoch using the USDA NAIP imagery to address the present uncertainties in AGB estimates. The process involves different tasks including data acquisition ingestion to pre-processing and running a state-of-art encoder-decoder based deep convolutional neural network (CNN) algorithm for automatically generating a tree non-tree map for almost a quarter million scenes. The entire processing chain including generation of the largest open source existing aerial satellite image training database was performed at the NEX supercomputing and storage facility. We believe the resulting forest cover product will substantially contribute to filling the gaps in ongoing carbon and ecological monitoring research and help quantifying the errors and uncertainties in derived products.

High Resolution↗

DOE OSTI2015

Supervised Gamma Process Poisson Factorization

This thesis develops the supervised gamma process Poisson factorization (S- GPPF) framework, a novel supervised topic model for joint modeling of count matrices and document labels. S-GPPF is fully generative and nonparametric: document labels and count matrices are modeled under a uni ed probabilistic framework and the number of latent topics is controlled automatically via a gamma process prior. The framework provides for multi-class classification of documents using a generative max-margin classifier. Several recent data augmentation techniques are leveraged to provide for exact inference using a Gibbs sampling scheme. The first portion of this thesis reviews supervised topic modeling and several key mathematical devices used in the formulation of S-GPPF. The thesis then introduces the S-GPPF generative model and derives the conditional posterior distributions of the latent variables for posterior inference via Gibbs sampling. The S-GPPF is shown to exhibit state-of-the-art performance for joint topic modeling and document classification on a dataset of conference abstracts, beating out competing supervised topic models. The unique properties of S-GPPF along with its competitive performance make it a novel contribution to supervised topic modeling.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

End-System Aware Peak Link Utilization Transport for Big Data Transfer over Dedicated Connections

High-performance networks provisioning 10/40-Gbps dedicated channels have been rapidly developed andincreasingly deployed to support big data transfers on the order of terabytes or petabytes in large-scale network-intensiveapplications. However, end users have not seen corresponding increase in application throughput mainly because traditional end-to-end transport methods are not optimized for such connections. New congestion or flow control mechanisms are desirable to meet the challenges brought to transport protocol design by dedicated connections. The advent and proliferation of multi-core processors make it now possible to improve application throughput by providing multiple processing and network resources to a single data transfer. Based on the existing PLUT method, we proposed a new dynamic buffer management scheme and an automatic parallelism tuning mechanism to achieve the maximal bottleneck rate. The performance superiority of PLUT is justified by the experimental results collected over a local 10Gbps dedicated connection.

Rao, Nageswara↗

NASA NTRS2019

Advancements in the Aerosol Robotic Network (AERONET) Version 3 database – automated near-real-time quality control algorithm with improved cloud screening for Sun photometer aerosol optical depth (AOD) measurements

The Aerosol Robotic Network (AERONET) has provided highly accurate, ground-truth measurements of the aerosol optical depth (AOD) using Cimel Electronique Sun–sky radiometers for more than 25 years. In Version 2 (V2) of the AERONET database, the near-real-time AOD was semiautomatically quality controlled utilizing mainly cloud-screening methodology, while additional AOD data contaminated by clouds or affected by instrument anomalies were removed manually before attaining quality-assured status (Level 2.0). The large growth in the number of AERONET sites over the past 25 years resulted in significant burden to the manual quality control of millions of measurements in a consistent manner. The AERONET Version 3 (V3) algorithm provides fully automatic cloud screening and instrument anomaly quality controls. All of these new algorithm updates apply to near-real-time data as well as post-field-deployment processed data, and AERONET reprocessed the database in 2018. A full algorithm redevelopment provided the opportunity to improve data inputs and corrections such as unique filter-specific temperature characterizations for all visible and near-infrared wavelengths, updated gaseous and water vapor absorption coefficients, and ancillary data sets. The Level 2.0 AOD quality-assured data set is now available within a month after post-field calibration, reducing the lag time from up to several months. Near-real-time estimated uncertainty is determined using data qualified as V3 Level 2.0 AOD and considering the difference between the AOD computed with the pre-field calibration and AOD computed with pre-field and post-field calibration. This assessment provides a near-real-time uncertainty estimate for which average differences of AOD suggest a +0.02 bias and one sigma uncertainty of 0.02, spectrally, but the bias and uncertainty can be significantly larger for specific instrument deployments. Long-term monthly averages analyzed for the entire V3 and V2 databases produced average differences (V3–V2) of +0.002 with a ±0.02 SD (standard deviation), yet monthly averages calculated using time-matched observations in both databases were analyzed to compute an average difference of −0.002 with a ±0.004 SD. The high statistical agreement in multiyear monthly averaged AOD validates the advanced automatic data quality control algorithms and suggests that migrating research to the V3 database will corroborate most V2 research conclusions and likely lead to more accurate results in some cases.

David M. Giles↗

DOE OSTI2025

Real-Time Automated pH Control within Batch Processes Relying on Raman pH Measurement

Nuclear fission is an energy source that can provide consistent power with very low associated carbon emissions. However, management of the used nuclear fuel is an important aspect of the application of nuclear power. Recycling of useful components from used fuel is an attractive option, but this involves chemical processing of the fuel. Possible chemical separation technologies that might be used in this regard are sensitive to solution pH. Raman spectroscopy is a promising technique for monitoring the pH of solutions in real time. Classical pH probes are too fragile to be used in the harsh environments encountered in nuclear fuel processing. Raman probes are robust and can withstand these harsh environments to track pH. Coupled with chemometric analysis, the demonstration of the use of Raman spectroscopy to track and predict the pH in carboxylate-buffered systems is made possible. Utilizing this spectroscopy in conjunction with Programmable Logic Controllers mimics industrial control systems used in many modern industrial settings. This showcases a pragmatic approach toward leveraging Raman spectroscopy and chemometric model outputs as inputs for a real-time control system. The model to predict pH created by chemometrics proved to be successful in tracking pH. The optimal pH for TALSPEAK extraction of lanthanides and actinides from aqueous solution is known to proceed in a narrow pH range of around pH = 2.8 ± 0.1. This study uses Raman optical monitoring and automated control to return and maintain solution pH within this range after acid or base perturbations move the solution pH well outside this region. Root-mean-square errors show that pH changes measured using Raman spectroscopy on the batch process solution are reliably measured and used to automatically correct and maintain solution pH. Measurement of solution pH tracks favorably with electrochemical pH probe comparison measurements. As a result, the ability to showcase Raman spectroscopy paired with chemometrics analysis acts as a durable, better alternative data source compared to traditional pH probes to optimize the separation efficiency in the used nuclear fuel processing.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2015

Processing ultrasonic inspection data from multiple scan patterns for turbine rotor weld build-up evaluations

The study presents a data processing methodology for weld build-up using multiple scan patterns. To achieve an overall high probability of detection for flaws with different orientations, an inspection procedure with three different scan patterns is proposed. The three scan patterns are radial-tangential longitude wave pattern, axial-radial longitude wave pattern, and tangential shear wave pattern. Scientific fusion of the inspection data is implemented using volume reconstruction techniques. The idea is to perform spatial domain forward data mapping for all sampling points. A conservative scheme is employed to handle the case that multiple sampling points are mapped to one grid location. The scheme assigns the maximum value for the grid location to retain the largest equivalent reflector size for the location. The methodology is demonstrated and validated using a realistic ring of weld build-up. Tungsten balls and bars are embedded to the weld build-up during manufacturing process to represent natural flaws. Flat bottomed holes and side drilled holes are installed as artificial flaws. Automatic flaw identification and extraction are demonstrated. Results indicate the inspection procedure with multiple scan patterns can identify all the artificial and natural flaws.

42 ENGINEERING↗

NASA NTRS1991

Fully automatic guidance and control for rotorcraft nap-of-the-Earth flight following planned profiles. Volume 1: Real-time piloted simulation

Developing a single-pilot, all-weather nap-of-the-earth (NOE) capability requires fully automatic NOE (ANOE) navigation and flight control. Innovative guidance and control concepts are investigated in a four-fold research effort that: (1) organizes the on-board computer-based storage and real-time updating of NOE terrain profiles and obstacles in course-oriented coordinates indexed to the mission flight plan; (2) defines a class of automatic anticipative pursuit guidance algorithms and necessary data preview requirements to follow the vertical, lateral, and longitudinal guidance commands dictated by the updated flight profiles; (3) automates a decision-making process for unexpected obstacle avoidance; and (4) provides several rapid response maneuvers. Acquired knowledge from the sensed environment is correlated with the forehand knowledge of the recorded environment (terrain, cultural features, threats, and targets), which is then used to determine an appropriate evasive maneuver if a nonconformity of the sensed and recorded environments is observed. This four-fold research effort was evaluated in both fixed-based and moving-based real-time piloted simulations, thereby, providing a practical demonstration for evaluating pilot acceptance of the automated concepts, supervisory override, manual operation, and re-engagement of the automatic system. Volume one describes the major components of the guidance and control laws as well as the results of the piloted simulations. Volume two describes the complete mathematical model of the fully automatic guidance system for rotorcraft NOE flight following planned flight profiles.

Clement, Warren F.↗

NASA NTRS1991

Fully Automatic Guidance and Control for Rotorcraft Nap-of-the-earth Flight Following Planned Profiles. Volume 2: Mathematical Model

Developing a single-pilot, all-weather nap-of-the-earth (NOE) capability requires fully automatic NOE (ANOE) navigation and flight control. Innovative guidance and control concepts are investigated in a four-fold research effort that: (1) organizes the on-board computer-based storage and real-time updating of NOE terrain profiles and obstacles in course-oriented coordinates indexed to the mission flight plan; (2) defines a class of automatic anticipative pursuit guidance algorithms and necessary data preview requirements to follow the vertical, lateral, and longitudinal guidance commands dictated by the updated flight profiles; (3) automates a decision-making process for unexpected obstacle avoidance; and (4) provides several rapid response maneuvers. Acquired knowledge from the sensed environment is correlated with the forehand knowledge of the recorded environment (terrain, cultural features, threats, and targets), which is then used to determine an appropriate evasive maneuver if a nonconformity of the sensed and recorded environments is observed. This four-fold research effort was evaluated in both fixed-base and moving-base real-time piloted simulations; thereby, providing a practical demonstration for evaluating pilot acceptance of the automated concepts, supervisory override, manual operation, and re-engagement of the automatic system. Volume one describes the major components of the guidance and control laws as well as the results of the piloted simulations. Volume two describes the complete mathematical model of the fully automatic guidance system for rotorcraft NOE flight following planned flight profiles.

Clement, Warren F.↗

NASA NTRS2024

Automation of the ICME Workflow Incorporating Material Digital Twins at Different Length Scales Within a Robust Information Management System

Recent successes in Integrated Computational Materials Engineering (ICME) have demonstrated the potential in designing ‘fit-for-purpose’ materials for a given application in a cost and time efficient manner. However, the material design process must contain a level of judicious automation in the material decision process; that is implementing some optimization algorithms to truly enable the benefits of ICME, particularly when considering materials at multiple length and time scales. Furthermore, the ability to effectively store developed material models, experimental data used for validation, and link models at multiple length and time scales must be implemented to ensure traceability across the material design process, such that the data gathered can be leveraged towards efficient material design. To enable such an optimization scheme a robust framework must exist: (1) that can capture changes made at a given length scale, (2) automatically propagate changes upstream to the highest scale, and (3) evaluate the material’s performance at the structural level. In this work, a developed framework for tracking material changes, automatically running the necessary simulations to determine the properties at the next highest scale, and saving each iteration of the design process to maintain the application’s digital thread is presented for polymer matrix composites (PMCs). The Automated Information Management Across Organizations and Scales (AIMAOS) program offers users an interactive graphical user interface (GUI) for defining constituent materials, building lamina and laminates, and applying effective laminate properties to finite element and composite optimization third party software. At each length scale, the necessary input files are automatically written, and subsequent analysis tools are called to solve for effective properties at the next scale, which are then read by the AIMAOS tool and displayed to the user. As changes are made to the material at lower length scales, information is automatically propagated upstream to higher length scales, and changes made are automatically tracked and versioned to maintain traceability during the design process. The AIMAOS tools serves as the first step in enabling optimized design of composites from the nano to the macroscale for a given application.

Brandon L Hearley↗

DOE OSTI2014

Supervised detection of anomalous light curves in massive astronomical catalogs

The development of synoptic sky surveys has led to a massive amount of data for which resources needed for analysis are beyond human capabilities. In order to process this information and to extract all possible knowledge, machine learning techniques become necessary. Here we present a new methodology to automatically discover unknown variable objects in large astronomical catalogs. With the aim of taking full advantage of all information we have about known objects, our method is based on a supervised algorithm. In particular, we train a random forest classifier using known variability classes of objects and obtain votes for each of the objects in the training set. We then model this voting distribution with a Bayesian network and obtain the joint voting distribution among the training objects. Consequently, an unknown object is considered as an outlier insofar it has a low joint probability. By leaving out one of the classes on the training set, we perform a validity test and show that when the random forest classifier attempts to classify unknown light curves (the class left out), it votes with an unusual distribution among the classes. This rare voting is detected by the Bayesian network and expressed as a low joint probability. Our method is suitable for exploring massive data sets given that the training process is performed offline. We tested our algorithm on 20 million light curves from the MACHO catalog and generated a list of anomalous candidates. After analysis, we divided the candidates into two main classes of outliers: artifacts and intrinsic outliers. Artifacts were principally due to air mass variation, seasonal variation, bad calibration, or instrumental errors and were consequently removed from our outlier list and added to the training set. After retraining, we selected about 4000 objects, which we passed to a post-analysis stage by performing a cross-match with all publicly available catalogs. Within these candidates we identified certain known but rare objects such as eclipsing Cepheids, blue variables, cataclysmic variables, and X-ray sources. For some outliers there was no additional information. Among them we identified three unknown variability types and a few individual outliers that will be followed up in order to perform a deeper analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Massive all-atom analysis of 2D materials with quantum properties (Final report)

Improvements in microscopy have enabled the acquisition of data at a scale that is difficult to process manually, making automated machine learning approaches to analyzing experimental images essential. In this project, we developed and applied machine learning (ML) workflows for atomic resolution scanning transmission electron microscopy (STEM) images. This development included improving both methodology as well as generating user-friendly codes. We developed machine learning architectures which, after training, automatically identify the location and types of defects throughout a material. We used these data to produce class-averaged images of 2D atomic coordinates with up to 0.3 pm precision, uncovering the structure and oscillations of long-range strain fields around point defects in WSe 2-2x Te 2x . We also resolved a long-standing problem in this field in the training of ML models, a lack of labeled experimental data, by developing a cycle-GAN that transformed simulated-generated labeled data into labeled data indistinguishable from experiment and therefore suitable for training. This removed the remaining parts of the ML data processing workflow where human intervention was still critical and therefore a bottleneck to working at scale. Codes have been developed and released for this full machine learning workflow. ML approaches to partially automate STEM acquisition were also developed. Finally we applied ML and other advanced data processing methods to several materials science problems in two-dimensional materials, including studying the evolution of hyperuniformity with defect concentration in WSe2, understanding phase transformations in transition metal dichalcogenides during in-situ heating in the STEM, and exploring how 2D interfaces transform from twisted into aligned structures.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2026

A total of 19 months of daily weather logging on the US east coast: the WFIP3 event log

The Third Wind Forecast Improvement Project (WFIP3) is a multi-institutional field campaign designed to advance the understanding and prediction of the offshore atmospheric boundary layer along the US east coast. Extending from February 2024 through August 2025, WFIP3 combines long-term coastal and offshore measurements with targeted modeling and forecasting efforts. This data paper presents the WFIP3 event log, a curated record of 578 d of meteorological phenomena and field observations that complements the campaign's extensive high-frequency datasets. The event log provides both manually documented daily weather discussions and automatically derived indicators of atmospheric processes – including low-level jets, wind ramps, extreme wind veer, and weak wind conditions – based on observations from scanning lidars deployed at three coastal and offshore sites. The dataset offers structured metadata, standardized time and site identifiers, and consistent terminology to facilitate its integration with WFIP3's observational and modeling data products. The log supports diverse applications, from model evaluation and forecast verification to the selection of case studies on offshore boundary-layer dynamics. The WFIP3 event log is publicly available through the US Department of Energy's Wind Data Hub, providing the research community with a transparent and enduring contextual reference for the interpretation and use of WFIP3 measurements.

17 WIND ENERGY↗

NASA NTRS1973

Utilization of ERTS-1 data to monitor and classify eutrophication of inland lakes

The author has identified the following significant results. Significant findings are: (1) one-acre lakes and one-acre islands are detectable; (2)removal of atmospheric parameters derived from RPMI measurements show test lakes to have reflectances of 3.1 to 5.5% in band 4 and 0.3 to 2.3% in band 5; (3) failure to remove reflectance caused by atmosphere results in errors up to 500% in computing lake reflectance from ERTS-1 data; (4) in band 4, up to seven reflectance levels were observed in test lakes; (5) reflectance patterns have been displayed on a color-coded TV monitor and on computer-generated gray scales; (6) deep and shallow water can be separated by a trained photointerpreter and automatic machine processing, with estimates of water depth possible in some cases; (7) RPMI provides direct spectral signature measurements of lakes and lake features such as algal scums and floating plants; (8) a method is reported for obtaining lake color, as estimated by Forel-Ule standards, from ERTS-1 data; (9) a strong correlation between browner water color, diminishing water transparency; and (10) classifying lake eutrophication by observation of surface scums or macrophytes in shallow water seems straightforward.

Rogers, R. H.↗

DOE OSTI2019

Graph Generation with a Focusing Lexicon

Reference datasets are an essential part of developing effective data exploration tools. Many reference datasets are synthetic in nature, seeding real-world properties into generation routines. Others are laboriously curated from original source materials. This is a time-consuming process that requires significant domain expertise all along the process. This paper presents a third path: deriving a network from real-world source materials using automated tools. By properly grounding the tools in a small collection of curated documents, a larger corpus be automatically processed with greater confidence. The end system leverages human knowledge (in the initial document curation), but with much less time commitment enabled by automation. Our ultimate goal is to produce reference datasets, derived from real-world data that capture aspects salient to a specific activity in the context of all of the activities in the data-set. Our reference datasets take the form of graphs.

Chandra Shekar, Mayanka↗