Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

The NANOGrav 11-year Data Set: High-Precision Timing of 45 Millisecond Pulsars

We present high-precision timing data over time spans of up to 11 years for 45 millisecond pulsars observed as part of the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) project, aimed at detecting and characterizing low-frequency gravitational waves. The pulsars were observed with the Arecibo Observatory and/or the Green Bank Telescope at frequencies ranging from 327 MHz to 2.3 GHz. Most pulsars were observed with approximately monthly cadence, and six high-timing-precision pulsars were observed weekly. All were observed at widely separated frequencies at each observing epoch in order to fit for time-variable dispersion delays. We describe our methods for data processing, time-of-arrival (TOA) calculation, and the implementation of a new, automated method for removing outlier TOAs. We fit a timing model for each pulsar that includes spin, astrometric, and (for binary pulsars) orbital parameters; time-variable dispersion delays; and parameters that quantify pulse-profile evolution with frequency. The timing solutions provide three new parallax measurements, two new Shapiro delay measurements, and two new measurements of significant orbital-period variations. We fit models that characterize sources of noise for each pulsar. We find that 11 pulsars show significant red noise, with generally smaller spectral indices than typically measured for non-recycled pulsars, possibly suggesting a different origin. A companion paper uses these data to constrain the strength of the gravitational-wave background

Arzoumanian, Zaven↗
NASA NTRS2016

DeepSAT: A Deep Learning Approach to Tree-Cover Delineation in 1-m NAIP Imagery for the Continental United States

High resolution tree cover classification maps are needed to increase the accuracy of current land ecosystem and climate model outputs. Limited studies are in place that demonstrates the state-of-the-art in deriving very high resolution (VHR) tree cover products. In addition, most methods heavily rely on commercial softwares that are difficult to scale given the region of study (e.g. continents to globe). Complexities in present approaches relate to (a) scalability of the algorithm, (b) large image data processing (compute and memory intensive), (c) computational cost, (d) massively parallel architecture, and (e) machine learning automation. In addition, VHR satellite datasets are of the order of terabytes and features extracted from these datasets are of the order of petabytes. In our present study, we have acquired the National Agriculture Imagery Program (NAIP) dataset for the Continental United States at a spatial resolution of 1-m. This data comes as image tiles (a total of quarter million image scenes with ~60 million pixels) and has a total size of ~65 terabytes for a single acquisition. Features extracted from the entire dataset would amount to ~8-10 petabytes. In our proposed approach, we have implemented a novel semi-automated machine learning algorithm rooted on the principles of "deep learning" to delineate the percentage of tree cover. Using the NASA Earth Exchange (NEX) initiative, we have developed an end-to-end architecture by integrating a segmentation module based on Statistical Region Merging, a classification algorithm using Deep Belief Network and a structured prediction algorithm using Conditional Random Fields to integrate the results from the segmentation and classification modules to create per-pixel class labels. The training process is scaled up using the power of GPUs and the prediction is scaled to quarter million NAIP tiles spanning the whole of Continental United States using the NEX HPC supercomputing cluster. An initial pilot over the state of California spanning a total of 11,095 NAIP tiles covering a total geographical area of 163,696 sq. miles has produced true positive rates of around 88 percent for fragmented forests and 74 percent for urban tree cover areas, with false positive rates lower than 2 percent for both landscapes.

Imagery↗
NASA NTRS2004

Data processing pipeline with transaction-oriented data sharing

This paper makes three contributions to the area of modern science data processing systems. First, the paper describes the science data processing pipeline, developed at the Multi-mission Image Processing Lab of JPL, for transforming raw space data into high quality image data and automating the distribution of data using a high-performance file transaction service. File Exchange Interface is the file transaction service developed MIPL. Second, it presents the FEI component architecture in the are of file transaction management, security,a nd file integrity verfication. Finally, the paper presents the federated model for the FEI service to demonstrate how to create a pool of file trasaction services to support load balancing and service fallover, and simplify service management.

science data processing↗
NASA NTRS2004

A Simple, Scalable, Script-based Science Processor

The production of Earth Science data from orbiting spacecraft is an activity that takes place 24 hours a day, 7 days a week. At the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC), this results in as many as 16,000 program executions each day, far too many to be run by human operators. In fact, when the Moderate Resolution Imaging Spectroradiometer (MODIS) was launched aboard the Terra spacecraft in 1999, the automated commercial system for running science processing was able to manage no more than 4,000 executions per day. Consequently, the GES DAAC developed a lightweight system based on the popular Per1 scripting language, named the Simple, Scalable, Script-based Science Processor (S4P). S4P automates science processing, allowing operators to focus on the rare problems occurring from anomalies in data or algorithms. S4P has been reused in several systems ranging from routine processing of MODIS data to data mining and is publicly available from NASA.

Lynnes, Christopher↗
NASA NTRS2001

NASA IVHM Technology Experiment for X-vehicles (NITEX)

The purpose of the NASA IVHM Technology Experiment for X-vehicles (NITEX) is to advance the development of selected IVHM technologies in a flight environment and to demonstrate the potential for reusable launch vehicle ground processing savings. The technologies to be developed and demonstrated include system-level and detailed diagnostics for real-time fault detection and isolation, prognostics for fault prediction, automated maintenance planning based on diagnostic and prognostic results, and a microelectronics hardware platform. Complete flight The Evolution of Flexible Insulation as IVHM consists of advanced sensors, distributed data acquisition, data processing that includes model-based diagnostics, prognostics and vehicle autonomy for control or suggested action, and advanced data storage. Complete ground IVHM consists of evolved control room architectures, advanced applications including automated maintenance planning and automated ground support equipment. This experiment will advance the development of a subset of complete IVHM.

Sandra, Hayden↗
DOE OSTI2025

An automated integrated web-based smart tool for open stope design

The Stability Graph is a widely used tool for the design of open stopes in underground mining. Many users of the Stability Graph still apply this design method manually. Although the manual approach has benefits, using multiple graphs and stability number computation charts for each stope surface is time-consuming, even for the experienced mining engineer. Current practice in the use of the method also limits data sharing. This paper presents a StopeSoft web-based tool for open stope stability prediction that is developed on the basis of the Stability Graph method and is available at openstope.com. StopeSoft incorporates flexibility in terms of Stability Graph options and incorporates additional critical factors often overlooked. As a web-based tool, StopeSoft encourages and makes data sharing possible globally, focused on expanding the database and improving the current limitations of the Stability Graph to provide practical, reliable solutions for mining engineers, consultants, and academics. The StopeSoft automated process facilitates the process of open stope stability prediction, saving time and minimizing potential human errors. Statistical treatment of the data accounts for the variability of input parameters to emphasize the probabilistic nature of the Stability Graph method. The probabilistic interpretation of the stability states of stope surfaces eliminates the false feeling of absolute stope performance based on its location on the Stability Graph , as implied by the deterministic approach.

58 GEOSCIENCES↗
DOE OSTI2021

Beyond Visual Analytics: Human-Machine Teaming for AI-Driven Data Sensemaking

"Detect the expected, discover the unexpected" was the founding principle of the field of visual analytics. This mantra implies that human stakeholders, like a domain expert or data analyst, could leverage visual analytics techniques to seek answers to known unknowns and discover unknown unknowns in the course of the data sensemaking process. We argue that in the era of AI-driven automation, we need to recalibrate the roles of humans and machines (e.g., a machine learning model) as teammates. We posit that by realizing human-machine teams as a stakeholder unit, we can better achieve the best of both worlds: automation transparency and human reasoning efficacy. However, this also increases the burden on analysts and domain experts towards performing more cognitively demanding tasks than what they are used to. In this paper, we reflect on the complementary roles in a human-machine team through the lens of cognitive psychology and map them to existing and emerging research in the visual analytics community. We discuss open questions and challenges around the nature of human agency and analyze the shared responsibilities in human-machine teams.

Sensemaking, human-machine teaming, agency, artifi↗
DOE OSTI2023

Panorama 360 (Final Report)

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Panorama 360 (Final Report)

This final technical report from the lead institution, USC grant #DE-SC0012636, serves as the final technical report for collaborative institution UNC-CH grant #DE-SC0012390. The goal was to develop a repository and associated capabilities for data collection, ingestion, and analysis for a broad class of DOE applications that span experimental and simulation science workflows. In particular, this work focuses on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: (1) A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); (2) A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; (3) A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and (4) Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1973

Automation of measurement of spatial damping parameters

A traveling wave unit was built to process measurement data on spatial damping parameters and to simplify statistical treatment of the results. Mean values of the loss coefficient, lengths of flexual waves, and wave propagation velocities for a given set of discrete functions were determined. Results are given in a table.

Akolzin, A. N.↗
NASA NTRS1985

Automation of assertion testing - Grid and adaptive techniques

Assertions can be used to automate the process of testing software. Two methods for automating the generation of input test data are described in this paper. One method selects the input values of variables at regular intervals in a 'grid'. The other, adaptive testing, uses assertion violations as a measure of errors detected and generates new test cases based on test results. The important features of assertion testing are that: it can be used throughout the entire testing cycle; it provides automatic notification of error conditions; and it can be used with automatic input generation techniques which eliminate the subjectivity in choosing test data.

Andrews, D. M.↗
NASA NTRS1987

Measuring Conductor Widths and Spacings Electrically

Widths and spacings of conductor lines on integrated circuits measured electrically by split cross-bridge resistor technique. Speeds evaluation of integrated-circuit fabrication processes. When measurement of probe currents and voltages and reduction of measurement data automated, test structure probed and characterized in less than one second.

Buehler, Martin G.↗
NASA NTRS1991

FTIR Spectroscopic Characterization Of II-VI Semiconductors

Combination of commercial Fourier-transform infrared (FTIR) spectrometer with computer and special-purpose software constitutes highly automated facility for acquisition and processing of infrared transmission or reflection spectral image data. Intended principally to acquire transmission spectra of some compounds of elements in groups II and VI of periodic table. System used to characterize specimens of II/VI alloy semiconductors grown by directional solidification and quenching. Transmission-edge maps helpful in studies of flows, gradients of temperature, and coefficients of diffusion in solidifying melts. Data acquired by system include optical characteristics, and they both verify and complement data obtained by such other techniques as measurements of density and x-ray-dispersion analysis.

Perry, G. L. E.↗
NASA NTRS1992

Geostationary satellite position determination for common-view two-way time transfer measurements

In common-view two-way time transfer, each earth station receives an unwanted return signal from its own transmission as well as the desired signal from the other earth station. National Institute of Standards and Technology (NIST), the National Research Council (NRC), and the U.S. Naval Observatory (USNO) have been cooperating in a three-corner common-view two-way time transfer experiment. Some systematic effects are known to depend on the position of the satellite (Sagnac effect and the cross-correlation pulling of the pseudo-random codes). A method is presented for deriving accurate satellite ranges from each of three stations doing common-view two-way satellite time transfer measurements, when one (and only one) station also takes ranging measurements on its 'unwanted return signal' for a brief period. The method is applied to determine the variations in position of the satellite used over the course of the NIST/NRC/USNO SBS-3 experiment, with ranging data taken at NRC, where no additional hardware was required to automate the process. The fit and extrapolation which are employed in this method have an estimated precision of 2 m. If the delays of SBS-3 satellite KU band transponder and earth station equipment were measured accurately as well as the tropospheric refractions were well modeled and corrected, we would expect a ranging accuracy of 2.5 m and satellite positioning accuracy would be 200 m (latitude) 50 m (longitude) and 20 m (height above ellipsoid).

Zhuang, Qixiang↗
NASA NTRS2000

Cavendish Balance Automation

This is the final report for a project carried out to modify a manual commercial Cavendish Balance for automated use in cryostat. The scope of this project was to modify an off-the-shelf manually operated Cavendish Balance to allow for automated operation for periods of hours or days in cryostat. The purpose of this modification was to allow the balance to be used in the study of effects of superconducting materials on the local gravitational field strength to determine if the strength of gravitational fields can be reduced. A Cavendish Balance was chosen because it is a fairly simple piece of equipment for measuring gravity, one the least accurately known and least understood physical constants. The principle activities that occurred under this purchase order were: (1) All the components necessary to hold and automate the Cavendish Balance in a cryostat were designed. Engineering drawings were made of custom parts to be fabricated, other off-the-shelf parts were procured; (2) Software was written in LabView to control the automation process via a stepper motor controller and stepper motor, and to collect data from the balance during testing; (3)Software was written to take the data collected from the Cavendish Balance and reduce it to give a value for the gravitational constant; (4) The components of the system were assembled and fitted to a cryostat. Also the LabView hardware including the control computer, stepper motor driver, data collection boards, and necessary cabling were assembled; and (5) The system was operated for a number of periods, data collected, and reduced to give an average value for the gravitational constant.

Thompson, Bryan↗
NASA NTRS2007

Automated Identification of Nucleotide Sequences

STITCH is a computer program that processes raw nucleotide-sequence data to automatically remove unwanted vector information, perform reverse-complement comparison, stitch shorter sequences together to make longer ones to which the shorter ones presumably belong, and search against the user s choice of private and Internet-accessible public 16S rRNA databases. ["16S rRNA" denotes a ribosomal ribonucleic acid (rRNA) sequence that is common to all organisms.] In STITCH, a template 16S rRNA sequence is used to position forward and reverse reads. STITCH then automatically searches known 16S rRNA sequences in the user s chosen database(s) to find the sequence most similar to (the sequence that lies at the smallest edit distance from) each spliced sequence. The result of processing by STITCH is the identification of the most similar well-described bacterium. Whereas previously commercially available software for analyzing genetic sequences operates on one sequence at a time, STITCH can manipulate multiple sequences simultaneously to perform the aforementioned operations. A typical analysis of several dozen sequences (length of the order of 103 base pairs) by use of STITCH is completed in a few minutes, whereas such an analysis performed by use of prior software takes hours or days.

Osman, Shariff↗
NASA NTRSDate not supplied

Artificial Intelligence (AI) Methods for Augmenting the IMPACT Tool Evidence Library

Development of the Evidence Library for use with the IMPACT probability risk assessment tool took several years and involved a staggering amount of effort from a multi-disciplinary team. A very significant amount of the labor effort to collect, assess and finalize the Clinical Finding Form (CliFF) for each of the 119 medical conditions was provided by physician subject matter experts from the Exploration Medical Capability (ExMC) Element Clinical and Science Team. Many AI tools such as ChatGPT are excellent at summarizing large amounts of information and the current project was initiated to determine how such tools might streamline laborious processes, e.g., review and summarization of many scientific research publications, to execute key steps more efficiently in the process of developing CliFFs. The process for collecting the evidence which is found in the CliFFs is well documented in the Evidence Library Methods document (ELM; HRP-48036*). Using ELM and the CliFF development instructions as a guideline, a team of developers is leveraging Microsoft Azure AI tools and services along with open-source frameworks, to construct an AI-assisted automated pipeline. This pipeline is designed to search, retrieve, and process the necessary data sources, and ultimately help generate the final version of a CliFF. Currently, the large language model evaluates the relevance of each source material to spaceflights, either as direct evidence or as an analog. Additionally, the model assists in extracting keywords and generating brief summaries to enhance augmented retrieval and search processes in later stages of CliFF development. Once the data is ready, the model can perform semantic search and retrieval, generating and extracting valuable information for the CliFF. For instance, it can handle epidemiological statistical data, such as incidence rates and the likelihood of best or worst-case scenarios. The steps that required reading and summarizing articles were viewed as providing the greatest return on investment since large language models are very efficient and accurate in summarizing large amounts of text. Since labor effort to complete the original CliFF was not recorded with sufficient granularity, comparisons with an AI tool-generated CliFF will provide merely an approximation of time saved. Upon completion of the process, the CliFF for the medical condition “appendicitis” generated with the support of AI-based methods will serve as a proof-of-concept and will be compared to the original appendicitis CliFF to determine if use of the tools resulted in content and conclusory similarity. Based upon the results from face validation of the two CliFFs, modifications to the process will be made if necessary and additional condition CliFFs will be evaluated. Ultimately, CliFFs for the entire set of medical conditions will be created with the assistance of AI tools. Depending on the cost savings realized, CliFFs for additional medical conditions can be created to expand the Evidence Library. Future direction includes specifying the characteristics of the reviewer (prompting the AI tools to generate output assuming the reviewer is a sub-specialist physician, or nurse or EMT/medic) to determine if the effects on AI-generated output are different based on knowledge, skills and abilities. *Exploration Medical Capability Evidence Library Methods, HRP-48036 Rev A, July 2022.

Ali Al↗
DOE OSTI2020

AUTONOMIE VID

Autonomie Vehicle Information Database (VID) offers a comprehensive list of vehicle specifications since 1990. The database details more than 65,000 vehicles with hundreds of attributes. The database is the result of the development of a general automated data collection framework as well as the development of building blocks for processing, cleaning, integrating and analyzing complex data. The data has undergone several layers of outlier detections processes, machine learning based imputations methods have been used to deal with missing data problems, and new fields have been created according to the rules of feature engineering. Thanks to this streamlined data pipelines, the resulting processed aggregated data should deliver a unique level of information to the user in which the content can be efficiently maintained and updated.

Moswd, Ayman↗