Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62

Laying the Foundations for FAIR-er Science: ISA and the LSDA Data Submission Process in NASA's Evolving Data Management Environment

The Life Sciences Data Archive (LSDA) archives data resulting from research on the effects of spaceflight on humans and the development of countermeasures to mitigate spaceflight hazards. Archivists work with researchers to ensure that unique and high value data products and their metadata are preserved and managed to support current and future research. Currently, LSDA is updating its procedures and data submission requirements in response to the evolving data preservation environment at NASA. LSDA is implementing best practices for research data management through the establishment of clear data submission guidelines, integration of the FAIR (Findability, Accessibility, Interoperability, Reusability) principles, and use of the ISA (Investigation, Study, Assay) research metadata framework for data discoverability and transparency into the data management processes. These changes directly impact LSDA’s requirements for research data submissions. The newly revised Research Data Submission Agreement (RDSA), formerly the Data Submission Agreement (DSA), introduces ISA-compatible metadata collection standards to LSDA’s process. Adherence to LSDA’s data submission guidelines enhances the FAIR-ness of the repository’s collections for future users. This presentation will discuss (1) how submission of research data and associated metadata are impacted by current data management policies, (2) benefits of the adoption of FAIR principles and the ISA metadata framework for retrospective studies utilizing existing LSDA datasets and historic data collections, and (3) the support LSDA will provide to researchers during this transition.

LSDA↗

Processing Raw HST Data With Up-to-Date Calibration Data

On-the-Fly Reprocessing (OTFR) is a collection of data-processing routines that work within the context of the Hubble Space Telescope (HST) pipeline data-flow system. The purpose served by OTFR is to generate, on demand, scientifically useful data products from raw HST data stored in an archive. First, on the basis of the requested final data products, OTFR retrieves the corresponding sets of raw data from the archives. Next, OTFR processes the raw data sets to remove artifacts and to establish proper header and other template information. Finally, the calibration routines appropriate to the specific data sets are invoked to produce the requested data products, and the data products are released to an archive distribution system for transmission to the requesting party. OTFR offers two notable advantages: (1) Inasmuch as calibrated data occupy about 8 times as much storage space as do raw data, by obviating storage of calibrated data, OTFR reduces the storage capacity needed by the archive; and (2) the calibration routines can be updated to give requesters the benefit of the most recent calibrations.

Miller, Warren↗

TNSL Overview

Thermal neutron scattering law (TNSL) data describe low-energy neutrons scattering off of bound materials, and can have a significant impact on modeling any system with slow neutrons, including nuclear reactors. Previous work to introduce TNSL data to neutron transport codes at LLNL focused on COG and TART [1], with the limitation that these codes require highly specialized data processing and formatting. We have recently increased efforts to process TNSL data with the central LLNL nuclear data processing code FUDGE, to be stored in the generalized nuclear database structure (GNDS) for use in any general transport code with the ability to read GNDS data. The first step in this effort is to verify that the TNSL processing with FUDGE yields results comparable to results obtained using the LANL nuclear data processing code NJOY. The next step is to verify the transport of thermal neutrons in Mercury (a Monte Carlo code) and Ardra (a deterministic code) against one another, as well as against the LANL Monte Carlo neutron transport code MCNP. This verification step has not been completed, due to a number of discrepancies between results obtained using differently processed data. There is ongoing effort to understand differences between FUDGE and NJOY. Finally, we map out our current capability to validate TNSL data against benchmark systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Processing ultrasonic inspection data from multiple scan patterns for turbine rotor weld build-up evaluations

The study presents a data processing methodology for weld build-up using multiple scan patterns. To achieve an overall high probability of detection for flaws with different orientations, an inspection procedure with three different scan patterns is proposed. The three scan patterns are radial-tangential longitude wave pattern, axial-radial longitude wave pattern, and tangential shear wave pattern. Scientific fusion of the inspection data is implemented using volume reconstruction techniques. The idea is to perform spatial domain forward data mapping for all sampling points. A conservative scheme is employed to handle the case that multiple sampling points are mapped to one grid location. The scheme assigns the maximum value for the grid location to retain the largest equivalent reflector size for the location. The methodology is demonstrated and validated using a realistic ring of weld build-up. Tungsten balls and bars are embedded to the weld build-up during manufacturing process to represent natural flaws. Flat bottomed holes and side drilled holes are installed as artificial flaws. Automatic flaw identification and extraction are demonstrated. Results indicate the inspection procedure with multiple scan patterns can identify all the artificial and natural flaws.

42 ENGINEERING↗

200-IA-1 Operable Unit Human Health Risk and Kd Screen

The purpose of this environmental calculation file (ECF) is to provide the following: Document the data processing and data reduction steps taken to prepare the 200-IA-1 Operable Unit (OU) data set that will be used to calculate the sample-specific screening level human health risk evaluation; Document the data processing and data reduction steps taken to prepare the 200-IA-1 OU data set that will be used to identify analytes that could potentially impact groundwater in the future beneath the 200-IA-1 OU representative waste sites; Document the assumptions, equations, and methodologies used to calculate the screening levels for human health cancer risks and noncancer hazards for each representative waste site assigned to the 200-IA-1 OU. Individual measured soil concentrations from 0 to 4.6 m (15 ft) below ground surface (bgs) (shallow vadose zone) are used to calculate the total excess lifetime cancer risk (ELCR) and hazard index (HI) for the outdoor worker scenario to determine if there is a basis for remedial action. Individual measured soil concentrations from the ground surface to the groundwater table are used for the distribution coefficient (Kd) screen to identify analytes that could potentially impact groundwater in the future beneath the 200-IA-1 OU representative waste sites. This ECF supports DOE/RL-2020-51, 200-IA-1 OU Focused Feasibility Study, under the Comprehensive Environmental Response, Compensation, and Liability Act of 1980 (CERCLA). A risk characterization based upon the evaluation of the health risk estimates developed in this ECF will be presented in the focused feasibility study report.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗

NASA GeneLab: Open Science for Life in Space

The NASA GeneLab project (genelab.nasa.gov) seeks to get the most from space-relevant biology experiments by providing and maintaining a public database consisting of DNA, RNA, protein, and metabolite data from spaceflight experiments. Since these types of data, referred to as omics data, are difficult to understand for non-bioinformaticians, the GeneLab data processing team works with the scientific community to develop methods to process these data. The processed data found on GeneLab reveals information about which genes are turned on and turned off in the space environment, which helps us understand how space changes our biology and how we can best mitigate these effects to travel deeper into space.

Jonathan Oribello↗

CLAS12 remote data-stream processing using ERSAP framework

Implementing a physics data processing application is relatively straightforward with the use of current containerization technologies and container image runtime services, which are prevalent in most high-performance computing (HPC) environments. However, the process is complicated by the challenges associated with data provisioning and migration, impacting the ease of workflow migration and deployment. Transitioning from traditional file-based batch processing to data-stream processing workflows is suggested as a method to streamline these workflows. This transition not only simplifies file provisioning and migration but also significantly reduces the necessity for extensive disk space. Data-stream processing is particularly effective for real-time processing during data acquisition, thereby enhancing data quality assurance. This paper introduces the integration of the JLAB CLAS12 event reconstruction application within the ERSAP data-stream processing framework that facilitates the execution of streaming event reconstruction at a remote data center and enables the return streaming of reconstructed events to JLAB while circumventing the need for temporary data storage throughout the process.

Gyurjyan, Vardan↗

Wind information display system user's manual

The Wind Information Display System (WINDS) provides flexible control through system-user interaction for collecting wind shear data, processing this data in real time, displaying the processed data, storing raw data on magnetic tapes, and post-processing raw data. The data are received from two asynchronous laser Doppler velocimeters (LDV's) and include position, velocity and intensity information. The raw data is written onto magnetic tape for permanent storage and is also processed in real time to depict wind velocities in a given spacial region.

Roe, J.↗

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗

Intrinsically stretchable neuromorphic devices for on-body processing of health data with artificial intelligence

For leveraging wearable technologies to advance precision medicine, personalized and learning-based analysis of continuously acquired health data is indispensable, for which neuromorphic computing could provide the most efficient implementation of artificial intelligence (AI) data processing. For realizing on-body neuromorphic computing, skin-like stretchability is required, but yet to be combined with the suite of desired neuromorphic metrics, including linear, symmetric weight update, and sufficient state retention, for achieving high computing efficiency. Here, we report an intrinsically stretchable neuromorphic device based on an electrochemical transistor, which provides a large number (>800) of states, linear/symmetric weight update, excellent switching endurance (>100 million), good state retention (>10 4 s), together with high stretchability of 100% strain. Further integration into a prototype array successfully realized the implementation of vector-matrix multiplication even at 100% strain. Finally, we demonstrate the feasibility of implementing AI-based classification of health signals (as exemplified by electrocardiograms) with a high accuracy that is minimally influenced by the stretched state of the neuromorphic hardware. Finally, this work breaks the ground for combining AI data analysis into skin-like wearable electronics for achieving human-integrated/mimetic intelligent systems.

60 APPLIED LIFE SCIENCES↗

Investigations in adaptive processing of multispectral data

Adaptive data processing procedures are applied to the problem of classifying objects in a scene scanned by multispectral sensor. These procedures show a performance improvement over standard nonadaptive techniques. Some sources of error in classification are identified and those correctable by adaptive processing are discussed. Experiments in adaptation of signature means by decision-directed methods are described. Some of these methods assume correlation between the trajectories of different signature means; for others this assumption is not made.

Kriegler, F. J.↗

Earth Science System of the Future: Observing, Processing, and Delivering Data Products Directly to Users

Advancement of our predictive capabilities will require new scientific knowledge, improvement of our modeling capabilities, and new observation strategies to generate the complex data sets needed by coupled modeling networks. New observation strategies must support remote sensing from a variety of vantage points and will include "sensorwebs" of small satellites in low Earth orbit, large aperture sensors in Geostationary orbits, and sentinel satellites at L1 and L2 to provide day/night views of the entire globe. Onboard data processing and high speed computing and communications will enable near real-time tailoring and delivery of information products (i.e., predictions) directly to users.

Crisp, David↗

Evolution of HEP Processing Frameworks

HEP data-processing software must support the disparate physics needs of many experiments. For both collider and neutrino environments, HEP experiments typically use data-processing frameworks to manage the computational complexities of their large-scale data processing needs. Data-processing frameworks are being faced with new challenges this decade. The computing landscape has changed from the past three decades of homogeneous single-core x86 batch jobs running on grid sites. Frameworks must now work on a heterogeneous mixture of different platforms: multi-core machines, different CPU architectures, and computational accelerators; and different computing sites: grid, cloud, and high-performance computing. We describe these challenges in more detail and how frameworks may confront them. Given their historic success, frameworks will continue to be critical software systems that enable HEP experiments to meet their computing needs. Frameworks have weathered computing revolutions in the past; they will do so again with support from the HEP community

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Completion processing for data communications instructions

Completion processing of data communications instructions in a distributed computing environment with computers coupled for data communications through communications adapters and an active messaging interface (`AMI`), injecting for data communications instructions into slots in an injection FIFO buffer a transfer descriptor, at least some of the instructions specifying callback functions; injecting a completion descriptor for each instruction that specifies a callback function into an injection FIFO buffer slot having a corresponding slot in a pending callback list; listing in the pending callback list callback functions specified by data communications instructions; processing each descriptor in the injection FIFO buffer, setting a bit in a completion bit mask corresponding to the slot in the FIFO where the completion descriptor was injected; and calling by the AMI any callback functions in the pending callback list as indicated by set bits in the completion bit mask.

97 MATHEMATICS AND COMPUTING↗

Completion processing for data communications instructions

Completion processing of data communications instructions in a distributed computing environment with computers coupled for data communications through communications adapters and an active messaging interface (`AMI`), injecting for data communications instructions into slots in an injection FIFO buffer a transfer descriptor, at least some of the instructions specifying callback functions; injecting a completion descriptor for each instruction that specifies a callback function into an injection FIFO buffer slot having a corresponding slot in a pending callback list; listing in the pending callback list callback functions specified by data communications instructions; processing each descriptor in the injection FIFO buffer, setting a bit in a completion bit mask corresponding to the slot in the FIFO where the completion descriptor was injected; and calling by the AMI any callback functions in the pending callback list as indicated by set bits in the completion bit mask.

97 MATHEMATICS AND COMPUTING↗

Iterative algorithms for processing experimental data

The need to solve linear and nonlinear integral equations arise, e.g., in recovering plasma parameters from the data of multichannel diagnostics. The paper presents an iterative method for solving integral equations with a singularity at the upper limit of integration. The method consists in constructing successive approximations and calculating the integral by quadrature formulas in each integration interval. An example of application of the iterative algorithm to numerically solve an integral equation similar to those arising in recovering the plasma density profile from reflectometry data is presented.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Improving public communication through environmental data transparency at Los Alamos National Laboratory - 15306

The Los Alamos National Laboratory (LANL) environmental data span a wide range of media, including air, soil, sediment, water, foodstuff, and biota as well as over 1000 analytes and time periods extending over 4 decades, resulting in over 14 million records to manage and analyze. At a time of budget cuts and rapid social and technological change, the public, regulatory and compliance agencies, and stakeholders demand accelerated data delivery and efficiency, information sharing, and transparency in the dissemination and handling of samples collected in and around LANL. Meeting these requirements, while constraining the associated costs can be challenging; however, LANL has achieved this by adopting a cloud-based database called the Environmental Information Management (EIM) system and its public database counterpart, IntellusNM (Intellus) (www.intellusnm.com). To be consistent with federal initiatives to move government computing into cloud-based services, LANL chose to partner with an existing cloud-based vendor, Locus Technologies, to implement a software-as-a-service model. As an additional cost-saving measure, local regulatory and compliance agencies collaborated with LANL to incorporate their data into EIM and Intellus database. By adopting a single platform cloud-based solution to replace 12 different pre-existing environmental data systems, LANL has streamlined environmental data processes, resulting in over $15 million cost savings through 2015. EIM, LANL's internal cloud database, provides a cradle-to-grave process for data handling. EIM supports LANL's entire data process from detailed sample planning, mobile data collection using iOS mobile devices, to automated bottle labeling, generation of chain of custody, analytical lab electronic data deliverable upload, auto data validation, sample tracking, and custom regulatory and compliance reports. EIM has functionality to assess data in various ways such as the ability to design new formatted reports, a free-form structured query language tool of the entire database, exceedance reports, calculation engine, charting, trends, and statistics. EIM also has interactive geographic information system layers and analyte concentration color contour mapping capabilities. Additionally, and of greater importance to LANL, EIM supports the ability for third-party landowners to review data before release to the public, thus ensuring transparency and confidence in LANL's stewardship of human health and the environment. Intellus, the database counterpart of EIM, is the publicly available view of the entire LANL database and requires no data feeds or transformations. Intellus allows user-friendly access to LANL's environmental data as it is collected, providing the public, scientists, remediation teams, regulators, and stakeholders within hours of receipt. Full transparency in giving the public access to environmental data information provides LANL's overall solution to environmental data management for the public. By making the data easy to obtain, search, and query for interested stakeholders, the system enables confidence in LANL's stewardship of the environment. Working together identifies key topics of improvements, develop strategies and help support innovation and substantial advances within LANL's environmental data management system. (authors)

54 ENVIRONMENTAL SCIENCES↗