Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

Towards Auto-Generated Data Systems

After decades of progress, database management systems (DBMSs) are now the backbones of many data applications that we interact with on a daily basis. Yet, with the emergence of new data types and hardware, building and optimizing new data systems remain as difficult as the heyday of relational databases. In this paper, we summarize our work towards automating the building and optimization of data systems. Drawing from our own experience, we further argue that any automation technique must address three aspects: user specification, code generation, and result validation. We conclude by discussing a case study using videos data processing, along with opportunities for future research towards designing data systems that are automatically generated.

Computer Science↗

Meld: Exploring the Feasibility of a Framework-less Framework

HEP data-processing frameworks are essential ingredients in getting from raw data to physics results. But they are often tricky to use well, and they present a significant learning barrier for the beginning HEP physicist. In addition, existing frameworks typically support rigid, collider-based data models, which do not map well to neutrino-physics experiments like DUNE. Neutrino physicists thus expend significant effort working around framework limitations instead of using a framework that directly supports their needs. Presented here is Meld, a Fermilab R&D project, which intends to address these limitations. By leveraging modern C++ capabilities, state-of-the-art concurrency libraries, and a flexible data model, it is possible for beginning (and seasoned) HEP physicists to execute framework programs easily and efficiently, with minimal coupling to framework-specific constructs. Meld aims to directly support the frameworks needs of neutrino experiments like DUNE as well as the more common collider-based experiments.

Knoepfel, Kyle J.↗

Dual modality endocavity biopsy imaging system and method

A dual modality endocavity imaging and treatment system for detection of cancer and targeted biopsy and treatment procedures, comprising: a housing; a nuclear detector system housed within the housing and configured for detecting nuclear radiation imaging data; an ultrasound detector system housed within the housing for detecting ultrasound imaging data; a needle associated with the housing and adjustably positionable relative thereto; and a data processing module configured to receive the nuclear radiation imaging data and the ultrasound imaging data and to generate and output an image showing the relative position of the needle and an endocavity object of interest. The needle may include a distinct radiation signature to facilitate imaging thereof.

Lall, Terrence↗

Electronic Bottleneck Suppression in Next‐Generation Networks with Integrated Photonic Digital‐to‐Analog Converters

Digital‐to‐analog converters (DAC) are indispensable functional units in signal processing instrumentation and wide‐band telecommunication links for both civil and military applications. As photonic systems are capable of high data throughput and low latency, an increasingly found system limitation stems from the required domain crossing such as digital to analog and electronic to optical. A photonic DAC implementation, in contrast, enables a seamless signal conversion with respect to both energy efficiency and short signal delay, often requiring bulky discrete optical components and electric–optic transformation, hence introducing inefficiencies. Herein, a novel coherent parallel photonic DAC concept along with a 4‐bit experimental prototype capable of performing this DAC without optic–electric–optic domain crossing is introduced. This new paradigm guarantees a linear intensity weighting among bits when operating at high sampling rates (50 GHz), featuring an exceptional sampling efficiency (> 100 GS ) and small footprint (≈1 mm 2 ) in an 8‐bit implementation. Importantly, this photonic DAC enables seamless interfaces of next‐generation data processing hardware with high relevance in data centers, task‐specific compute accelerators such as neuromorphic engines, and network edge processing applications.

Meng, Jiawei↗

Efficient Workflows for Curation of Heterogeneous Data Supporting Modeling of U-Nb Alloy Aging

These are slides from a presentation summarizing a graduate research associate's summer project. The following topics are covered in these slides: data challenges in materials, aging in U-Nb Alloys, Building an Aging Model, Different Phase Trans. in U-Nb, the Challenge, Storing Materials Data, Example Data Source, Organizing Data: What is a Schema?, What does a "XML Schema" look like?, Our Data Schema: Nice and Simple, Storing Data: Materials Data Curation System (MDCS), Problem with MDCS: Slow Data Entry, Getting Literature into MDCS, Staging Data in Excel Document, Final Result: MDCS Records, Analyzing Image Data, Process for Making TTT Diagram, Bottleneck Number 1: Image Analysis, Fitting a TTP Boundary, Fitting a TTP Curve: Comparable Results, How Does it Compare to Our Data?, Image Analysis Workflow, Curating Hardness Records, Hardness Data: Two Key Decisions, Before Peak Age? - Automation, Interactive Viz, Which Transformation?, Microstructure-Informed Model, Tracking the Entire Process, General Problem with Property Models, Pinyon: Toolkit for Managing Model Creation, Tracking Individual Decisions, Jupyter: Docs and Code in One File, Hardness Analysis Workflow, Workflow for Aging Models, and conclusions.

36 MATERIALS SCIENCE↗

Experiences with a Flexible User Research Process to Build Data Change Tools

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects.In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

97 MATHEMATICS AND COMPUTING↗

Leveraging the digital thread for physics-based prediction of microstructure heterogeneity in additively manufactured parts

A major limitation of additive manufacturing (AM) processes is that local conditions of material deposition frequently lead to unintentional heterogeneities in microstructure and properties within a single component, despite nominally uniform process conditions. Up to now, there has been no way to a priori determine the distribution of these heterogeneities, requiring expensive trial-and-error approaches to fabrication, testing, and characterization. Here, a physics-based framework for creating a digital representation of the laser powder bed fusion (PBF) process is proposed to predict the variation in solidification behavior that leads to heterogeneous microstructures in an as-built part. By leveraging in situ process data stored in the part’s digital thread, the scan path and process parameters were input into a heat transfer model which predicted solidification data at the melt pool scale. A two-step unsupervised clustering algorithm was used to first cluster the local solidification conditions (12.5µm 3 voxels) and then to cluster the regional behavior on the scale of multiple scan passes and print layers (250µm 3 super-voxels). This process was used to identify regions with similar solidification characteristics for multiple locations in a Stainless Steel 316-L component. The corresponding as-built part was sectioned and characterized using electron backscatter diffraction (EBSD). Quantitative analysis of the pole figures confirmed that the predicted regions of heterogeneity in the solidification conditions corresponded with differences in the observed microstructure. In conclusion, this work shows a viable path for estimating the microstructural heterogeneity for additively manufactured parts to either limit microstructural variation throughout a part or to enable functionality-based variation of the microstructure.

36 MATERIALS SCIENCE↗

Installation and Testing of a Jorin Visual Process Analyzer

The Jorin Visual Process Analyzer (ViPA) is an on-line instrument that uses video microscope imaging to detect and measure the physical characteristics of dispersed objects within a process stream or laboratory sample. Object analysis is performed by capturing an on-going sequence of single frames from the video feed and relaying the images in real time to a nearby control computer where the ViPA software then processes and transfigures the information from the images into meaningful process data. The ViPA captures and analyzes approximately 15 images per second and continuously records 17 material parameters including size, shape and optical density. The ViPA software uses the measured parameters to differentiate between different classes of objects including organic droplets, gas bubbles, and solid particles. Procurement of this instrument provides a unique capability to support predictive modeling and further understanding of mass transfer during solvent extraction processes. Organic droplet data collected using the ViPA can be used to develop dispersion profiles of the liquid-liquid mixing and disengagements sections for each type of process equipment. These profiles will provide insight into mixing dynamics and will guide the prevention of emulsion formation that leads to system losses. Additionally, the measurement capabilities of the ViPA will provide the input needed to create new two-phase Computational Fluid Dynamics (CFD) models that characterize both mixing and separation operations in the various types of equipment. These models can then be used to improve process efficiency by optimizing operation parameters for each proposed extraction cycle.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

NEON AOP Imaging Spectroscopy Survey of Upper East River Colorado Watersheds: Raw-Space Radiance and Observational Variable Dataset

Lawrence Berkeley National Laboratory (LBNL) contracted the National Ecological Observatory Network Airborne Observation Platform (NEON AOP) to observe watersheds of interest near Crested Butte, CO with remotely sensed data including imaging spectroscopy. The flight box design encompassed the watersheds, and flight design targeted a minimum of 30% overlap between adjacent flight lines. The total area surveyed was 334 km2 across 72 lines. Flight lines were oriented north-south for consistency in solar illumination geometry. Seven days were required to cover the flight area. Some areas were flown additional times to improve data quality or temporal coincidence between airborne acquisitions and field sampling. In total, 102 lines were collected and can be downloaded here: https://portal.nersc.gov/wfsfa/doi-10-15485-1617204/. The instrument used for the acquisition was an Airborne Visible-Infrared Imaging Spectrometer - Next Generation (AVIRIS-NG), which records radiance within 426 contiguous spectral bands in the wavelength range from 380 to 2510 nm. The instrument was flown at a target aboveground altitude of 1000 m for a processed data spatial resolution of 1 m. These data are non-orthorectified, radiance data that require further processing prior to use as a surface reflectance data set.

2018 NEON and 2025 CHESS Campaigns↗

Standardising the “Gregory method” for calculating equilibrium climate sensitivity

The equilibrium climate sensitivity (ECS) – the equilibrium global mean temperature response to a doubling of atmospheric CO 2 – is a high-profile metric for quantifying the Earth system's response to human-induced climate change. A widely applied approach to estimating the ECS is the “Gregory method” (Gregory et al., 2004), which uses an ordinary least squares (OLS) regression between the net radiative flux, N, and surface air temperature anomalies, ΔT, from a 150 year experiment in which atmospheric CO 2 concentrations are quadrupled. The ECS is determined by extrapolating the linear fit to N=0, i.e. the ΔT-intercept, indicating the point at which the system is back in equilibrium. This method has been used to compare ECS estimates across the CMIP5 and CMIP6 ensembles and will likely be a key diagnostic for CMIP7. Despite its widespread application, there is little consistency or transparency between studies in how the climate model data is processed prior to the regression, leading to potential discrepancies in ECS estimates. We identify 32 alternative data processing pathways, varying by differences in global mean weighting, net radiative flux variable, anomaly calculation method, and linear regression fit. Using 44 CMIP6 models, we systematically assess the impact of these choices on ECS estimates and calculate uncertainty ranges using two bootstrap approaches. While the inter-model ECS range is insensitive to the data processing pathway, individual outlier models exhibit notable differences. Approximating a model's native grid cell area (if irregular) with cosine of the latitude can decrease the ECS by 11 %, the choice of N-variable can change the ECS by 6 %, and some anomaly calculation methods can introduce spurious temporal correlations in the processed data. Beyond data processing choices, we also evaluate an alternative linear regression method – total least squares (TLS) – which has a more statistically robust basis than OLS. However, for consistency with previous literature, and given TLS may reduce the ECS compared to OLS (by up to 24 %), thereby making a known bias in the Gregory method worse, we do not feel there is sufficient clarity to recommend a transition to TLS in all cases. To improve reproducibility and comparability in future studies, we recommend a standardised Gregory method: weighting the global mean by cell area, using the top of the atmosphere (as opposed to the top of model) N-variable, and calculating anomalies by first applying a rolling average to the preindustrial control timeseries then subtracting from the raw CO 2 quadrupling experiment. This approach accounts for model drift while reducing noise in the data to best meet the pre-conditions of the linear regression. While CMIP6 results of the multi-model mean ECS appear insensitive to these processing choices, similar assumptions may not hold for CMIP7, underscoring the need for standardised data preparation in future climate sensitivity assessments.

Geosciences↗

A Monte Carlo Analysis of Weight Data from UF 6 Cylinder Feed and Withdrawal Stations

As the number of nuclear facilities handling uranium hexafluoride (UF 6 ) cylinders (e.g., UF 6 production, enrichment, and fuel fabrication) increase in number and throughput, more automated safeguards measures will likely be needed to enable the International Atomic Energy Agency (IAEA) to achieve its safeguards objectives in a fiscally constrained environment. Monitoring the process data from the load cells built into the cylinder feed and withdrawal (F/W) stations (i.e., cylinder weight data) can significantly increase the IAEA’s ability to efficiently achieve the fundamental safeguards task of confirming operations as declared (i.e., no undeclared activities). Researchers at the Oak Ridge National Laboratory, Los Alamos National Laboratory, the Joint Research Center (in Ispra, Italy), and University of Glasgow are investigating how this weight data can be used for IAEA safeguards purposes while fully protecting the operator’s proprietary and sensitive information related to operations. A key question that must be resolved is, what is the necessary frequency of recording data from the process F/W stations to achieve safeguards objectives? This paper summarizes Monte Carlo simulations of typical feed, product, and tails withdrawal cycles and evaluates longer sampling frequencies to determine the expected errors caused by low-frequency sampling and its impact on material balance calculations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

An Intelligent Adaptable Monitoring Package. Final Report

The “Intelligent Adaptable Monitoring Package” project was a four-year effort that demonstrated the feasibility of integrated sensing packages at tidal and wave energy sites. Such integration is generally required by the breadth of sensors required to understand environmental effects at marine energy sites and the operational difficulty of deploying, maintaining, and recovering such sensors. Over the course of the project, the Adaptable Monitoring Package (AMP) was deployed in multiple settings, each corresponding to a project budget period: - Budget Period 1: Demonstration of cabled deployment at Pacific Northwest National Laboratory’s Marine Science Laboratory. The deployment highlighted AMP hardware endurance over a 4-month deployment in a tidally-dominated environment and laid the groundwork for machine learning algorithms to detect and classify targets present in active sonar data. - Budget Period 2: Demonstration of an autonomous deployment at PacWave South off the coast of Newport, Oregon. The deployment highlighted the stability of AMP hardware and software, with the autonomous package collecting data on a duty cycle over a 1.5-month deployment. - Budget Period 3: Demonstration of an autonomous deployment powered by a wave energy converter at the U.S. Navy’s Wave Energy Test Site. The deployment highlighted the potential of wave energy to power ocean observatories and led to the development of machine learning algorithms to detect and classify targets in optical camera data. In aggregate, this project’s greatest success was demonstrating the AMP’s flexibility in a range of deployment scenarios. Each budget period represented a “first of a kind” demonstration of integrated instrumentation – cabled AMP, autonomous AMP, wave-energy powered AMP – and each deployment helped to identify and set goals for the next. Further, despite the exploratory nature of these deployments, each one achieved high system up-time and proved that flexible integration of multiple sensors in a single package represents a viable strategy for marine energy environmental monitoring. The key lessons learned from the project are: - Without continuous power, either from a shore cable or in situ source, many of the benefits of integration are lost (If continuous power is not available, the ability to detect rare events is lost, as is the ability to minimize the risk of behavioral changes through adaptive sensing. However, even on a duty cycle, there is still value in being able to acquire synchronous data from multiple sensors.); and - Observations from a moving platform present substantially greater data processing challenges than those from stationary platforms. Finally, these deployments also demonstrate an important truth: successful integration alone does not guarantee that relevant data are collected. To grow the knowledge base about environmental interactions with marine energy converters, integrated systems, like the AMP, need to include the right sensor mix and connect the data pipelines to effective processing algorithms. These deployments establish a strong foundation for future collaborations with the environmental research community: not only to understand the environmental effects of marine energy, but also to improve our general ability to study life in the sea.

16 TIDAL AND WAVE POWER↗

Using Xrootd to federate regional storage

While the LHC data movement systems have demonstrated the ability to move data at the necessary throughput, we have identified two weaknesses: the latency for physicists to access data and the complexity of the tools involved. To address these, both ATLAS and CMS have begun to federate regional storage systems using Xrootd. Xrootd, referring to a protocol and implementation, allows us to provide data access to all disk-resident data from a single virtual endpoint. This redirector discovers the actual location of the data and redirects the client to the appropriate site. The approach is particularly advantageous since typically the redirection requires much less than 500 milliseconds and the Xrootd client is conveniently built into LHC physicists analysis tools. Currently, there are three regional storage federations - a US ATLAS region, a European CMS region, and a US CMS region. The US ATLAS and US CMS regions include their respective Tier 1, Tier 2 and some Tier 3 facilities, a large percentage of experimental data is available via the federation. Additionally, US ATLAS has begun studying low-latency regional federations of close-by sites. From the base idea of federating storage behind an endpoint, the implementations and use cases diverge. The CMS software framework is capable of efficiently processing data over high-latency links, so using the remote site directly is comparable to accessing local data. The ATLAS processing model allows a broad spectrum of user applications with varying degrees of performance with regard to latency, a particular focus has been optimizing n-tuple analysis. Both VOs use GSI security. ATLAS has developed a mapping of VOMS roles to specific file system authorizations, while CMS has developed callouts to the site's mapping service. Each federation presents a global namespace to users. For ATLAS, the global-to-local mapping is based on a heuristic-based lookup from the site's local file catalog, while CMS does the mapping based on translations given in a configuration file. We will also cover the latest usage statistics and interesting use cases that have developed over the previous 18 months.

Bauerdick, L.↗

Small-wedge synchrotron and serial XFEL datasets for Cysteinyl leukotriene GPCRs

Structural studies of challenging targets such as G protein-coupled receptors (GPCRs) have accelerated during the last several years due to the development of new approaches, including small-wedge and serial crystallography. Here, we describe the deposition of seven datasets consisting of X-ray diffraction images acquired from lipidic cubic phase (LCP) grown microcrystals of two human GPCRs, Cysteinyl leukotriene receptors 1 and 2 (CysLT 1 R and CysLT 2 R), in complex with various antagonists. Five datasets were collected using small-wedge synchrotron crystallography (SWSX) at the European Synchrotron Radiation Facility with multiple crystals under cryo-conditions. Two datasets were collected using X-ray free electron laser (XFEL) serial femtosecond crystallography (SFX) at the Linac Coherent Light Source, with microcrystals delivered at room temperature into the beam within LCP matrix by a viscous media microextrusion injector. All seven datasets have been deposited in the open-access databases Zenodo and CXIDB. Here, we describe sample preparation and annotate crystallization conditions for each partial and full datasets. We also document full processing pipelines and provide wrapper scripts for SWSX and SFX data processing. A Correction to this paper has been published: https://doi.org/10.1038/s41597-020-00759-w

97 MATHEMATICS AND COMPUTING↗

Development of an Ontology-Directed Signal Processing Toolbox

This project was focused on the development of tools for the automatic configuration of signal processing systems. The goal is to develop tools that will be useful in a variety of Government and commercial areas and useable by people who are not signal processing experts. In order to get the most benefit from signal processing techniques, deep technical expertise is often required in order to select appropriate algorithms, combine them into a processing chain, and tune algorithm parameters for best performance on a specific problem. Therefore a significant benefit would result from the assembly of a toolbox of processing algorithms that has been selected for their effectiveness in a group of related problem areas, along with the means to allow people who are not signal processing experts to reliably select, combine, and tune these algorithms to solve specific problems. Defining a vocabulary for problem domain experts that is sufficiently expressive to drive the configuration of signal processing functions will allow the expertise of signal processing experts to be captured in rules for automated configuration. In order to test the feasibility of this approach, we addressed a lightning classification problem, which was proposed by DOE as a surrogate for problems encountered in nuclear nonproliferation data processing. We coded a toolbox of low-level signal processing algorithms for extracting features of RF waveforms, and demonstrated a prototype tool for screening data. We showed examples of using the tool for expediting the generation of ground-truth metadata, for training a signal recognizer, and for searching for signals with particular characteristics. The public benefits of this approach, if successful, will accrue to Government and commercial activities that face the same general problem - the development of sensor systems for complex environments. It will enable problem domain experts (e.g. analysts) to construct signal and image processing chains without the aid of signal processing experts. Thus enabled, problem domain experts will be able to work more quickly and produce better quality work.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Preliminary analysis of the International Data Centre pipeline.

The International Data Centre of the Comprehensive Nuclear-Test-Ban Treaty Organization relies on automatic data processing as the first step in identifying seismic events from seismic waveform data. However, more than half of the automatically identified seismic events are eliminated by IDC analysts. Here, an IDC dataset is analyzed to determine if the number of automatically generated false positives could be reduced. Data that could be used to distinguish false positives from analyst-accepted seismic events includes the number of stations, the number of phases, the signal-to-noise ratio, and the pick error. An empirical method is devised to determine whether an automatically identified seismic event is acceptable, and the method is found to identify a significant number of the false positives in IDC data. This work could help reduce seismic analyst workload and could help improve the calibration of seismic monitoring stations. This work could also be extended to address identification of seismic events missed by automatic processing.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

UAS remote sensing (DJI Phantom 4 RTK platform): RGB orthomosaic, digital surface and canopy height models, plant functional type map, Seward Peninsula, Alaska, 2019

Airborne remote sensing data collected using a DJI Phantom 4 RTK unoccupied aerial system (UAS) ? operated by the University of Maine Wheatland Geospatial Lab (https://wheatlandlab.org/). This package includes data from 17 flights flown over the NGEE-Arctic Council, Kougarok, Kougarok Mile80, Teller, and Teller Mile32 sites in July 2019. The Phantom 4 RTK is a drone platform that collects very high spatial resolution optical red/green/blue (RGB) imagery. Derived image products include point cloud, ortho-mosaiced RGB, a digital surface model (DSM) using the structure from motion (SfM) technique, and a canopy height model (CHM). Unprocessed and processed data products (1,000+ files) are included in this package (processing levels 0-3). Data and metadata are provided as text (*.txt, *.json, *hdr,), ENVI image file (*.dat), point cloud (*.laz) and image (*.jpg, *.tif, *png) formats. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Processing communications events in parallel active messaging interface by awakening thread from wait state

Processing data communications events in a parallel active messaging interface (`PAMI`) of a parallel computer that includes compute nodes that execute a parallel application, with the PAMI including data communications endpoints, and the endpoints are coupled for data communications through the PAMI and through other data communications resources, including determining by an advance function that there are no actionable data communications events pending for its context, placing by the advance function its thread of execution into a wait state, waiting for a subsequent data communications event for the context; responsive to occurrence of a subsequent data communications event for the context, awakening by the thread from the wait state; and processing by the advance function the subsequent data communications event now pending for the context.

97 MATHEMATICS AND COMPUTING↗