Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,711 records · Page 95

Component based dataflow processing framework

In this paper we present SOA based CLAs12 event Reconstruction and Analyses (CLARA) framework used to develop Earth Science multi-sensor data fusion, processing, and analytics applications (NAIADS: NASA JLAB collaboration). CLARA design focus is on two main traits: a) real-time data stream processing, and b) service oriented architecture (SOA) in a flow based programming (FBP) paradigm. Data driven and data centric architecture of CLARA presents an environment for developing agile, elastic, multilingual data processing applications. The CLARA framework presents solutions, capable of processing large volumes of data interactively and substantially faster than batch systems.

Gyurjyan, V.↗

Pixel-Based Correction for Charge Transfer Inefficiency in the Hubble Space Telescope Advanced Camera for Surveys

Charge Transfer Inefficiency (CTI) due to radiation damage above the Earth's atmosphere creates spurious trailing in Hubble Space Telescope (HST) images. Radiation damage also creates unrelated warm pixels - but these happen to be perfect for measuring CTI. We model CTI in the Advanced Camera for Surveys (ACS)/Wide Field Channel (WFC) and construct a physically motivated correction scheme. This operates on raw data, rather than secondary science products, by returning individual electrons to pixels from which they were unintentionally dragged during readout. We apply our correction to images from the HST COSMOS survey, successfully reducing the CTI trails by a factor of ~30 everywhere in the CCD and at all flux levels. We quantify changes in galaxy photometry, astrometry and shape. The remarkable 97% level of correction is more than sufficient to enable a (forthcoming) reanalysis of downstream science products, and the collection of larger surveys.

Massey, Richard↗

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗

UOWDetection

The code provides machine learning, data collection, and computer science tools used to detect undocumented orphaned wells in United States from aerial imagery.

Kim, Anastasiia↗

Variability in Wind Energy Generation across the Contiguous United States

ERA5 provides high-resolution, high-quality hourly wind speeds at 100 m and is a unique resource for quantifying temporal variability in likely wind-derived power production across the United States. Gross capacity factors (CF) in seven independent system operators (ISOs) are estimated using the location and rated power of each wind turbine, a simplified power curve, and ERA5 output from 1979 to 2018. Excluding the California ISO, the marginal probability of a calm (zero power production) is less than 0.1 in any ERA5 grid cell. When a calm occurs, the mean co-occurrence across wind-turbine-containing grid cells ranges from 0.38 to 0.39 for ISOs in the Midwest and central plains [Midcontinent (or Midwest) ISO (MISO), Southwest Power Pool (SPP), and the Electric Reliability Council of Texas (ERCOT) region], increasing to 0.54–0.58 for ISOs in the eastern United States [Pennsylvania–New Jersey–Maryland interconnection (PJM), New York ISO (NYISO), and New England ISO (NEISO)]. Periods with low gross CF have a median duration of ≤6 h, except in California, and are most likely during summer. Additionally, gross CF exhibit highest variance at periods of 1 day in ERCOT and SPP; on synoptic scales in MISO, NEISO, and NYISO; and on interannual time scales in PJM. This implies differences in optimal strategies for ensuring resilience of supply. Theoretical scenarios show adding wind energy capacity near existing wind farms is advantageous even in areas with high existing installed capacity (IC), while expanding into areas with lower IC is more beneficial to reducing ramps and the probability of gross CF falling below 20%. These results emphasize the benefits of large balancing areas and aggregation in reducing wind power variability and the likelihood of wind droughts.

17 WIND ENERGY↗

Computational analysis of pathogen-borne metallo β-lactamases reveals discriminating structural features between B1 types

Background: Genes conferring antibiotic resistance to groups of bacterial pathogens are cause for considerable concern, as many once-reliable antibiotics continue to see a reduction in efficacy. The recent discovery of the metallo b-lactamase blaNDM-1 gene, which appears to grant antibiotic resistance to a variety of Enterobacteriaceae via a mobile plasmid, is one example of this distressing trend. The following work describes a computational analysis of pathogen-borne MBLs that focuses on the structural aspects of characterized proteins. Results: Using both sequence and structural analyses, we examine residues and structural features specific to various pathogen-borne MBL types. This analysis identifies a linker region within MBL-like folds that may act as a discriminating structural feature between these proteins, and specifically resistance-associated acquirable MBLs. Recently released crystal structures of the newly emerged NDM-1 protein were aligned against related MBL structures using a variety of global and local structural alignment methods, and the overall fold conformation is examined for structural conservation. Conservation appears to be present in most areas of the protein, yet is strikingly absent within a linker region, making NDM-1 unique with respect to a linker-based classification scheme. Variability analysis of the NDM-1 crystal structure highlights unique residues in key regions as well as identifying several characteristics shared with other transferable MBLs. Conclusions: A discriminating linker region identified in MBL proteins is highlighted and examined in the context of NDM-1 and primarily three other MBL types: IMP-1, VIM-2 and ccrA. The presence of an unusual linker region variant and uncommon amino acid composition at specific structurally important sites may help to explain the unusually broad kinetic profile of NDM-1 and may aid in directing research attention to areas of this protein, and possibly other MBLs, that may be targeted for inactivation or attenuation of enzymatic activity.

59 BASIC BIOLOGICAL SCIENCES↗

BAAD: a Biomass And Allometry Database for woody plants

Quantifying the amount of mass or energy invested in plant tissues is of fundamental interest across a range of disciplines, including ecology, forestry, ecosystem science, and climate change science (Niklas, 1994; Chave et al. 2005; Falster et al. 2011). The allocation of net primary production into different plant components is an important process affecting the lifetime of carbon in ecosystems, and resource use and productivity by plants (Cannell & Dewar, 1994; Litton et al. 2007; Poorter et al. 2012). While many studies in have destructively harvested woody plants in the name of science, most of these data have only been made available in the form of summary tables or figures included in publications. Until now, the raw data has resided piecemeal on the hard drives of individual scientists spread around the world. Several studies have gathered together the fitted (allometric) equations for separate datasets (Ter-Mikaelian & Korzukhin, 1997; Jenkins et al. 2003; Zianis et al. 2005; Henry et al. 2013), but none have previously attempted to organize and share the raw individual plant data underpinning these equations on a large scale. Gathered together, such data would represent an important resource for the community, meeting a widely recognised need for rich, open data resources to solve ecological problems (Costello et al. 2013; Fady et al. 2014; Harfoot & Roberts, 2014; Costello et al. 2013). We (D.S. Falster and R.A. Duursma, with the help of D.R. Barneche, R.G. FitzJohn and A. Vårhammar) set out to create such a resource, by asking authors directly whether they would be willing to make their raw data files freely available. The response was overwhelming: nearly everyone we contacted was interested to contribute their raw data. Moreover, we were invited to incorporate another compilation led by M. Ishihara and focussing on Japanese literature. As a result, we present BAAD: a Biomass And Allometry Database for woody plants, comprising data collected in 174 different published and unpublished studies.

Falster, Daniel↗

Ca L 2,3 -edge near edge X-ray absorption fine structure of tricalcium aluminate, gypsum, and calcium (sulfo)aluminate hydrates

Tricalcium aluminate (cement clinker phase), gypsum, katoite, ettringite, and calcium monosulfoaluminate hydrate (abbreviated as kuzelite) are the major minerals in the hydration reaction of tricalcium aluminate in the presence of gypsum and have critical impacts on the kinetics and thermodynamics of early-age cement hydration mechanisms. In this paper, spectroscopic analysis of these minerals is conducted using scanning transmission X-ray microscopy (STXM). Their Ca L 2,3 -edge near edge X-ray absorption fine structure (NEXAFS) spectra are measured and correlated to the known Ca coordination environments. The results indicate that these minerals have unique Ca environments that can be differentiated from one another based on the intensities and positions of the absorption peaks at 346.5–348.5 and 350.5–351.5 eV. It is concluded that Ca in tricalcium aluminate (cubic and orthorhombic polymorphs) and katoite is in cubic-like coordination with negative 10Dq, whereas Ca is in an octahedral-like coordination with positive 10Dq in ettringite, gypsum, and kuzelite. For tricalcium aluminate, the Ca atoms in both polymorphs are in similar chemical environments with slightly more distortion in the orthorhombic polymorph. As a common issue in STXM experiments, absorption saturation of NEXAFS spectra is also investigated. It is demonstrated that the optical density difference between pre- and post-edge absorption levels provides a reliable indication of the sample thickness in the systems studied. Finally, the present work provides a reference for the STXM study of the calcium (sulfo)aluminate reactions in cement hydration and natural aqueous environments, and in the former case, provides a more complete understanding of a system that may serve as a low-C alternative to Portland cement.

36 MATERIALS SCIENCE↗

CAGE100: Real-Time Multi-Port Packet Capture System for 100 Gigabit Ethernet Traffic

Future large scale sciences are anticipated to use massive amount of data in their experiments. DOE's ESnet (Energy Science Network) is developing a 100 Gbps backbone based on this state-of-the-art 100 Gigabit Ethernet standard. ESnet will serve thousands of DOE and non-DOE scientists with its high bandwidth backbone, and connect several national laboratories. Current Ethernet test and debug solutions, such as network traffic capturer/analyzer tools, support up to 10 Gbps speed, and the very few capable of handling 100 Gbps are extremely costly. Such tools are essential in the development of high speed devices and routers, and ultimately the success of 100 Gigabit Ethernet.

60 APPLIED LIFE SCIENCES↗

Parallel Tensor Compression for Large-Scale Scientific Data.

As parallel computing trends towards the exascale, scientific data produced by high-fidelity simulations are growing increasingly massive. For instance, a simulation on a three-dimensional spatial grid with 512 points per dimension that tracks 64 variables per grid point for 128 time steps yields 8 TB of data. By viewing the data as a dense five way tensor, we can compute a Tucker decomposition to find inherent low-dimensional multilinear structure, achieving compression ratios of up to 10000 on real-world data sets with negligible loss in accuracy. So that we can operate on such massive data, we present the first-ever distributed memory parallel implementation for the Tucker decomposition, whose key computations correspond to parallel linear algebra operations, albeit with nonstandard data layouts. Our approach specifies a data distribution for tensors that avoids any tensor data redistribution, either locally or in parallel. We provide accompanying analysis of the computation and communication costs of the algorithms. To demonstrate the compression and accuracy of the method, we apply our approach to real-world data sets from combustion science simulations. We also provide detailed performance results, including parallel performance in both weak and strong scaling experiments.

59 BASIC BIOLOGICAL SCIENCES↗

Improving Data Mobility & Management for International Cosmology

In February 2015 the third workshop in the CrossConnects series, with a focus on Improving Data Mobility & Management for International Cosmology, was held at Lawrence Berkeley National Laboratory. Scientists from fields including astrophysics, cosmology, and astronomy collaborated with experts in computing and networking to outline strategic opportunities for enhancing scientific productivity and effectively managing the ever-increasing scale of scientific data. While each field has unique details which depend on the instruments employed, the type and scale of the data, and the structure of scientific collaborations, several important themes emerged from the workshop discussions. Findings, as well as a set of recommendations, are contained in their respective sections in this report.

79 ASTRONOMY AND ASTROPHYSICS↗

Atmospheric Radiation Measurement (ARM) Management Plan

Mission and Vision Statements for the U.S. Department of Energy (DOE)’s Atmospheric Radiation Measurement (ARM) Climate Research Facility Mission The ARM Climate Research Facility, a DOE scientific user facility, provides the climate research community with strategically located in situ and remote-sensing observatories designed to improve the understanding and representation, in climate and earth system models, of clouds and aerosols as well as their interactions and coupling with the Earth’s surface. Vision To provide a detailed and accurate description of the Earth atmosphere in diverse climate regimes to resolve the uncertainties in climate and Earth system models toward the development of sustainable solutions for the nation's energy and environmental challenges.

54 ENVIRONMENTAL SCIENCES↗

Fission in a Plasma

A three-year theory project was undertaken to study the fission process in extreme astrophysical environments, such as the crust of neutron stars. In the first part of the project, the effect of electron screening on the fission process was explored using a microscopic approach. For the first time, these calculations were carried out to the breaking point of the nucleus. In the second part of the project, the population of the fissioning nucleus was calculated within the same microscopic framework. These types of calculations are extremely computer-intensive and have seldom been applied to heavy deformed nuclei, such as fissioning actinides. The results, tools and methodologies produced in this work will be of interest to both the basic-science and nuclear-data communities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Measuring and Modeling the Earth's Gravity - Introduction to Ground-Based Gravity Surveys and Analysis of Local Gravity Data

We can measure changes in gravity from place to place on the earth. These measurements require careful recording of location, elevation and time for each reading. These readings must be adjusted for known effects (such as elevation, latitude, tides) that can bias our data and mask the signal of interest. After making corrections to our data, we can remove regional trends to obtain local Bouguer anomalies. The Bouguer anomalies arise from variations in the subsurface density structure. We can build models to explain our observations, but these models must be consistent with what is known about the local geology. Combining gravity models with other information – geologic, seismic, electromagnetic, will improve confidence in the results.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Raman Lidar Water Vapor Mixing Ratio and Temperature Value-Added Products

This report describes the methods used to estimate profiles of water vapor mixing ratio (WVMR) and temperature from the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) user facility Raman lidars (RL). This includes a description of the equations and the calibration methods that are used, as well as the output datastreams and variable names. We begin with an overview of the ARM RL systems, with a focus on the detection channels used for the WVMR and temperature measurements.

54 ENVIRONMENTAL SCIENCES↗

HOOVER

HOOVER is a distributed framework for solving streaming/dynamic graph problems where the structure of a graph changes over time based on the state of its vertices.

97 MATHEMATICS AND COMPUTING↗