Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Big data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Advances in Machine and Deep Learning for Modeling and Real-time Detection of Multi-Messenger Sources

We live in momentous times. The science community is empowered with an arsenal of cosmic messengers to study the universe in unprecedented detail. Gravitational waves, electromagnetic waves, neutrinos, and cosmic rays cover a wide range of wavelengths and timescales. Combining and processing these datasets that vary in volume, speed, and dimensionality requires new modes of instrument coordination, funding, and international collaboration with a specialized human and technological infrastructure. In tandem with the advent of large-scale scientific facilities, the last decade has experienced an unprecedented transformation in computing and signal-processing algorithms. The combination of graphics processing units, deep learning, and the availability of open source, high-quality datasets has powered the rise of artificial intelligence. This digital revolution now powers a multibillion dollar industry, with far-reaching implications in technology and society. In this chapter, we describe pioneering efforts to adapt artificial intelligence algorithms to address computational grand challenges in multi-messenger astrophysics. We review the rapid evolution of these disruptive algorithms, from the first class of algorithms introduced in early 2017 to the sophisticated algorithms that now incorporate domain expertise in their architectural design and optimization schemes. We discuss the importance of scientific visualization and extreme-scale computing in reducing time-to-insight and obtaining new knowledge from the interplay between models and data.

Artificial Intelligence↗
DOE OSTI2024

7th World Congress on Integrated Computational Materials Engineering (ICME 2023) (Final Technical Report)

Integrated Computational Materials Engineering (ICME) has received international attention due to its potential to shorten product development time, while lowering cost and improving design and manufacturing outcomes. ICME is an approach to designing materials solutions for specific applications that use computer modeling programs to predict the behavior of materials and integrate this information into the overall materials, processing, and manufacturing design cycle. The 7th World Congress on Integrated Computational Materials Engineering (ICME 2023) was held in Orlando, Florida from May 21–25, 2023 with the goal to convene stakeholders from across all areas of modeling and simulation, experimental specialization, and design, as well as from across academia, government, and industry, to address ICME tools and techniques and their integration, as well as to examine their application in engineering. This atmosphere facilitated rich interactions between the experimentalists, modelers, and computational and design, from academia, government, and industry, to discuss ICME tools and techniques and their application in engineering.

36 MATERIALS SCIENCE↗
DOE OSTI2013

OBSERVATION OF MAGNETIC RECONNECTION DRIVEN BY GRANULAR SCALE ADVECTION

We report the first evidence of magnetic reconnection driven by advection in a rapidly developing large granule using high spatial resolution observations of a small surge event (base size {approx} 4'' Multiplication-Sign 4'') with the 1.6 m aperture New Solar Telescope at the Big Bear Solar Observatory. The observations were carried out in narrowband (0.5 A) He I 10830 A and broadband (10 A) TiO 7057 A. Since He I 10830 A triplet has a very high excitation level and is optically thin, its filtergrams enable us to investigate the surge from the photosphere through the chromosphere into the lower corona. Simultaneous space data from the Atmospheric Imaging Assembly and Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory were used in the analysis. It is shown that the surge is spatio-temporally associated with magnetic flux emergence in the rapidly developing large granule. During the development of the granule, its advecting flow ({approx}2 km s{sup -1}) squeezed the magnetic flux into an intergranular lane area, where a magnetic flux concentration was formed and the neighboring flux with opposite magnetic polarity was canceled. During the cancellation, the surge was produced as absorption in He I 10830 A filtergrams while simultaneous EUV brightening occurred at its base. The observations clearly indicate evidence of a finest-scale reconnection process driven by the granule's motion.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Window Attachments: The Next Big Energy-Savings Measure?

Windows in residential buildings are estimated to generate 29% of all HVAC heating and cooling load, approximately 2.67 quadrillion Btu (Apte and Arasteh 2006). Although more than a third of residential windows are single pane, very few utility programs offer incentives for installation of energy-efficient windows (Bickel et al. 2013). The reason is simple: unless someone is already planning to replace their windows, the payback can often take a decade, if not longer. Preliminary data indicates that some energy-efficient window attachments, such as low- emissivity (low-e) storm windows and interior panels, cellular shades, interior and exterior roller shades and shutters, window films, solar screens, window quilts, and awnings, can deliver cost-effective energy savings. Additionally, the means to reliably identify and compare energy-efficient window attachment products will soon be available through the Attachment Energy Rating Council (AERC), a public interest energy performance rating and certification organization co-funded by the U.S. Department of Energy (DOE) and the window attachment industry. This paper presents, for the first time, national and regional energy savings estimates for cellular shades and low-emissivity storm windows and panels using existing data on household energy savings potential. This paper is an early step in the process of determining the cost-effectiveness of window attachments for utility programs. The next step is for utilities to develop pilot programs to conduct field tests of these products, which will generate the energy savings data to determine if full-scale programs are cost-effective.

Bickel, Stephen↗
DOE OSTI2019

Analysis on Part Distortion and Residual Stress in Big Area Additive Manufacturing with Carbon Fiber-Reinforced Thermoplastic using Dehomogenization Technique

Part distortion and bottom surface warpage often occur in extrusion-based additive manufacturing. The undesirable deformations during the printing process are due to residual stresses caused by material shrinkage. Analysis on residual stress and deformation requires accurate thermo-mechanical material properties. Polymer composites reinforced with short fibers have intrinsic inhomogeneities with nonuniform fiber orientation. Therefore, homogenized macro properties may not accurately represent the progressive damage behavior or distortion. In this study, fiber orientations in Acrylonitrile Butadiene Styrene (ABS) reinforced with 20%wt carbon fiber are calculated using a de-homogenization technique. Thermal expansion coefficients in multiple directions are obtained from Thermo-Mechanical Analysis (TMA) tests. Temperature dependent stiffness is measured, and temperature dependent strength is estimated. Thermal conductivities in multiple directions and thermal capacity are measured. The calibrated thermo-mechanical properties of the composite with de-homogenized technique are used to analyze the residual stress and distortion of a 4 ft-wide wall printed in the Big Area Additive Manufacturing (BAAM) system. For experimental measurement on the wall printing, Infra-Red (IR) camera captures the temperature field, and Linear Variable Differential Transformer (LVDT) is installed to measure the warpage at the bottom surface. The experimental data are compared to the numerical analysis results. The temperature profile and the distortion profile from experiment are close to the simulation results. Damages due to residual stress and distortion are analyzed.

Kim, Pum↗
DOE OSTI2025

OLCF’s Advanced Computing Ecosystem (ACE): FY25 Update for Ongoing Efforts

The advent of widespread use of artificial intelligence (AI) and machine learning (ML) models in science, coupled with fast data production rates of scientific instruments strain the traditional batch-oriented high-performance computing (HPC) environment. As scientific exploration continues to require more data and faster processing and analysis, new emerging technologies and capabilities to enable cross-facility and time-sensitive workflows are required for seamless integration of HPC and experimental facilities. The Advanced Computing Ecosystem (ACE) is a strategic initiative within the Oak Ridge Leadership Computing Facility (OLCF) established in 2024 to support the development of cutting-edge technologies to advance computational research and infrastructure at OLCF and across the Department of Energy (DOE). Several DOE initiatives are spearheading the evolution of the scientific landscape by blurring facility boundaries and connecting the user facilities to advance scientific capabilities and ensure energy dominance. The DOE Integrated Research Infrastructure (IRI) program is one example that is laying a foundation to support complex cross-facility workflows. The IRI program aims to integrate diverse computational resources, data infrastructures, and scientific instruments to facilitate collaboration and accelerate scientific discovery. The Interconnected Science Ecosystem (INTERSECT) initiative at Oak Ridge National Laboratory (ORNL) is another example that aims to revolutionize scientific research through AI-driven, interconnected autonomous laboratories and research facilities. Finally, the American Science Cloud (AmSC), recently announced in the “One Big Beautiful Bill”, aims to leverage prior infrastructure efforts of the IRI and automation and AI efforts of INTERSECT (and others) to build a federated, AI-augmented AmSC platform to unify the DOE’s computing, experimental, and data resources to catalyze scientific innovation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Survey-wide asteroid discovery with a high-performance computing enabled non-linear digital tracking framework

Modern astronomical surveys detect asteroids by linking together their appearances across multiple images taken over time. This approach faces limitations in detecting faint asteroids and handling the computational complexity of trajectory linking. Here, we present a novel method that adapts “digital tracking” – traditionally used for short-term linear asteroid motion across images – to work with large-scale synoptic surveys such as the Vera Rubin Observatory Legacy Survey of Space and Time (Rubin/LSST). Our approach combines hundreds of sparse observations of individual asteroids across their non-linear orbital paths to enhance detection sensitivity by several magnitudes. To address the computational challenges of processing massive data sets and dense orbital phase spaces, we developed a specialized high-performance computing architecture. We demonstrate the effectiveness of our method through experiments that take advantage of the extensive computational resources at Lawrence Livermore National Laboratory. This work enables the detection of significantly fainter asteroids in existing and future survey data, potentially increasing the observable asteroid population by orders of magnitude across different orbital families, from near-Earth objects (NEOs) to Kuiper belt objects (KBOs).

Asteroid discovery↗
DOE OSTI2022

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond: Preprint

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same field, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications. To address these challenges, the open source Time-Series Data Pipelines (Tsdat) software was developed by a joint collaboration between Pacific Northwest National Laboratory, the National Renewable Energy Laboratory, and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the Marine Energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat software and the data standards within which the framework operates. A beta version of the framework has been released and is currently being used by several projects in marine energy, wind energy, and building energy systems.

big data↗
DOE OSTI2015

Audits Made Simple

A company just got notified there is a big external audit coming in 3 months. Getting ready for an audit can be challenging, scary, and full of surprises. This Gold Paper describes a typical audit from notification of the intent to audit through disposition of the final report including Best Practices, Opportunities for Improvement (OFI), and issues that must be fixed. Good preparation can improve the chances of success. Ensuring the auditors understand the environment and requirements is paramount to success. It helps the auditors understand that the enterprise really does think that security is important. Understanding and following a structured process ensures a smooth audit process. Ensuring follow-up on OFIs and issues in a structured fashion will also make the next audit easier. It is important to keep in mind that the auditors will use the previous report as a starting point. Now the only worry is the actual audit and subsequent report and how well the company has done. “Virtualization, mobilization, and cloud technology have created new points of entry into business, leaving them vulnerable to covert cyber-attacks,” states a report by Ernst & Young. Executives are struggling with the potential for a data breach, and resolving this business risk is high on their agenda. While most audit committee members are financially solid, they lack knowledge of technology issues, making understanding of this risk difficult. Success with an audit helps management understand the risk management is accepting, highlight gaps in security, and generate ideas on how to improve. These results are an important aspect of managing the business risk.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Large-Scale Compute-Intensive Analysis via a Combined In-situ and Co-scheduling Workflow Approach

Large-scale simulations can produce tens of terabytes of data per analysis cycle, complicating and limiting the efficiency of workflows. Traditionally, outputs are stored on the file system and analyzed in post-processing. With the rapidly increasing size and complexity of simulations, this approach faces an uncertain future. Trending techniques consist of performing the analysis in situ, utilizing the same resources as the simulation, and/or off-loading subsets of the data to a compute-intensive analysis system. We introduce an analysis framework developed for HACC, a cosmological N-body code, that uses both in situ and co-scheduling approaches for handling Petabyte-size outputs. An initial in situ step is used to reduce the amount of data to be analyzed, and to separate out the data-intensive tasks handled off-line. The analysis routines are implemented using the PISTON/VTK-m framework, allowing a single implementation of an algorithm that simultaneously targets a variety of GPU, multi-core, and many-core architectures.

Big data↗
DOE OSTI2021

Learning to Correct Climate Projection Biases

The fidelity of climate projections is often undermined by biases in climate models due to their simplification or misrepresentation of unresolved climate processes. While various bias correction methods have been developed to post-process model outputs to match observations, existing approaches usually focus on limited, low-order statistics, or break either the spatiotemporal consistency of the target variable, or its dependency upon model resolved dynamics. We develop a Regularized Adversarial Domain Adaptation (RADA) methodology to overcome these deficiencies, and enhance efficient identification and correction of climate model biases. Instead of pre-assuming the spatiotemporal characteristics of model biases, we apply discriminative neural networks to distinguish historical climate simulation samples and observation samples. The evidences based on which the discriminative neural networks make distinctions are applied to train the domain adaptation neural networks to bias correct climate simulations. We regularize the domain adaptation neural networks using cycle-consistent statistical and dynamical constraints. An application to daily precipitation projection over the contiguous United States shows that our methodology can correct all the considered moments of daily precipitation at approximately $1^\circ$ resolution, ensures spatiotemporal consistency and inter-field correlations, and can discriminate between different dynamical conditions. Our methodology offers a powerful tool for disentangling model parameterization biases from their interactions with the chaotic evolution of climate dynamics, opening a novel avenue toward big-data enhanced climate predictions.

58 GEOSCIENCES↗
DOE OSTI2024

Testing Protocol Development for the Fracture Toughness of Parts Built with Big Area Additive Manufacturing

The mechanical testing of additively manufactured parts has largely relied on the existing standards developed for traditional manufacturing. While this approach leverages the investment made in current standards development, it inaccurately assumes that the mechanical response of additive manufacturing (AM) parts is identical to that of parts manufactured through traditional processes. When considering thermoplastic, material extrusion AM, the differences in response can be attributed to an AM part’s inherent inhomogeneity caused by porosity, interlayer zones, and surface texture. Additionally, the interlayer bonding of parts printed with large-scale AM is difficult to adequately assess, as much testing is performed such that stress is distributed across many layer interfaces; therefore, the lack of AM-specific standards to assess interlayer bonding is a significant research gap. To quantify interlayer bonding via fracture toughness, double cantilever beam (DCB) testing has been used for some AM materials, and DCB has been generally used for a variety of materials including metal, wood, and laminates. Mode I DCB testing was performed on thermoplastic matrix composites printed with Big Area Additive Manufacturing (BAAM). Of particular interest was the notch shape and deflection speed during testing. The results examine the differences when using two notch types and three deflection speeds. The testing method introduced by the following paper differentiates itself from the ones described in the standards used by modernizing the methodology. This was conducted with the introduction of Digital Image Correlation (DIC) to gather displacement and load data simultaneously without human intervention.

Polymer Science↗
DOE OSTI2015

A SEMI-COHERENT SEARCH FOR WEAK PULSATIONS IN AQUILA X–1

Non-pulsating neutron stars in low mass X-ray binaries largely outnumber those that show pulsations. The lack of detectable pulses represents a big open problem for two important reasons. The first is that the structure of the accretion flow in the region closest to the neutron star is not well understood and it is therefore unclear what is the mechanism that prevents the pulse formation. The second is that the detection of pulsations would immediately reveal the spin of the neutron star. AQUILA X–1 is a special source among low mass X-ray binaries because it has showed the unique property of pulsating for only ∼150 s out of a total observing time of more than 1.5 million seconds. However, the existing upper limits on the pulsed fraction leave open two alternatives. Either AQUILA X–1 has very weak pulses which have been undetected, or it has genuinely pulsed only for a tiny amount of the observed time. Understanding which of the two scenarios is the correct one is fundamental to increase our knowledge about the pulse formation process and understand the chances we have to detect weak pulses in other low-mass X-ray binaries. In this paper we perform a semi-coherent search on the entire X-ray data available for AQUILA X–1. We find no evidence for (new) weak pulsations with the most stringent upper limits being of the order of 0.3% in the 7–25 keV energy band.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

ASCR Workshop on In Situ Data Management: Enabling Scientific Discovery from Diverse Data Sources

In January 2019, the U.S. Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions for in situ data management (ISDM). The workshop defined ISDM as the practices, capabilities, and procedures to control the organization of data and enable the coordination and communication among heterogeneous tasks, executing simultaneously in a high-performance computing system, cooperating toward a common objective. The workshop revealed two primary, interdependent motivations for processing and managing data in situ. The first motivation is that the in situ methodology enables scientific discovery from a broad range of data sources over a wide scale of computing platforms: leadership-class systems, clusters, clouds, workstations, and embedded devices at the edge. The successful development of ISDM capabilities will benefit real-time decision-making, design optimization, and data-driven scientific discovery. The second motivation is the need to decrease data volumes. ISDM can make critical contributions to managing large data volumes from computations and experiments to minimize data movement, save storage space, and boost resource efficiency, often while simultaneously increasing scientific precision. A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery across the Office of Science mission space. Applications wanting to use the in situ capabilities include those where data analysis feeds back to the simulation, decisions are made autonomously, big data or machine learning is among the tasks to be coordinated, and computations need to be completed in real time. The workshop identified six priority research directions that highlight the components and capabilities needed for ISDM to be successful for the wide variety of applications discussed: making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same domain, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications, which rely on large quantities of data from multiple sources. To address these challenges, the open-source Time-Series Data Pipelines (Tsdat) Python framework was developed by Pacific Northwest National Laboratory, with strategic guidance and direction provided by the National Renewable Energy Laboratory and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the marine energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat framework and the data standards within which it operates. A beta version of Tsdat has been released and is being used by several projects in marine energy, wind energy, and building energy systems.

big data↗
DOE OSTI2025

Reinventing wastewater treatment plants: energy neutral treatment and enhanced fertilizer production through a novel resource recovery center

Wastewater treatment plants (WWTPs) are typically energy intensive, mainly due to the secondary treatment processes such as activated sludge (AS) for treatment of organics as well as nutrients like nitrogen. Nitrogen removal presents a big problem for WWTPs. The main form of nitrogen in wastewater is ammonium, and an AS process uses oxygen to convert ammonium into nitrite and nitrate which is then converted to nitrogen through denitrification process. During anaerobic digestion (AD), organic nitrogen gets degraded, resulting in an effluent stream (centrate) with a high nitrogen content, mostly in the form of ammonium. This contributes 15-30% of total nitrogen to the wastewater influent which further increases energy consumption for aeration. The project aims to transform this conventional municipal WWTPs into energy-neutral, resource-recovering facilities by integrating three core technologies: • Cloth Media Filtration (CMF) to replace conventional primary sedimentation (CPS) and increase the diversion of organics from the energy intensive secondary treatment to AD. This results in reduced energy demand for aeration in the secondary process while simultaneously increasing the biogas production in the anaerobic digesters. • Anerobic Digester to increase biogas and ammonia production. • Membrane Evaporation (ME) to recover ammonia from AD centrate and produce marketable fertilizer. The benefits of proposed WWTP process modifications were evaluated using techno economic analysis (TEA) and life cycle assessment (LCA). For CMF portion of the research a statistical analysis was employed to develop data-driven tools that could be used to enhance and optimize its performance in terms of energy savings and effluent quality. The main objective of this project is to reduce the energy demand for secondary treatment at municipal WWTPs by at least 50%, increase anaerobic digester (AD) biogas and ammonia production by 100% and 120%, respectively, and recover 90% of ammonia from the AD. Integrated CMF, AD, and ME was shown to work synergistically toward achieving these decarbonization targets through energy-positive treatment and fertilizer recovery techniques.

42 ENGINEERING↗
DOE OSTI2024

Development of a multi-layer canopy model for E3SM Land Model with support for heterogeneous computing

The vertical structure of vegetation canopies creates micro-climates. However, the land components of most Earth System Models, including the Energy Exascale Earth System Model (E3SM), typically neglect vertical canopy structure by using a single layer big-leaf representation to simulate water, CO 2 , and energy exchanges between the land and the atmosphere. In this study, we developed a Multi-Layer Canopy Model for the E3SM Land Model to resolve the micro-climate created by vegetation canopies. The model developed in this study re-implements the CLM-ml_v1 to support heterogeneous computing architectures consisting of CPUs and GPUs and includes three additional optimization-based stomatal conductance models. The use of Portable, Extensible Toolkit for Scientific Computation provides a speedup of 25–50 times on a GPU relative to a CPU. The numerical implementation of the model was verified against CLM-ml_v1 for a month-long simulation using data from the Ameriflux US-University of Michigan Biological Station site. Model structural uncertainty was explored by performing control simulations for five stomatal conductance models that exclude and include the control of plant hydrodynamics (PHD) on photosynthesis. The bias in simulated sensible and latent heat fluxes was lower when PHD was accounted for in the model. Additionally, six idealized simulations were performed to study the impact of three environmental variables (i.e. air temperature, atmospheric CO 2 , and soil moisture) on canopy processes (i.e. net CO 2 assimilation, leaf temperature, and leaf water potential). Increasing air temperature reduced net CO 2 assimilation and increased air temperature. Net CO 2 assimilation increased at higher atmospheric CO 2 , while decreasing soil moisture resulted in lower leaf water potential.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Specification of the near-Earth space environment with SHIELDS

Here, predicting variations in the near-Earth space environment that can lead to spacecraft damage and failure is one example of “space weather” and a big space physics challenge. A project recently funded through the Los Alamos National Laboratory (LANL) Directed Research and Development (LDRD) program aims at developing a new capability to understand, model, and predict Space Hazards Induced near Earth by Large Dynamic Storms, the SHIELDS framework. The project goals are to understand the dynamics of the surface charging environment (SCE), the hot (keV) electrons representing the source and seed populations for the radiation belts, on both macro- and micro-scale. Important physics questions related to particle injection and acceleration associated with magnetospheric storms and substorms, as well as plasma waves, are investigated. These challenging problems are addressed using a team of world-class experts in the fields of space science and computational plasma physics, and state-of-the-art models and computational facilities. A full two-way coupling of physics-based models across multiple scales, including a global MHD (BATS-R-US) embedding a particle-in-cell (iPIC3D) and an inner magnetosphere (RAM-SCB) codes, is achieved. New data assimilation techniques employing in situ satellite data are developed; these provide an order of magnitude improvement in the accuracy in the simulation of the SCE. SHIELDS also includes a post-processing tool designed to calculate the surface charging for specific spacecraft geometry using the Curvilinear Particle-In-Cell (CPIC) code that can be used for reanalysis of satellite failures or for satellite design.

54 ENVIRONMENTAL SCIENCES↗