Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Big data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

The research data alliance photon and neutron science interest group

Scientific research data provides unique challenges that are distinct from classic “Big Data” sources. One common element in research data is that the experiment, observations, or simulation were designed, and data were specifically acquired, to shed light on an open scientific question. The data and methods are usually “owned” by the researcher(s) and the data itself might not be viewed to have long-term scientific significance after the results have been published. Often, the data volume was relatively low, with data sometimes easier to reproduce than to catalog and store. Some data and meta-data were not collected in a digital form, or were stored on antiquated or obsolete media. Generally speaking, policies, tools, and management of digital research data have reflected an ad hoc approach that varies domain by domain and research group by research group. This model, which treats research data as disposable, is proving to be a serious limitation as the volume and complexity of research data explodes. Changes are required at every level of scientific research: within the individual groups, and across scientific domains and interdisciplinary collaborations. Enabling researchers to learn about available tools, processes, and procedures should encourage a spirit of cooperation and collaboration, allowing researchers to come together for the common good. In conclusion, these community-oriented efforts provide the potential for targeted projects with high impact.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2024

End-to-End Mission Design & Trajectory Optimization

Need: A need exists for a generalized, robust, user-friendly and accessible end-to-end mission design optimization tool. Solution: Our solution to developing this capability was to interface two JSC tools—Copernicus and Genesis. Each of these tools has a specific area of the mission design process that it excels at. By utilizing them both, we can gain performance benefits not seen by either on their own. - Copernicus is a trajectory design and optimization software used for in-space trajectories around multiple bodies. - Genesis is a flight mechanics tool used to model ascent, entry, descent, and landing trajectories around a single planetary body. Year 1 was focused on combining these 2 software packages—allowing Copernicus to incorporate the ascent/descent capabilities of Genesis into the optimization problem—and developing this end-to-end mission design capability. Year 2 we focused on increasing the robustness of this capability by building the initial guess generator (IGG), which produces initial guesses based on simplifying assumptions and the physics of the problem. Year 3 of our project focused on utilizing the end-to-end mission design and optimization capabilities developed in the previous 2 years to analyze specific mission scenarios—scaling up from proof of concept to real analyses—capturing any resulting performance benefits, as well as addressing the Big Data challenges we’re faced with—namely, how we’re going to manage and interpret all the data that’s generated.

Kristin Nichols↗
DOE OSTI2020

Data-Driven Template Discovery Using Graph Convolutional Neural Networks

Modeling adversarial activities is a critical component of developing high-con?dence indicators of efforts to acquire, fabricate, proliferate, and/or deploy weapons of mass terror (WMTs). Current approaches to generating representative patterns of interest (a.k.a templates) from the real-world domains involve a Subject Matter Expert (SME)-guided manual process. The goal of Data-Driven Template Discovery (DDTD) is to use a (potentially small) set of SME generated templates to discover other previously unknown and interesting templates in an attributed graph. A template is an activity pattern describing a set of interactions among a group of nodes in the graph. The motivation behind DDTD is to expand the original set of templates, without having SMEs craft all the templates by hand. DDTD also provides seed templates to SMEs, to help them construct larger, high-?delity, and scenario-oriented templates. In these cases, obtaining a larger set of templates that are related (contain similar signals) to the original set is of great value. In this work, we propose to use Graph Convolutional Neural Networks (GCNs) to discover new templates that are heavily related to the original set. GCNs are a family of Neural Network (NN) architectures especially designed to work directly on graphs. In contrast to the traditional NNs, that require considerable amounts of labeled data, GCNs do not require a big labeled training set because they can directly leverage the graph structure instead. This property makes GCNs the perfect tool for creating activity templates.

Joaristi, Mikel↗
NASA NTRS1992

Infrared photometric behavior and opposition effect of Mars

Although the instrument wasn't designed for this purpose, data from the imaging spectrometer ISM may be used for studying photometric variations of Mars reflectance, that are related to the surface materials and aerosols physical properties. ISM flew aboard the Phobos-2 spacecraft which orbited Mars from January to March, 1989. About 40,000 spectra were acquired in 128 channels ranging from 0.76 to 3.16 micro-m, with a spatial resolution of 25 km and a signal-to-noise ratio ranging up to 1000. Analysis of the results leads to the following conclusions: width variations of the opposition surge can be related to differences in porosity or grain size distribution on the various domains, with little or no effect from suspended dust. As the biggest effects are observed on dark and bright materials, intermediate behaviors on average-bright regions cannot result from a mixing process, but are more likely to come from either cementation processes or modification of the grain size distribution under the influence of wind, which under Martian conditions preferentially removes the biggest particles. Thus, a surface dust consisting in big bright and small dark grains could explain the observations.

Erard, S.↗
DOE OSTI2022

Advances in Machine and Deep Learning for Modeling and Real-time Detection of Multi-Messenger Sources

We live in momentous times. The science community is empowered with an arsenal of cosmic messengers to study the universe in unprecedented detail. Gravitational waves, electromagnetic waves, neutrinos, and cosmic rays cover a wide range of wavelengths and timescales. Combining and processing these datasets that vary in volume, speed, and dimensionality requires new modes of instrument coordination, funding, and international collaboration with a specialized human and technological infrastructure. In tandem with the advent of large-scale scientific facilities, the last decade has experienced an unprecedented transformation in computing and signal-processing algorithms. The combination of graphics processing units, deep learning, and the availability of open source, high-quality datasets has powered the rise of artificial intelligence. This digital revolution now powers a multibillion dollar industry, with far-reaching implications in technology and society. In this chapter, we describe pioneering efforts to adapt artificial intelligence algorithms to address computational grand challenges in multi-messenger astrophysics. We review the rapid evolution of these disruptive algorithms, from the first class of algorithms introduced in early 2017 to the sophisticated algorithms that now incorporate domain expertise in their architectural design and optimization schemes. We discuss the importance of scientific visualization and extreme-scale computing in reducing time-to-insight and obtaining new knowledge from the interplay between models and data.

Artificial Intelligence↗
DOE OSTI2024

7th World Congress on Integrated Computational Materials Engineering (ICME 2023) (Final Technical Report)

Integrated Computational Materials Engineering (ICME) has received international attention due to its potential to shorten product development time, while lowering cost and improving design and manufacturing outcomes. ICME is an approach to designing materials solutions for specific applications that use computer modeling programs to predict the behavior of materials and integrate this information into the overall materials, processing, and manufacturing design cycle. The 7th World Congress on Integrated Computational Materials Engineering (ICME 2023) was held in Orlando, Florida from May 21–25, 2023 with the goal to convene stakeholders from across all areas of modeling and simulation, experimental specialization, and design, as well as from across academia, government, and industry, to address ICME tools and techniques and their integration, as well as to examine their application in engineering. This atmosphere facilitated rich interactions between the experimentalists, modelers, and computational and design, from academia, government, and industry, to discuss ICME tools and techniques and their application in engineering.

36 MATERIALS SCIENCE↗
DOE OSTI2013

OBSERVATION OF MAGNETIC RECONNECTION DRIVEN BY GRANULAR SCALE ADVECTION

We report the first evidence of magnetic reconnection driven by advection in a rapidly developing large granule using high spatial resolution observations of a small surge event (base size {approx} 4'' Multiplication-Sign 4'') with the 1.6 m aperture New Solar Telescope at the Big Bear Solar Observatory. The observations were carried out in narrowband (0.5 A) He I 10830 A and broadband (10 A) TiO 7057 A. Since He I 10830 A triplet has a very high excitation level and is optically thin, its filtergrams enable us to investigate the surge from the photosphere through the chromosphere into the lower corona. Simultaneous space data from the Atmospheric Imaging Assembly and Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory were used in the analysis. It is shown that the surge is spatio-temporally associated with magnetic flux emergence in the rapidly developing large granule. During the development of the granule, its advecting flow ({approx}2 km s{sup -1}) squeezed the magnetic flux into an intergranular lane area, where a magnetic flux concentration was formed and the neighboring flux with opposite magnetic polarity was canceled. During the cancellation, the surge was produced as absorption in He I 10830 A filtergrams while simultaneous EUV brightening occurred at its base. The observations clearly indicate evidence of a finest-scale reconnection process driven by the granule's motion.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Window Attachments: The Next Big Energy-Savings Measure?

Windows in residential buildings are estimated to generate 29% of all HVAC heating and cooling load, approximately 2.67 quadrillion Btu (Apte and Arasteh 2006). Although more than a third of residential windows are single pane, very few utility programs offer incentives for installation of energy-efficient windows (Bickel et al. 2013). The reason is simple: unless someone is already planning to replace their windows, the payback can often take a decade, if not longer. Preliminary data indicates that some energy-efficient window attachments, such as low- emissivity (low-e) storm windows and interior panels, cellular shades, interior and exterior roller shades and shutters, window films, solar screens, window quilts, and awnings, can deliver cost-effective energy savings. Additionally, the means to reliably identify and compare energy-efficient window attachment products will soon be available through the Attachment Energy Rating Council (AERC), a public interest energy performance rating and certification organization co-funded by the U.S. Department of Energy (DOE) and the window attachment industry. This paper presents, for the first time, national and regional energy savings estimates for cellular shades and low-emissivity storm windows and panels using existing data on household energy savings potential. This paper is an early step in the process of determining the cost-effectiveness of window attachments for utility programs. The next step is for utilities to develop pilot programs to conduct field tests of these products, which will generate the energy savings data to determine if full-scale programs are cost-effective.

Bickel, Stephen↗
DOE OSTI2019

Analysis on Part Distortion and Residual Stress in Big Area Additive Manufacturing with Carbon Fiber-Reinforced Thermoplastic using Dehomogenization Technique

Part distortion and bottom surface warpage often occur in extrusion-based additive manufacturing. The undesirable deformations during the printing process are due to residual stresses caused by material shrinkage. Analysis on residual stress and deformation requires accurate thermo-mechanical material properties. Polymer composites reinforced with short fibers have intrinsic inhomogeneities with nonuniform fiber orientation. Therefore, homogenized macro properties may not accurately represent the progressive damage behavior or distortion. In this study, fiber orientations in Acrylonitrile Butadiene Styrene (ABS) reinforced with 20%wt carbon fiber are calculated using a de-homogenization technique. Thermal expansion coefficients in multiple directions are obtained from Thermo-Mechanical Analysis (TMA) tests. Temperature dependent stiffness is measured, and temperature dependent strength is estimated. Thermal conductivities in multiple directions and thermal capacity are measured. The calibrated thermo-mechanical properties of the composite with de-homogenized technique are used to analyze the residual stress and distortion of a 4 ft-wide wall printed in the Big Area Additive Manufacturing (BAAM) system. For experimental measurement on the wall printing, Infra-Red (IR) camera captures the temperature field, and Linear Variable Differential Transformer (LVDT) is installed to measure the warpage at the bottom surface. The experimental data are compared to the numerical analysis results. The temperature profile and the distortion profile from experiment are close to the simulation results. Damages due to residual stress and distortion are analyzed.

Kim, Pum↗
DOE OSTI2025

OLCF’s Advanced Computing Ecosystem (ACE): FY25 Update for Ongoing Efforts

The advent of widespread use of artificial intelligence (AI) and machine learning (ML) models in science, coupled with fast data production rates of scientific instruments strain the traditional batch-oriented high-performance computing (HPC) environment. As scientific exploration continues to require more data and faster processing and analysis, new emerging technologies and capabilities to enable cross-facility and time-sensitive workflows are required for seamless integration of HPC and experimental facilities. The Advanced Computing Ecosystem (ACE) is a strategic initiative within the Oak Ridge Leadership Computing Facility (OLCF) established in 2024 to support the development of cutting-edge technologies to advance computational research and infrastructure at OLCF and across the Department of Energy (DOE). Several DOE initiatives are spearheading the evolution of the scientific landscape by blurring facility boundaries and connecting the user facilities to advance scientific capabilities and ensure energy dominance. The DOE Integrated Research Infrastructure (IRI) program is one example that is laying a foundation to support complex cross-facility workflows. The IRI program aims to integrate diverse computational resources, data infrastructures, and scientific instruments to facilitate collaboration and accelerate scientific discovery. The Interconnected Science Ecosystem (INTERSECT) initiative at Oak Ridge National Laboratory (ORNL) is another example that aims to revolutionize scientific research through AI-driven, interconnected autonomous laboratories and research facilities. Finally, the American Science Cloud (AmSC), recently announced in the “One Big Beautiful Bill”, aims to leverage prior infrastructure efforts of the IRI and automation and AI efforts of INTERSECT (and others) to build a federated, AI-augmented AmSC platform to unify the DOE’s computing, experimental, and data resources to catalyze scientific innovation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Survey-wide asteroid discovery with a high-performance computing enabled non-linear digital tracking framework

Modern astronomical surveys detect asteroids by linking together their appearances across multiple images taken over time. This approach faces limitations in detecting faint asteroids and handling the computational complexity of trajectory linking. Here, we present a novel method that adapts “digital tracking” – traditionally used for short-term linear asteroid motion across images – to work with large-scale synoptic surveys such as the Vera Rubin Observatory Legacy Survey of Space and Time (Rubin/LSST). Our approach combines hundreds of sparse observations of individual asteroids across their non-linear orbital paths to enhance detection sensitivity by several magnitudes. To address the computational challenges of processing massive data sets and dense orbital phase spaces, we developed a specialized high-performance computing architecture. We demonstrate the effectiveness of our method through experiments that take advantage of the extensive computational resources at Lawrence Livermore National Laboratory. This work enables the detection of significantly fainter asteroids in existing and future survey data, potentially increasing the observable asteroid population by orders of magnitude across different orbital families, from near-Earth objects (NEOs) to Kuiper belt objects (KBOs).

Asteroid discovery↗
DOE OSTI2022

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond: Preprint

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same field, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications. To address these challenges, the open source Time-Series Data Pipelines (Tsdat) software was developed by a joint collaboration between Pacific Northwest National Laboratory, the National Renewable Energy Laboratory, and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the Marine Energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat software and the data standards within which the framework operates. A beta version of the framework has been released and is currently being used by several projects in marine energy, wind energy, and building energy systems.

big data↗
DOE OSTI2015

Audits Made Simple

A company just got notified there is a big external audit coming in 3 months. Getting ready for an audit can be challenging, scary, and full of surprises. This Gold Paper describes a typical audit from notification of the intent to audit through disposition of the final report including Best Practices, Opportunities for Improvement (OFI), and issues that must be fixed. Good preparation can improve the chances of success. Ensuring the auditors understand the environment and requirements is paramount to success. It helps the auditors understand that the enterprise really does think that security is important. Understanding and following a structured process ensures a smooth audit process. Ensuring follow-up on OFIs and issues in a structured fashion will also make the next audit easier. It is important to keep in mind that the auditors will use the previous report as a starting point. Now the only worry is the actual audit and subsequent report and how well the company has done. “Virtualization, mobilization, and cloud technology have created new points of entry into business, leaving them vulnerable to covert cyber-attacks,” states a report by Ernst & Young. Executives are struggling with the potential for a data breach, and resolving this business risk is high on their agenda. While most audit committee members are financially solid, they lack knowledge of technology issues, making understanding of this risk difficult. Success with an audit helps management understand the risk management is accepting, highlight gaps in security, and generate ideas on how to improve. These results are an important aspect of managing the business risk.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Large-Scale Compute-Intensive Analysis via a Combined In-situ and Co-scheduling Workflow Approach

Large-scale simulations can produce tens of terabytes of data per analysis cycle, complicating and limiting the efficiency of workflows. Traditionally, outputs are stored on the file system and analyzed in post-processing. With the rapidly increasing size and complexity of simulations, this approach faces an uncertain future. Trending techniques consist of performing the analysis in situ, utilizing the same resources as the simulation, and/or off-loading subsets of the data to a compute-intensive analysis system. We introduce an analysis framework developed for HACC, a cosmological N-body code, that uses both in situ and co-scheduling approaches for handling Petabyte-size outputs. An initial in situ step is used to reduce the amount of data to be analyzed, and to separate out the data-intensive tasks handled off-line. The analysis routines are implemented using the PISTON/VTK-m framework, allowing a single implementation of an algorithm that simultaneously targets a variety of GPU, multi-core, and many-core architectures.

Big data↗
DOE OSTI2021

Learning to Correct Climate Projection Biases

The fidelity of climate projections is often undermined by biases in climate models due to their simplification or misrepresentation of unresolved climate processes. While various bias correction methods have been developed to post-process model outputs to match observations, existing approaches usually focus on limited, low-order statistics, or break either the spatiotemporal consistency of the target variable, or its dependency upon model resolved dynamics. We develop a Regularized Adversarial Domain Adaptation (RADA) methodology to overcome these deficiencies, and enhance efficient identification and correction of climate model biases. Instead of pre-assuming the spatiotemporal characteristics of model biases, we apply discriminative neural networks to distinguish historical climate simulation samples and observation samples. The evidences based on which the discriminative neural networks make distinctions are applied to train the domain adaptation neural networks to bias correct climate simulations. We regularize the domain adaptation neural networks using cycle-consistent statistical and dynamical constraints. An application to daily precipitation projection over the contiguous United States shows that our methodology can correct all the considered moments of daily precipitation at approximately $1^\circ$ resolution, ensures spatiotemporal consistency and inter-field correlations, and can discriminate between different dynamical conditions. Our methodology offers a powerful tool for disentangling model parameterization biases from their interactions with the chaotic evolution of climate dynamics, opening a novel avenue toward big-data enhanced climate predictions.

58 GEOSCIENCES↗
DOE OSTI2024

Testing Protocol Development for the Fracture Toughness of Parts Built with Big Area Additive Manufacturing

The mechanical testing of additively manufactured parts has largely relied on the existing standards developed for traditional manufacturing. While this approach leverages the investment made in current standards development, it inaccurately assumes that the mechanical response of additive manufacturing (AM) parts is identical to that of parts manufactured through traditional processes. When considering thermoplastic, material extrusion AM, the differences in response can be attributed to an AM part’s inherent inhomogeneity caused by porosity, interlayer zones, and surface texture. Additionally, the interlayer bonding of parts printed with large-scale AM is difficult to adequately assess, as much testing is performed such that stress is distributed across many layer interfaces; therefore, the lack of AM-specific standards to assess interlayer bonding is a significant research gap. To quantify interlayer bonding via fracture toughness, double cantilever beam (DCB) testing has been used for some AM materials, and DCB has been generally used for a variety of materials including metal, wood, and laminates. Mode I DCB testing was performed on thermoplastic matrix composites printed with Big Area Additive Manufacturing (BAAM). Of particular interest was the notch shape and deflection speed during testing. The results examine the differences when using two notch types and three deflection speeds. The testing method introduced by the following paper differentiates itself from the ones described in the standards used by modernizing the methodology. This was conducted with the introduction of Digital Image Correlation (DIC) to gather displacement and load data simultaneously without human intervention.

Polymer Science↗
DOE OSTI2015

A SEMI-COHERENT SEARCH FOR WEAK PULSATIONS IN AQUILA X–1

Non-pulsating neutron stars in low mass X-ray binaries largely outnumber those that show pulsations. The lack of detectable pulses represents a big open problem for two important reasons. The first is that the structure of the accretion flow in the region closest to the neutron star is not well understood and it is therefore unclear what is the mechanism that prevents the pulse formation. The second is that the detection of pulsations would immediately reveal the spin of the neutron star. AQUILA X–1 is a special source among low mass X-ray binaries because it has showed the unique property of pulsating for only ∼150 s out of a total observing time of more than 1.5 million seconds. However, the existing upper limits on the pulsed fraction leave open two alternatives. Either AQUILA X–1 has very weak pulses which have been undetected, or it has genuinely pulsed only for a tiny amount of the observed time. Understanding which of the two scenarios is the correct one is fundamental to increase our knowledge about the pulse formation process and understand the chances we have to detect weak pulses in other low-mass X-ray binaries. In this paper we perform a semi-coherent search on the entire X-ray data available for AQUILA X–1. We find no evidence for (new) weak pulsations with the most stringent upper limits being of the order of 0.3% in the 7–25 keV energy band.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

ASCR Workshop on In Situ Data Management: Enabling Scientific Discovery from Diverse Data Sources

In January 2019, the U.S. Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions for in situ data management (ISDM). The workshop defined ISDM as the practices, capabilities, and procedures to control the organization of data and enable the coordination and communication among heterogeneous tasks, executing simultaneously in a high-performance computing system, cooperating toward a common objective. The workshop revealed two primary, interdependent motivations for processing and managing data in situ. The first motivation is that the in situ methodology enables scientific discovery from a broad range of data sources over a wide scale of computing platforms: leadership-class systems, clusters, clouds, workstations, and embedded devices at the edge. The successful development of ISDM capabilities will benefit real-time decision-making, design optimization, and data-driven scientific discovery. The second motivation is the need to decrease data volumes. ISDM can make critical contributions to managing large data volumes from computations and experiments to minimize data movement, save storage space, and boost resource efficiency, often while simultaneously increasing scientific precision. A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery across the Office of Science mission space. Applications wanting to use the in situ capabilities include those where data analysis feeds back to the simulation, decisions are made autonomously, big data or machine learning is among the tasks to be coordinated, and computations need to be completed in real time. The workshop identified six priority research directions that highlight the components and capabilities needed for ISDM to be successful for the wide variety of applications discussed: making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗