Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “big”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Enabling discovery data science through cross-facility workflows

Experimental and observational instruments for scientific research (such as light sources, genome sequencers, accelerators, telescopes and electron microscopes) increasingly require High Performance Computing (HPC) scale capabilities for data analysis and workflow processing. Next-generation instruments are being deployed with higher resolutions and faster data capture rates, creating a big data crunch that cannot be handled by modest institutional computing resources. Often these big data analysis pipelines also require near real-time computing and have higher resilience requirements than the simulation and modeling workloads more traditionally seen at HPC centers. While some facilities have enabled workflows to run at a single HPC facility, there is a growing need to integrate capabilities across HPC facilities to enable cross-facility workflows, either to provide resilience to an experiment, increase analysis throughput capabilities, or to better match a workflow to a particular architecture. In this paper we describe the barriers to executing complex data analysis workflows across HPC facilities and propose an architectural design pattern for enabling scientific discovery using cross-facility workflows that includes orchestration services, application programming interfaces (APIs), data access and co-scheduling.

Antypas, Katerina B.↗
DOE OSTI2021

Creating a Tools Ecosystem for Cross-Discipline Environmental Data Reuse

Reusing data is difficult even within well-defined science communities and only gets worse when combining data from multiple communities and disciplines. Through the lens of current work on constructing an environmental epidemiological data set from multiple disciplinary sources, we demonstrate the need for a new tool ecosystem to support heterogeneous Big Data science. Extending existing community standards for schemas and/or data formats through human auditing and wrangling of the data is not feasible at scale. This work therefore suggests new approaches for the multi-disciplinary communities to build a shared tool ecosystem for big data. We discuss both the larger context of data wrangling of epidemiological data sets for novel artificial intelligence algorithms and the specific lessons from working with these multi-disciplinary data sets. Adopting a more model-driven, automatable approach promises not only better efficiency but also removes key sources of human-generated errors and promotes reuse and reproducibility of science data.

Logan, Jeremy↗
DOE OSTI2021

Virtual Log-Structured Storage for High-Performance Streaming

Over the past decade, given the higher number of data sources (e.g., Cloud applications, Internet of things) and critical business demands, Big Data transitioned from batch-oriented to real-time analytics. Stream storage systems, such as Apache Kafka, are well known for their increasing role in real-time Big Data analytics. For scalable stream data ingestion and processing, they logically split a data stream topic into multiple partitions. Stream storage systems keep multiple data stream copies to protect against data loss while implementing a stream partition as a replicated log. This architectural choice enables simplified development while trading cluster size with performance and the number of streams optimally managed. This paper introduces a shared virtual log-structured storage approach for improving the cluster throughput when multiple producers and consumers write and consume in parallel data streams. Stream partitions are associated with shared replicated virtual logs transparently to the user, effectively separating the implementation of stream partitioning (and data ordering) from data replication (and durability). We implement the virtual log technique in the KerA stream storage system. When comparing with Apache Kafka, KerA improves the cluster ingestion throughput by up to 4x when multiple producers write over hundreds of data streams.

consistent stream ordering↗
DOE OSTI2023

Traction Motor Design Trade-Offs with Additively Manufactured Anisotropic Bonded Magnets

Extrusion based big area additive manufacturing (BAAM) has been previously demonstrated for the fabrication of dense anisotropic bonded magnets. This paper reports on design trade-offs when such magnets are employed in a motor designed for electric vehicle applications. An external rotor fractional slot permanent magnet synchronous machine designed with sintered dysprosium free magnets rated for 100kW peak and 50kW continuous power is used as the baseline. The impact of using the bonded BAAM magnets on the motor and inverter volumes, motor loss and cooling systems as well as demagnetization margins is evaluated. It is found that although the motor power density reduces due to the lower energy product of the BAAM magnets, there are other system level benefits including reductions in the cooling system requirement, lower inverter rating and volume, decrease in short circuit current and improved demagnetization marains.

additive manufacturing↗
DOE OSTI2024

DICER: Data Intensive Computing Environment and Runtime for Evaluating Unprecedented Scale of Geospatial-Temporal Human Mobility Data

With the significant increase in sources and volume of human mobility data through commercial data vendors as well as microsimulation of cities, the scale of geospatial-temporal data to analyze and assess for mobility characterization has grown to the level of Big Data. There are mobility related commercial organizations deploying scalable computing, but often the system architecture, workflow, and intermediate processing components are not fully disclosed in relevant scope. Current research literature has a notable lack of studies demonstrating architectures and workflows for human mobility analytics that are implemented on a TeraByte scale of geospatial-temporal data. In this context, this paper presents a hyperscale-level system solution named DICER (Data Intensive Computing Environment and Runtime) for processing and analytics of geospatial-temporal data at big data scale. Although the cluster computing architecture of DICER with Apache Spark job running on Kubernetes cluster is not new, there are innovations in the workflow, hierarchical processing logic, and a wide range of intermediate preprocessing and mobility metrics calculation. We have performed case studies to validate the effectiveness of DICER system solution by performing detailed analytics and assessment of human mobility microsimulation output at three different scopes and scale, including a usecase with 16.97 TeraByte and 259.2 Billion rows of data. In addition, we have presented another case study of utilizing DICER to perform the same mobility processing and comparative analytics on large-scale commercially available geospatial-temporal data. All these case studies validate the efficiency and usefulness of DICER in computing population mobility characteristics from geospatial-temporal trajectory data at an unprecedented scale (not only just data volume, but also combination of: number of user entities, temporal frequency, spatial resolution, data duration).

De, Debraj↗
DOE OSTI2020

A Survey of Singular Value Decomposition Methods for Distributed Tall/Skinny Data

The Singular Value Decomposition (SVD) is one of the most important matrix factorizations, enjoying a wide variety of applications across numerous application domains. In statistics and data analysis, the common applications of SVD inclue Principal Components Analysis (PCA) and regression. Usually these applications arise on data that has far more rows than columns, so-called "tall/skinny" matrices. In the big data analytics context, this may take the form of hundreds of millions to billions of rows with only a few hundred columns. There is a need, therefore, for fast, accurate, and scalable tall/skinny SVD implementations which can fully utilize modern computing resources. To that end, we present a survey of three different algorithms for computing the SVD for these kinds of tall/skinny data layouts using MPI for communication. We contextualize these with common big data analytics techniques. Finally, we present both CPU and GPU timing results from the Summit supercomputer, and discuss possible alternative approaches.

Schmidt, Drew↗
DOE OSTI2024

Time series methods for the analysis of soundscapes and other cyclical ecological data

Biodiversity monitoring has entered an era of ‘big data’, exemplified by a near-continuous collection of sounds, images, chemical and other signals from organisms in diverse ecosystems. Such data streams have the potential to help identify new threats, assess the effectiveness of conservation interventions, as well as generate new ecological insights. However, appropriate analytical methods are often still missing, particularly with respect to characterizing cyclical temporal patterns. Here, we present a framework for characterizing and analysing ecological responses that represent nonstationary, complex temporal patterns and demonstrate the value of using Fourier transforms to decorrelate continuous data points. In our example, we use a framework based on three approaches (spectral analysis, magnitude squared coherence, and principal component analysis) to characterize differences in tropical forest soundscapes within and across sites and seasons in Gabon. By reconstructing the underlying, cyclic behaviour of the soundscape for each site, we show how one can identify circadian patterns in acoustic activity. Soundscapes in the dry season had a complex diel cycle, requiring multiple harmonics to represent daily variation, while in the wet season there was less variance attributable to the daily cyclic patterns. Our framework can be applied to most continuous, or near-continuous ecological data collected at a fine temporal resolution, allowing ecologists to explore patterns of temporal autocorrelation at multiple levels for biologically meaningful trends. Such methods will become indispensable as biological big data are used to understand the impact of anthropogenic pressures on biodiversity and to inform efforts to mitigate them.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Braneworld cosmological effect on freeze-in dark matter density and lifetime frontier

In the 5-dimensional braneworld cosmology, the Friedmann equation of our 4-dimensional universe on a brane is modified at high temperatures while the standard Big Bang cosmology is reproduced at low temperatures. Based on two well-known scenarios, the Randall–Sundrum and Gauss–Bonnet braneworld cosmologies, we investigate the braneworld cosmological effect on the relic density of a non-thermal dark matter particle whose interactions with the Standard Model particles are so weak that its relic density is determined by the freeze-in mechanism. For dark matter production processes in the early universe, we assume a simple scenario with a light vector-boson mediator for the dark matter particle to communicate with the Standard Model particles. We find that the braneworld cosmological effect can dramatically alters the resultant dark matter relic density from the one in the standard Big Bang cosmology. As an application, we consider a right-handed neutrino dark matter in the minimal B – L extended Standard Model with a light B – L gauge boson (Z') as a mediator. We find an impact of the braneworld cosmological effect on the search for the long-lived Z' boson at the planned/proposed Lifetime Frontier experiments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Region-adaptive, Error-controlled Scientific Data Compression using Multilevel Decomposition

The increase of computer processing speed is significantly outpacing improvements in network and storage bandwidth, leading to the big data challenge in modern science, where scientific applications can quickly generate much more data than that can be transferred and stored. As a result, big scientific data must be reduced by a few orders of magnitude while the accuracy of the reduced data needs to be guaranteed for further scientific explorations. Moreover, scientists are often interested in some specific spatial/temporal regions in their data, where higher accuracy is required. The locations of the regions requiring high accuracy can sometimes be prescribed based on application knowledge, while other times they must be estimated based on general spatial/temporal variation. In this paper, we develop a novel multilevel approach which allows users to impose region-wise compression error bounds. Our method utilizes the byproduct of a multilevel compressor to detect regions where details are rich and we provide the theoretical underpinning for region-wise error control. With spatially varying precision preservation, our approach can achieve significantly higher compression ratios than single-error bounded compression approaches and control errors in the regions of interest.We conduct the evaluations on two climate use cases – one targeting small-scale, node features and the other focusing on long, areal features. For both use cases, the locations of the features were unknown ahead of the compression. By selecting approximately 16% of the data based on multi-scale spatial variations and compressing those regions with smaller error tolerances than the rest, our approach improves the accuracy of post-analysis by approximately 2 × compared to single-error-bounded compression at the same compression ratio. Using the same error bound for the region of interest, our approach can achieve an increase of more than 50% in overall compression ratio.

Gong, Qian↗
DOE OSTI2023

RAPIDS: Reconciling Availability, Accuracy, and Performance in Managing Geo-Distributed Scientific Data

In modern science, big data plays an increasingly important role. Many scientific applications, such as running simulations on supercomputers or conducting experiments on advanced instruments, produce huge amount of data at unprecedented speed. Analyzing and understanding such big data is the key for scientists to make scientific breakthroughs. However, data might become unavailable for scientists to access when outages or maintenance of the storage system occur, which severely hinders scientific discovery. To improve the data availability, data duplication and erasure coding (EC) are often used. But as the scientific data gets larger, using these two methods can cause considerable storage and network overhead.In this paper, we propose RAPIDS, a hybrid approach that combines the multigrid-based error-bounded lossy compression with erasure coding, to significantly reduce the storage and network overhead required for maintaining high data availability. Our experiments show that RAPIDS reduces the storage overhead by up to 7.5x and network overhead by up to 3x to achieve the same level of availability compared to the regular EC method. We improve RAPIDS by building two models to optimize the fault tolerance configurations and data gathering strategy. We demonstrate that RAPIDS significantly improves performance when running on many CPU cores in parallel or on GPUs.

Wan, Lipeng↗
DOE OSTI2022

Development of a simplified calibrated building simulation model of a supermarket for proposed ECMs and control strategies impact evaluation

Calibrated building energy simulation is an important pathway to more energy-efficient buildings, but the information requirements of some approaches to this problem are significant. This is particularly true for supermarkets and other so-called “big-box” retail stores. Another characteristic of supermarkets is the significant interaction between Heating Ventilating and Air Conditioning (HVAC) and refrigeration systems in these buildings. These buildings could contain a wide variety of systems and a degree of load diversity that makes calibrated modeling a challenge. This paper describes a simplified approach that uses OpenStudio and EnergyPlus to combine known building parameters with “typical” parameters, resulting in a simplified building that is amenable to calibration. This approach was applied to a big-box store located in Nashville, Tennessee, and a calibrated model was obtained that was used to study potential energy conservation measures. Further, the paper also explores the capabilities of whole-building energy modeling tools, such as EnergyPlus, for modeling the HVAC controls and sequences and their impact evaluation. Although some measures are precluded by the model simplicity, several measures were found to improve the efficiency of the model and demonstrate that the simplified modeling approach is effective. Practical Application: This paper introduces a hybrid approach of building energy model calibration using limited information available from the actual building in combination with characteristics of a “typical” building of the same type. This hybrid approach would also be applicable for other building types than discussed in this paper to calibrate the building energy model using limited information from the actual building.

42 ENGINEERING↗
DOE OSTI2021

AmeriFlux US-xPU NEON Pu'u Maka'ala Natural Area Reserve (PUUM)

This is the AmeriFlux version of the carbon flux data for the site US-xPU NEON Pu'u Maka'ala Natural Area Reserve (PUUM). Site Description - NEON's PUUM field site is located in the Pu'u Maka'ala Natural Area Reserve (NAR) on the eastern side of Hawaii’s “Big Island,” managed by the Hawaii Division of Forestry and Wildlife (DOFAW). More than 18,000 acres in size, the NAR is home to a rainforest with many native species, some of them endangered. It was established to protect some of the Big Island’s best wet native forest and unique geologic features.

Network), NEON (National Ecological Observatory↗
DOE OSTI2025

AmeriFlux FLUXNET-1F US-xPU NEON Pu'u Maka'ala Natural Area Reserve (PUUM)

This is the AmeriFlux Management Project (AMP) created FLUXNET-1F version of the carbon flux data for the site US-xPU NEON Pu'u Maka'ala Natural Area Reserve (PUUM). This is the FLUXNET version of the carbon flux data for the site US-xPU NEON Pu'u Maka'ala Natural Area Reserve (PUUM) produced by applying the standard ONEFlux (1F) software. Site Description - NEON's PUUM field site is located in the Pu'u Maka'ala Natural Area Reserve (NAR) on the eastern side of Hawaii’s “Big Island,” managed by the Hawaii Division of Forestry and Wildlife (DOFAW). More than 18,000 acres in size, the NAR is home to a rainforest with many native species, some of them endangered. It was established to protect some of the Big Island’s best wet native forest and unique geologic features.

Network), NEON (National Ecological Observatory [N↗
DOE OSTI2020

AI for Science: Report on the Department of Energy (DOE) Town Halls on Artificial Intelligence (AI) for Science

The report documents the DOE Town Halls held during 2019 at Argonne National Laboratory, Oak Ridge National Laboratory, Lawrence Berkeley National Laboratory, and in Washington, DC. From July to October 2019, the Argonne, Oak Ridge, and Berkeley National Laboratories hosted a series of four town hall meetings attended by more than 1,000 U.S. scientists and engineers. The goal of the town hall series was to examine scientific opportunities in the areas of artificial intelligence (AI), Big Data, and high-performance computing (HPC) in the next decade, and to capture the big ideas, grand challenges, and next steps to realizing these opportunities. In this report and in the Department of Energy (DOE) laboratory community, we use the term “AI for Science” to broadly represent the next generation of methods and scientific opportunities in computing, including the development and application of AI methods (e.g., machine learning, deep learning, statistical methods, data analytics, automated control, and related areas) to build models from data and to use these models alone or in conjunction with simulation and scalable computing to advance scientific research. The AI for Science town hall discussions focused on capturing the transformational uses of AI that employ HPC and/or data analysis, leveraging data sets from HPC simulations or instruments and user facilities, and addressing scientific challenges unique to DOE user facilities and the agency’s wide-ranging fundamental and applied science enterprise.

36 MATERIALS SCIENCE↗
DOE OSTI2020

ASCR@40

One page flyers on: A Revolution in Modeling and Simulation; Applying Equations to Complex Problems; Uncertainty Quantification; Moving Big Data; Grid Computing for High-Speed Collaboration; Making Sense of Big Data; Overcoming Scaling Challenges; Building Better Computers; World-Leading Computing Facilities; Supporting Science Through Open-Source Software; and Building the Computational Workforce. Content provided by Department of Energy multiprogram laboratory researchers. Prepared by the Krell Institute for the ASCAC Subcommittee on the 40-year History of ASCR.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Hanging String Cuts in SPR Caverns: Modeling Investigation and Comparison with Sonar Data

Investigation of leaching for oil sales includes looking closely at cavern geometries. Anomalous cavern "features" have been observed near the foot of some caverns subsequent to partial drawdowns. One potential mitigation approach to reducing further growth of preexisting features is based on the hypothesis that reducing the brine string length via a "string cue' would serve to move the zone associated with additional leaching to a location higher up in the cavern and thus away from the preexisting feature. Cutting of the hanging string is expected to provide a control of leaching depth that could be used to "smooth" existing features and thus reduce geomechanical instability in that region of the cavern. The SANSMIC code has been used to predict cavern geometry changes (i.e., the extent of cavern growth with depth) based on variable input parameters for four caverns: West Hackberry 11 (WH11), West Hackberry 113 (WH113), Big Hill 104 (BH104), Big Hill 114 (BH114). By comparing the initial sonar geometry with resultant geometries calculated by the SANSMIC code, conclusions may be drawn about the potential impact of these variables on future cavern growth. Ultimately, these conclusions can be used to assess possible mitigation strategies such as the potential advantage of cutting versus not cutting a brine string. This work has resulted in a recommendation that a hanging string cut of 80 ft in WH11 would be beneficial to future cavern geometry, while there would be little to no benefit to string cuts in the other three caverns investigated here. The WH11 recommendation was followed in 2019, resulting in an operational string cut. A sonar performed after the string cut showed no adverse leaching in the area of the preexisting flare, as expected from the results of the preliminary SANSMIC runs described in this report. Additional SANSMIC modeling of the actual amount of injected raw water resulted in good agreement with the post-cut sonar.

02 PETROLEUM↗
DOE OSTI2020

Opportunities and Challenges from Artificial Intelligence and Machine Learning for the Advancement of Science, Technology, and the Office of Science Missions

In February 2019, the President signed Executive Order 13859, Maintaining American Leadership in Artificial Intelligence. This order launched the American Artificial Intelligence Initiative, a concerted effort to promote and protect AI technology and innovation in the United States. The Initiative implements a government-wide strategy in collaboration and engagement with the private sector, academia, the public, and like-minded international partners. Among other actions, key directives in the Initiative called for Federal agencies to: Prioritize AI research and development investments, Enhance access to high-quality cyberinfrastructure and data, Ensure that the US maintains an international leadership role in the development of technical standards for AI, and Provide education and training opportunities to prepare the American workforce for the new era of AI. The mission of the Department of Energy (DOE) is to ensure America’s security and prosperity by addressing its energy, environmental, and nuclear challenges through transformative science and technology solutions. In terms of Science and Innovation, the DOE’s mission is to maintain a vibrant US effort in science and engineering as a cornerstone of our economic prosperity with clear leadership in strategic areas. From July to October in 2019, the Argonne, Oak Ridge, and Berkeley National Laboratories hosted a series of four AI for Science Town Hall meetings in Chicago, Oak Ridge, Berkeley, and Washington DC. The four meetings were attended by over 1300 scientists from the 17 DOE Labs, 39 companies, and over 90 universities. The goal of the Town Hall series was ‘to examine scientific opportunities in the areas of artificial intelligence, Big Data, and high-performance computing (HPC) in the next decade, and to capture the big ideas, grand challenges, and next steps to realizing these.’ The discussions at the meetings were captured in the final report of the AI for Science Town Hall meetings.

42 ENGINEERING↗
DOE OSTI2021

Elucidating and predicting the dynamic evolution of water and land systems due to natural and energy-related forcings

Focal Area(s): 3. Insight gleaned from complex data (both observed and simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI; & 1. Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization, unsupervised learning (including deep learning), and hardware-related efforts involving AI (e.g., edge computing). Science Challenge: Interactions between water, land, and energy systems are complex and occur on a variety of scales, ranging from local to basinal to regional. Accurately predicting the behavior of ground water and surface water systems for 5-10 years and beyond requires an understanding of the current system and the ability to model both the natural system at scale and human-induced forcings related to energy and other activities. Artificial intelligence and machine learning (AI/ML) combined with modern compilation and integration efforts for U.S. groundwater and surface water systems present potential solutions to bolstering detailed physics-based models of these systems. Big data tied with ML and physics-based modeling can drive breakthroughs in understanding the earth system, but research is often impeded by data access (e.g., privacy issues), quality, formats, gaps, multi-source, multi-scale, integration, and spatiotemporal challenges. Effective integration of real data and simulated (synthetic) data that fill gaps is critical. Overcoming these complex data and model integration challenges will enable a transformational approach to acquiring enhanced understanding of environmental systems.

54 ENVIRONMENTAL SCIENCES↗