Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Big data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2013

Integrating Online Compression to Accelerate Large-Scale Data Analytics Applications

Abstract—With growing dataset sizes, and as computing cycles are increasing faster than storage and wide-area bandwidths, compression appears like a promising approach for improving the performance of large-scale data analytics applications. In this context, this paper makes the following contributions. First, we develop a new compression methodology, which exploits the similarities between spatial and/or temporal neighbors in a simulation dataset, and enables high compression ratios and low decompression costs. Second, we have developed a framework that can be used to incorporate a variety of compression and decompression algorithms. This framework also supports a simple API to allow integration with an existing application or data processing middleware. Once a compression algorithm is implemented, this framework can allow multi-threaded retrieval, multi-threaded data decompression, and use of informed prefetching and caching. By integrating this framework with a data-intensive middleware, we have applied our compression methodology and framework to three applications over two datasets, including a GCRM climate model dataset. We obtained an average compression ratio of 51.68%, and up to 53.27% improvement in execution time of data analysis applications.

Big Data, I/O, HPC↗
DOE OSTI2016

Making Advanced Scientific Algorithms and Big Scientific Data Management More Accessible

Synchrotrons such as the Advanced Light Source (ALS) at Lawrence Berkeley National Laboratory are known as user facilities. They are sources of extremely bright X-ray beams, and scientists come from all over the world to perform experiments that require these beams. As the complexity of experiments has increased, and the size and rates of data sets has exploded, managing, analyzing and presenting the data collected at synchrotrons has been an increasing challenge. The ALS has partnered with high performance computing, fast networking, and applied mathematics groups to create a"super-facility", giving users simultaneous access to the experimental, computational, and algorithmic resources to overcome this challenge. This combination forms an efficient closed loop, where data despite its high rate and volume is transferred and processed, in many cases immediately and automatically, on appropriate compute resources, and results are extracted, visualized, and presented to users or to the experimental control system, both to provide immediate insight and to guide decisions about subsequent experiments during beam-time. In this paper, We will present work done on advanced tomographic reconstruction algorithms to support users of the 3D micron-scale imaging instrument (Beamline 8.3.2, hard X-ray micro-tomography).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

An effective online data monitoring and saving strategy for large-scale climate simulations

Large-scale climate simulation models have been developed and widely used to generate historical data and study future climate scenarios. These simulation models often have to run for a couple of months to understand the changes in the global climate over the course of decades. This long-duration simulation process creates a huge amount of data with both high temporal and spatial resolution information; however, how to effectively monitor and record the climate changes based on these large-scale simulation results that are continuously produced in real time still remains to be resolved. Due to the slow process of writing data to disk, the current practice is to save a snapshot of the simulation results at a constant, slow rate although the data generation process runs at a very high speed. This study proposes an effective online data monitoring and saving strategy over the temporal and spatial domains with the consideration of practical storage and memory capacity constraints. Finally, our proposed method is able to intelligently select and record the most informative extreme values in the raw data generated from real-time simulations in the context of better monitoring climate changes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2015

UPR/Mayaguez High Energy Physics

For the period of sixteen years covered by this report (June 1, 1997 - July 31, 2013) the High Energy Physics Group at the University of Puerto Rico’s Mayaguez Campus (UPRM) carried out an extensive research program that included major experiments at Fermi National Accelerator Laboratory (Fermilab), the Cornell Electron-positron Collider and CERN. In particular, these were E831 (FOCUS) at Fermilab, CLEOc at Cornell and the Compact Muon Solenoid (CMS) at the Large Hadron Collider (LHC) at CERN. The group’s history is one of successful execution and growth. Beginning with one faculty researcher in 1985, it eventually included four faculty researchers, one post-doctoral research associate, two undergraduates and as many as six graduate students at one time working on one of the experiments that discovered the Higgs boson. Some of this expansion was due to the group’s leveraging of funds from the Department of Energy’s core grant to attract funds from National Science Foundation programs not targeted to high energy physics. Besides the group’s research productivity, its other major contribution was the training of a large number of MS students who later went on to successful technical careers in industry as well as academia including many who obtained PhD degrees at US universities. In an attempt to document this history, this final report gives a general description of the Group’s work prior to June 1, 2010, the starting date for the last grant renewal period. Much more detail can, of course, be found in the annual reports submitted up to that date. The work during the last grant period is discussed in detail in a separate section. To summarize the group’s scientific accomplishments, one can point to the results of the experiments. Both FOCUS and CLEOc were designed to carry out precise measurements of processes involving the heavy quarks, charm and bottom. Heavy quarks are particularly interesting because, due to their mass, theoretical calculations based on the Standard Model have less uncertainty than those for the light quarks. Precise heavy quark experiments can therefore yield some of the best tests of the Standard Model and of the approximations that are made in calculating measurable observables. Both FOCUS and CLEOc were highly successful achieving significant improvement in the precision of measurements such as lifetimes and decay branching ratios. For example, FOCUS obtained a data sample that contained ten times as many heavy quark decay events as its predecessor. CMS was a big shift in the group’s research. During the first decade of the century it became clear that the LHC would be the world’s highest energy accelerator offering a unique opportunity for discovery. Given the UPRM’s group record of achievement, it was successful in obtaining admission to the CMS collaboration in March, 2006, becoming the first institution to do so that did not have a PhD program. CMS is one of two major experiments at the LHC. Although the plans are for these experiments to run for many years with increased energy and event rates, they have already achieved one of their principal goals. The test for the existence of the Higgs boson, a particle which plays a unique role in the Standard Model but had not been observed, was answered in the affirmative in 2012.The particular contributions of the UPRM group to these experiments make up the majority of this report although other contributions such as the training of students, outreach to the general community and the organization of scientific meetings are also discussed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Python and HPC for High Energy Physics Data Analyses

High level abstractions in Python that can utilize computing hardware well seem to be an attractive option for writing data reduction and analysis tasks. In this paper, we explore the features available in Python which are useful and efficient for end user analysis in High Energy Physics (HEP). A typical vertical slice of an HEP data analysis is somewhat fragmented: the state of the reduction/analysis process must be saved at certain stages to allow for selective reprocessing of only parts of a generally time-consuming workflow. Also, algorithms tend to to be modular because of the heterogeneous nature of most detectors and the need to analyze different parts of the detector separately before combining the information. This fragmentation causes difficulties for interactive data analysis, and as data sets increase in size and complexity (O10 TiB for a “small” neutrino experiment to the O10 PiB currently held by the CMS experiment at the LHC), data analysis methods traditional to the field must evolve to make optimum use of emerging HPC technologies and platforms. Mainstream big data tools, while suggesting a direction in terms of what can be done if an entire data set can be available across a system and analysed with high-level programming abstractions, are not designed with either scientific computing generally, or modern HPC platform features in particular, such as data caching levels, in mind. Our example HPC use case is a search for a new elementary particle which might explain the phenomenon known as “Dark Matter”. Here, using data from the CMS detector, we will use HDF5 as our input data format, and MPI with Python to implement our use case.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Near real-time streaming analysis of big fusion data

Experiments on fusion plasmas produce high-dimensional data time series with ever-increasing magnitude and velocity, but turn-around times for analysis of this data have not kept up. For example, many data analysis tasks are often performed in a manual, ad-hoc manner some time after an experiment. In this article, we introduce the Delta framework that facilitates near real-time streaming analysis of big and fast fusion data. By streaming measurement data from fusion experiments to a high-performance compute center, Delta allows computationally expensive data analysis tasks to be performed in between plasma pulses. This article describes the modular and expandable software architecture of Delta and presents performance benchmarks of individual components as well as of an example workflow. Focusing on a streaming analysis workflow where electron cyclotron emission imaging (ECEi) data is measured at KSTAR on the National Energy Research Scientific Computing Center's (NERSC's) supercomputer we routinely observe data transfer rates of about 4 Gigabit per second. In NERSC, a demanding turbulence analysis workflow effectively utilizes multiple nodes and graphical processing units and executes them in under 5 min. We further discuss how Delta uses modern database systems and container orchestration services to provide web-based real-time data visualization. For the case of ECEi data we demonstrate how data visualizations can be augmented with outputs from machine learning models. Here, by providing session leaders and physics operators, results of higher-order data analysis using live visualizations may make more informed decisions on how to configure the machine for the next shot.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2018

Using Knowledge Analytics to Search and Characterize Mass Properties Aerospace Data

There is growing capability in the field of “Big Data” and “Data Analytics” which Mass Properties Engineers might like to take advantage of. This paper utilizes an implementation of the IBM Knowledge Analytics and Watson search capabilities to explore a corpus of material developed primarily with the interests of Mass Properties Engineers and vehicle concept developers at its forefront. The full collection of SAWE (Society of Allied Weight Engineers, Inc.) Technical Papers from 1939 through 2015 is a major portion of the knowledge content. Additional aerospace vehicle design information includes metadata from AIAA (American Institute for Aeronautics and Astronautics), and INCOSE (International Council on Systems Engineering) as well as author-provided personal search material. This data is processed with respect to certain expected content, data taxonomies and key words to become the core data in NASA Langley Research Center’s “Vehicle Analysis Analytics”, IBM Watson Content. Processed data becomes the corpus of information which is interrogated to provide examples of finding data for mass regression analysis, technology impacts on MPE (Mass Properties Engineering), mass properties control, standards, and other aspects of interest.

Cerro, Jeffrey A.↗
DOE OSTI2019

Astrophysical S -factors, Thermonuclear Rates, and Electron Screening Potential for the 3 He(d,p) 4 He Big Bang Reaction via a Hierarchical Bayesian Model

Here, we developed a hierarchical Bayesian framework to estimate S-factors and thermonuclear rates for the 3 He(d,p) 4 He reaction, which impacts the primordial abundances of 3 He and 7 Li. The available data are evaluated and all direct measurements are taken into account in our analysis for which we can estimate separate uncertainties for systematic and statistical effects. For the nuclear reaction model, we adopt a single-level, two-channel approximation of R-matrix theory, suitably modified to take the effects of electron screening at lower energies into account. In addition to the usual resonance parameters (resonance location and reduced widths for the incoming and outgoing reaction channel), we include the channel radii and boundary condition parameters in the fitting process. Our new analysis of the 3 He(d,p) 4 He S-factor data results in improved estimates for the thermonuclear rates. This work represents the first nuclear rate evaluation using R-matrix theory embedded into a hierarchical Bayesian framework, properly accounting for all known sources of uncertainty. Therefore, it provides a test bed for future studies of more complex reactions.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2019

Ask-The-Expert: Minimizing Human Review for Big Data Analytics Through Active Learning

In this CIF project, we worked toward semi-automating knowledge discovery from anomaly detection algorithms through the use of active learning. Active learning is an area of research within machine learning that uses an "expert in the loop" to learn from large data sets that have very few annotations or labels available, and where providing such labels is expensive. In our case, the task can be defined as the identification of safety events from flight operational data. Since traditional anomaly detection algorithms cannot differentiate between operationally relevant and irrelevant statistical anomalies, Subject Matter Experts (SMEs) have a lengthy and expensive burden of investigating every example identified by the detection algorithm, classifying and labeling them as relevant or irrelevant. Active learningidentifies the unlabeled example for which a label would most improve the classifier, asks the domain expert for a label, and repeats this process until there are no more resources (time, budget) available for labeling or a minimum required performance is reached. A positive label indicates an operationally significant safety event whereas a negative label indicates otherwise. Based on these few labels we propose to build an active learning system that utilizes the SME's time in the most effective manner by iteratively asking for labels for as few informative instances as possible. Our work was proposed to be a stepping stone toward implementation and deployment of the system with user interface to be pursued by the Aviation Operations and Safety Program (AOSP) given its interest in safety monitoring and discovery of safety incidents.

aviation safety↗
DOE OSTI2020

DeepFreeze: Towards Scalable Asynchronous Checkpointing of Deep Learning Models

In the age of big data, deep learning has emerged as a powerful tool to extract insight and exploit its value, both in industry and scientific applications. One common pattern emerging in such applications is frequent checkpointing of the state of the learning model during training, needed in a variety of scenarios: analysis of intermediate states to explain features and correlations with training data, exploration strategies involving alternative models that share a common ancestor, knowledge transfer, resilience, etc. However, with increasing size of the learning models and popularity of distributed data-parallel training approaches, simple checkpointing techniques used so far face several limitations: low serialization performance, blocking I/O, stragglers due to the fact that only a single process is involved in checkpointing. This paper proposes a checkpointing technique specifically designed to address the aforementioned limitations, introducing efficient asynchronous techniques to hide the overhead of serialization and I/O, and distribute the load over all participating processes. Experiments with two deep learning applications (CANDLE and ResNet) on a pre-Exascale HPC platform (Theta) shows significant improvement over state-of-art, both in terms of checkpointing duration and runtime overhead.

checkpointing↗
DOE OSTI2017

Use of DAGMan in CRAB3 to improve the splitting of CMS user jobs

CRAB3 is a workload management tool used by CMS physicists to analyze data acquired by the Compact Muon Solenoid (CMS) detector at the CERN Large Hadron Collider (LHC). Research in high energy physics often requires the analysis of large collections of files, referred to as datasets. The task is divided into jobs that are distributed among a large collection of worker nodes throughout the Worldwide LHC Computing Grid (WLCG). Splitting a large analysis task into optimally sized jobs is critical to efficient use of distributed computing resources. Jobs that are too big will have excessive runtimes and will not distribute the work across all of the available nodes. However, splitting the project into a large number of very small jobs is also inefficient, as each job creates additional overhead which increases load on infrastructure resources. Currently this splitting is done manually, using parameters provided by the user. However the resources needed for each job are difficult to predict because of frequent variations in the performance of the user code and the content of the input dataset. As a result, dividing a task into jobs by hand is difficult and often suboptimal. In this work we present a new feature called “automatic splitting” which removes the need for users to manually specify job splitting parameters. We discuss how HTCondor DAGMan can be used to build dynamic Directed Acyclic Graphs (DAGs) to optimize the performance of large CMS analysis jobs on the Grid. We use DAGMan to dynamically generate interconnected DAGs that estimate the processing time the user code will require to analyze each event. This is used to calculate an estimate of the total processing time per job, and a set of analysis jobs are run using this estimate as a specified time limit. Some jobs may not finish within the alloted time; they are terminated at the time limit, and the unfinished data is regrouped into smaller jobs and resubmitted.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2003

Factors Affecting the Evolution of Hurricane Erin and the Distributions of Hydrometeors: Role of Microphysical Processes

Fine-resolution simulations of Hurricane Erin 2001 are conducted using the Penn State University/National Center for Atmospheric Research mesoscale model version 3.5 to investigate the role of thermodynamic, boundary layer and microphysical processes in Erin's growth and maintenance, and their effects on the horizontal and vertical distributions of hydrometeors. Through comparison against radar, radiometer, and dropsonde data collected during the Convection and Moisture Experiment 4, it is seen that realistic simulations of Erin are obtained provided that fine resolution simulations with detailed representations of physical processes are conducted. The principle findings of the study are as follows: 1) a new iterative condensation scheme, which limits the unphysical increase of equivalent potential temperature associated with most condensation schemes, increases the horizontal size of the hurricane, decreases its maximum rainfall rate, reduces its intensity, and makes its eye more moist; 2) in general, microphysical parameterization schemes with more categories of hydrometeors produce more intense hurricanes, larger hydrometeor mixing ratios, and more intense updrafts and downdrafts; 3) the choice of coefficients describing hydrometeor fall velocities has as big of an impact on the hurricane simulations as does choice of microphysical parameterization scheme with no clear relationship between fall velocity and hurricane intensity; and 4) in order for a tropical cyclone to adequately intensify, an advanced boundary layer scheme (e.g., Burk-Thompson scheme) must be used to represent boundary layer processes. The impacts of varying simulations on the horizontal and vertical distributions of different categories of hydrometeor species, on equivalent potential temperature, and on storm updrafts and downdrafts are examined to determine how the release of latent heat feedbacks upon the structure of Erin. In general, all simulations tend to overpredict precipitation rate and hydrometeor mixing ratios. The ramifications of these findings for quantitative precipitation forecasts (QPFs) of tropical cyclones are discussed.

McFarquhar, Greg M.↗
NASA NTRS1999

Introduction to this Special Issue on Geostatistics and Scaling of Remote Sensing

The germination of this special PE&RS issue began at the Royal Geographical Society (with the Institute of British Geographers)(RCS-IBC) annual meeting in January, 1997 held at the University of Exeter in Exeter, England. The cold and snow of an England winter were greatly tempered by the friendly and cordial discussions that ensued at the meeting on possible ways to foster both dialog and research across "the Big Pond" between geographers in the US and the UK on the use of geostatistics and geospatial techniques for remote sensing of land surface processes. It was decided that one way to stimulate and enhance cooperation on the application of geostatistics and geospatial methods in remote sensing was to hold parallel sessions on these topics at appropriate meeting venues in 1998 in both the US and the UK Selected papers given at these sessions would be published as a special issue of PE&RS on the US side, and as a special issue of Computers and Geosciences (C&G) on the UK side, to highlight the commonality in research on geostatistics and geospatial methods in remote sensing and spatial data analysis on both sides of the Atlantic Ocean. As a consequence, a session on "Ceostatistics and Geospatial Techniques for Remote Sensing of Land Surface Processes" was held at the Association of American Geographers (AAG) annual meeting in Boston, Massachusetts in March, 1998, sponsored by the AAG's Remote Sensing Specialty Group (RSSG). A similar session was held at the RGS-IBG annual meeting in Guildford, Surrey, England in January 1998, organized by the Modeling and Advanced Techniques Special Interest Group (MAT SIG) of the Remote Sensing Society (RSS). The six papers that in part, comprise this issue of PE&RS, are the US complement to such a dual journal publication effort. Both of us are co-editors of each of the journal special issues, with the lead editor of each journal being from their respective side of the Atlantic where the journals are published. The special issue of C&G that constitutes the other half of this co-edited journal series will be published in early 1999, with 3 papers by US authors being published along with 6 papers authored by individuals from the UK and other places in Europe.

Quattrochi, Dale A.↗
DOE OSTI2017

Parallel Multivariate Spatio-Temporal Clustering of Large Ecological Datasets on Hybrid Supercomputers

A proliferation of data from vast networks of remote sensing platforms (satellites, unmanned aircraft systems (UAS), airborne etc.), observational facilities (meteorological, eddy covariance etc.), state-of-the-art sensors, and simulation models offer unprecedented opportunities for scientific discovery. Unsupervised classification is a widely applied data mining approach to derive insights from such data. However, classification of very large data sets is a complex computational problem that requires efficient numerical algorithms and implementations on high performance computing (HPC) platforms. Additionally, increasing power, space, cooling and efficiency requirements has led to the deployment of hybrid supercomputing platforms with complex architectures and memory hierarchies like the Titan system at Oak Ridge National Laboratory. The advent of such accelerated computing architectures offers new challenges and opportunities for big data analytics in general and specifically, large scale cluster analysis in our case. Although there is an existing body of work on parallel cluster analysis, those approaches do not fully meet the needs imposed by the nature and size of our large data sets. Moreover, they had scaling limitations and were mostly limited to traditional distributed memory computing platforms. We present a parallel Multivariate Spatio-Temporal Clustering (MSTC) technique based on k-means cluster analysis that can target hybrid supercomputers like Titan. We developed a hybrid MPI, CUDA and OpenACC implementation that can utilize both CPU and GPU resources on computational nodes. We describe performance results on Titan that demonstrate the scalability and efficacy of our approach in processing large ecological data sets.

Sreepathi, Sarat↗
DOE OSTI2012

An Assessment of Geological Carbon Storage Options in the Illinois Basin: Validation Phase

The Midwest Geological Sequestration Consortium (MGSC) assessed the options for geological carbon dioxide (CO 2 ) storage in the 155,400 km 2 (60,000 mi 2 ) Illinois Basin, which underlies most of Illinois, western Indiana, and western Kentucky. The region has annual CO 2 emissions of about 265 million metric tonnes (292 million tons), primarily from 122 coal-fired electric generation facilities, some of which burn almost 4.5 million tonnes (5 million tons) of coal per year (U.S. Department of Energy, 2010). Validation Phase (Phase II) field tests gathered pilot data to update the Characterization Phase (Phase I) assessment of options for capture, transportation, and storage of CO 2 emissions in three geological sink types: coal seams, oil fields, and saline reservoirs. Four small-scale field tests were conducted to determine the properties of rock units that control injectivity of CO 2 , assess the total storage resources, examine the security of the overlying rock units that act as seals for the reservoirs, and develop ways to control and measure the safety of injection and storage processes. The MGSC designed field test operational plans for pilot sites based on the site screening process, MVA program needs, the selection of equipment related to CO 2 injection, and design of a data acquisition system. Reservoir modeling, computational simulations, and statistical methods assessed and interpreted data gathered from the field tests. Monitoring, Verification, and Accounting (MVA) programs were established to detect leakage of injected CO 2 and ensure public safety. Public outreach and education remained an important part of the project; meetings and presentations informed public and private regional stakeholders of the results and findings. A miscible (liquid) CO 2 flood pilot project was conducted in the Clore Formation sandstone (Mississippian System, Chesterian Series) at Mumford Hills Field in Posey County, southwestern Indiana, and an immiscible CO 2 flood pilot was conducted in the Jackson sandstone (Mississippian System Big Clifty Sandstone Member) at the Sugar Creek Field in Hopkins County, western Kentucky. Up to 12% incremental oil recovery was estimated based on these pilots. A CO 2 huff puff (HNP) pilot project was conducted in the Cypress Sandstone in the Loudon Field. This pilot was designed to measure and record data that could be used to calibrate a reservoir simulation model. A pilot project at the Tanquary Farms site in Wabash County, southeastern Illinois, tested the potential storage of CO 2 in the Springfield Coal Member of the Carbondale Formation (Pennsylvanian System), in order to gauge the potential for large-scale CO 2 storage and/or enhanced coal bed methane recovery from Illinois Basin coal beds. The pilot results from all four sites showed that CO 2 could be injected into the subsurface without adversely affecting groundwater. Additionally, hydrocarbon production was enhanced, giving further evidence that CO 2 storage in oil reservoirs and coal beds offers an economic advantage. Results from the MVA program at each site indicated that injected CO 2 did not leave the injection zone. Topical reports were completed on the Middle and Late Devonian New Albany Shale and Basin CO 2 emissions. The efficacy of the New Albany Shale as a storage sink could be substantial if low injectivity concerns can be alleviated. CO 2 emissions in the Illinois Basin were projected to be dominated by coal-fired power plants.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2023

Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images

Abstract The rise of automation and machine learning (ML) in electron microscopy has the potential to revolutionize materials research through autonomous data collection and processing. A significant challenge lies in developing ML models that rapidly generalize to large data sets under varying experimental conditions. We address this by employing a cycle generative adversarial network (CycleGAN) with a reciprocal space discriminator, which augments simulated data with realistic spatial frequency information. This allows the CycleGAN to generate images nearly indistinguishable from real data and provide labels for ML applications. We showcase our approach by training a fully convolutional network (FCN) to identify single atom defects in a 4.5 million atom data set, collected using automated acquisition in an aberration-corrected scanning transmission electron microscope (STEM). Our method produces adaptable FCNs that can adjust to dynamically changing experimental variables with minimal intervention, marking a crucial step towards fully autonomous harnessing of microscopy big data.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2024

CMOS-Based Single-Cycle in-Memory XOR/XNOR

Big data applications are on the rise, and so is the number of data centers. The ever-increasing massive data pool needs to be periodically backed up in a secure environment. Moreover, a massive amount of securely backed-up data is required for training binary convolutional neural networks for image classification. XOR and XNOR operations are essential for large-scale data copy verification, encryption, and classification algorithms. The disproportionate speed of existing compute and memory units makes the von Neumann architecture inefficient to perform these Boolean operations. Compute-in-memory (CiM) has proved to be an optimum approach for such bulk computations. The existing CiM-based XOR/XNOR techniques either require multiple cycles for computing or add to the complexity of the fabrication process. Here, we propose a CMOS-based hardware topology for single-cycle in-memory XOR/XNOR operations. Our design provides at least 2× improvement in the latency compared with other existing CMOS-compatible solutions. We verify the proposed system through circuit/system-level simulations and evaluate its robustness using a 5000-point Monte Carlo variation analysis. This all-CMOS design paves the way for practical implementation of CiM XOR/XNOR at scaled technology nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling

Abstract The behaviors and skills of models in many geosciences (e.g., hydrology and ecosystem sciences) strongly depend on spatially-varying parameters that need calibration. A well-calibrated model can reasonably propagate information from observations to unobserved variables via model physics, but traditional calibration is highly inefficient and results in non-unique solutions. Here we propose a novel differentiable parameter learning (dPL) framework that efficiently learns a global mapping between inputs (and optionally responses) and parameters. Crucially, dPL exhibits beneficial scaling curves not previously demonstrated to geoscientists: as training data increases, dPL achieves better performance, more physical coherence, and better generalizability (across space and uncalibrated variables), all with orders-of-magnitude lower computational cost. We demonstrate examples that learned from soil moisture and streamflow, where dPL drastically outperformed existing evolutionary and regionalization methods, or required only ~12.5% of the training data to achieve similar performance. The generic scheme promotes the integration of deep learning and process-based models, without mandating reimplementation.

54 ENVIRONMENTAL SCIENCES↗