Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

BioSentinel: Monitoring DNA Damage Repair Beyond Low Earth Orbit on a 6U Nanosatellite

We are designing and developing a 6U nanosatellite as a secondary payload to fly aboard NASAs Space Launch System (SLS) Exploration Mission (EM) 1, scheduled for launch in late 2017. For the first time in over forty years, direct experimental data from biological studies beyond low Earth orbit (LEO) will be obtained during BioSentinels 12- to 18-month mission. BioSentinel will measure the damage and repair of DNA in a biological organism and allow us to compare that to information from onboard physical radiation sensors. This data will be available for validation of existing models and for extrapolation to humans.The BioSentinel experiment will use the organism Saccharomyces cerevisiae (yeast) to report DNA double-strand-break (DSB) events that result from space radiation. DSB repair exhibits striking conservation of repair proteins from yeast to humans. The flight strain will include engineered genetic defects that prevent growth and division until a radiation-induced DSB activates the yeasts DNA repair mechanisms. The triggered culture growth and metabolic activity directly indicate a DSB and its repair. The yeast will be carried in the dry state in independent microwells with support electronics. The measurement subsystem will sequentially activate and monitor wells, optically tracking cell growth and metabolism. BioSentinel will also include TimePix radiation sensors implemented by JSCs RadWorks group. Dose and Linear Energy Transfer (LET) data will be compared directly to the rate of DSB-and-repair events measured by the S. cerevisiae biosentinels. BioSentinel will mature nanosatellite technologies to include: deep space communications and navigation, autonomous attitude control and momentum management, and micropropulsion systems to provide an adaptable nanosatellite platform for deep space uses.

cosmic rays↗

Distributed ADMM Using Private Blockchain for Power Flow Optimization in Distribution Network With Coupled and Mixed-Integer Constraints

The optimization problem for scheduling distributed energy resources (DERs) and battery energy storage systems (BESS) integrated with the power grid is important to minimize energy consumption from conventional sources in response to demand. Conventionally this optimization problem is solved in a centralized manner, limiting the size of the problem that can be solved and creating a high communication overhead because all the data is transferred to the central controller. These limitations are addressed by the proposed distributed consensus-based alternating direction method of multiplier (DC-ADMM) optimization algorithm, which decomposes the optimization problem into subproblems with private cost function and constraints. The distribution feeder is partitioned into low coupling subnetworks/regions, which solves the private subproblem locally and exchanges information with the neighboring regions to reach consensus. The relaxation strategy is employed for mixed-integer and coupled constraints introduced in the optimal power flow (OPF) problem by stationary and transportable BESS because DC-ADMM convergence is only guaranteed for strict convex problems. The information exchange and synchronization between subnetworks/regions are vital for distributed optimization. In this work, both of these aspects are addressed by the blockchain. The smart contract deployed on the blockchain network acts as a mediator for secure data exchange and synchronization in distributed computation. The blockchain-based distributed optimization problem’s effectiveness is tested for a 0.5-MW laboratory microgrid for one hour ahead and day-ahead for the IEEE 123-bus and EPRI J1 test feeders, and results are compared with a centralized solution.

25 ENERGY STORAGE↗

Transfer-Learnt Energy Models for Predicting Electricity Consumption in Buildings with Limited and Sparse Field Data

Modeling energy consumption is critical for energy-efficient utilization of the electric appliances in a building, smart grid programs (like demand-response), and many other smart home applications. State-of-the-art energy modeling techniques either rely on theoretical models, or extensive instrumentation of the building envelope to gather ``big" data to train a deep neural network. While theoretical models are often limited by their estimation accuracy, it is not always feasible to gather a significant amount of field data. In this paper, we explore transfer learning-based strategies to train much more accurate model for energy estimation when using a sparse field data. We transferred knowledge, in the form of data and parameters, from the simulation framework to the field data. We evaluated the efficacy of our approach on field data collected from six commercial buildings and our results indicate that transfer learning-based models trained over one month data can perform comparative (and in some cases better) than the state-of-the-art machine learning and deep learning solutions.

Jain, Milan↗

Real-time Intrusion Detection for High-bandwidth Research Networks using Unsupervised Deep Learning

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks across distributed collaboration sites for processing, storage, and analysis. Like general purpose networks, research networks experience attacks. However, monitoring traffic in such high-bandwidth network environments is challenging with conventional deep packet inspection (DPI)- based network intrusion detection systems (NIDS). Recently, machine learning (ML) algorithms have shown potential in detecting anomalous patterns with better generalizability and faster speed. Compared to the general purpose networks, traffic movement in research network is less volatile and more likely to benefit from the ML-based anomaly analysis. We present ThunderSecure, an efficient AI-powered system for detecting anomalies in 100G research networks. ThunderSecure implements a high-throughput packet processing infrastructure using multi-cores and GPUs. It takes time-series statistics from a variety of network measurements and uses 2D Convolutional Neural Network (CNN) to explore both spatial and temporal dependencies in network data stream. Patterns of science data traffic are learned with a one-class neural network, which blends an Adversarial Autoencoder (AAE) with Gaussian mixture density estimation. Testing traffic flows exhibiting significant deviations from the learned baseline of normality are marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets recorded from a 100G research network at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. To the best of our knowledge, this is the first ML work in the area of network anomaly detection that has been validated on such extreme scale datasets. Results show that ThunderSecure can recognize science data traffic that are captured long after the training and make nearly certain detection on those with anomalous flows injected, even in the case when the anomaly-to-normal mixing ratio is 0.1%.

Gong, Qian↗

Acquisition of and Access to Research Omics Data

Omics data are essential for understanding the myriad and complex effects of space environments on humans. To assure maximum benefit from these kinds of data, the NASA Human Research Program Data Management Plan stipulates that human omics data should be archived within and accessed through the NASA Life Sciences Portal (NLSP). The NLSP has the capability to acquire and provision access to omics (and other kinds of) research results for individual and ad-hoc groups of subjects at the direction of institutional review boards, or other authorizing bodies or individuals, per institutional, program and investigation-specific policies and procedures. However, because some single-subject omics data, like CT scans and other kinds of large, complex biomedical data, could be used to identify heretofore unknown risks to the subject’s health, or, in certain cases, be used to identify a subject, NASA Policy Directive 7170.1 describes various policies regarding the management of and access to “research genetic testing” data, which includes many kinds of omics data. For example, NPD 7170.1 prohibits access to human research genetic data by NASA personnel who make employment decisions for the subjects from whom the data were obtained. To meet the objective of acquiring research omics data for NLSP in compliance with the policies in NPD 7170.1 and other applicable NASA policies, we designed NOMADS (the NLSP Omics Multimodal Acquisition of Data System), a new component that supports the transfer of large research data files, including research genetic testing data, using one of several different transfer mechanisms. The choice of mechanism is made by the submitter of the data, with guiding information from the system, and is likely to often be determined in large part by the nature and source location of the data. For example, for small files where the source data files are not already stored in a cloud storage system, users are likely to prefer to transfer their data to the NLSP via a web browser. Conversely, for large sets of files already organized and stored in a cloud storage system, users may opt for NOMAD’s cloud-to-cloud transfer method. All omics datasets targeted for the NASA Life Sciences Data Archive must pass a variety of quality checks to ensure data integrity and adherence to the standards defined by the LSDA Data Submission Guidelines (DSG) (see https://nlsp.nasa.gov/explore/lsdahome/datasubmit). These include requirements that data are consistent with open standards established by the omics community. Non-compliant data will not be accepted however archivists are available to advise submitters on how to revise data submissions and re-submit until compliance is achieved. Following compliance with the LSDA DSG, omics data next undergo a variety of additional quality checks to ensure the data meet omics community standards. Domain specific Omics data quality control tools and techniques are continually evolving and linked to the advancements in omics assays utilized and thus, the tools and techniques utilized by the LSDA for data quality control and validation will need to be sustained accordingly. All human omics data will be access controlled according to the policies described above, and requiring IRB approval for any additional access grants once the data are acquired (including access for analysis using the NLSP workspace tools).

Omics↗

Acquisition of and Access to Research Omics Data

Omics data are essential for understanding the myriad and complex effects of space environments on humans. To assure maximum benefit from these kinds of data, the NASA Human Research Program Data Management Plan stipulates that human omics data should be archived within and accessed through the NASA Life Sciences Portal (NLSP). The NLSP has the capability to acquire and provision access to omics (and other kinds of) research results for individual and ad-hoc groups of subjects at the direction of institutional review boards, or other authorizing bodies or individuals, per institutional, program and investigation-specific policies and procedures. However, because some single-subject omics data, like CT scans and other kinds of large, complex biomedical data, could be used to identify heretofore unknown risks to the subject’s health, or, in certain cases, be used to identify a subject, NASA Policy Directive 7170.1 describes various policies regarding the management of and access to “research genetic testing” data, which includes many kinds of omics data. For example, NPD 7170.1 prohibits access to human research genetic data by NASA personnel who make employment decisions for the subjects from whom the data were obtained. To meet the objective of acquiring research omics data for NLSP in compliance with the policies in NPD 7170.1 and other applicable NASA policies, we designed NOMADS (the NLSP Omics Multimodal Acquisition of Data System), a new component that supports the transfer of large research data files, including research genetic testing data, using one of several different transfer mechanisms. The choice of mechanism is made by the submitter of the data, with guiding information from the system, and is likely to often be determined in large part by the nature and source location of the data. For example, for small files where the source data files are not already stored in a cloud storage system, users are likely to prefer to transfer their data to the NLSP via a web browser. Conversely, for large sets of files already organized and stored in a cloud storage system, users may opt for NOMAD’s cloud-to-cloud transfer method. All omics datasets targeted for the NASA Life Sciences Data Archive must pass a variety of quality checks to ensure data integrity and adherence to the standards defined by the LSDA Data Submission Guidelines (DSG) (see https://nlsp.nasa.gov/explore/lsdahome/datasubmit). These include requirements that data are consistent with open standards established by the omics community. Non-compliant data will not be accepted however archivists are available to advise submitters on how to revise data submissions and re-submit until compliance is achieved. Following compliance with the LSDA DSG, omics data next undergo a variety of additional quality checks to ensure the data meet omics community standards. Domain specific Omics data quality control tools and techniques are continually evolving and linked to the advancements in omics assays utilized and thus, the tools and techniques utilized by the LSDA for data quality control and validation will need to be sustained accordingly. All human omics data will be access controlled according to the policies described above, and requiring IRB approval for any additional access grants once the data are acquired (including access for analysis using the NLSP workspace tools).

Omics↗

CCD data acquisition systems at Lick and Keck Observatories

This paper will describe and compare two distinct but related CCD data acquisition systems (DAS) currently under development at Lick and Keck Observatories. Although these two systems have a number of major architectural differences, they share a considerable amount of common hardware and software. Both of these new systems build on a large body of proven software that is the foundation of the existing CCD DAS currently in use at Lick Observatory. Both will provide support for reading up to four on-chip amplifiers per CCD and/or reading out mosaics of CCD chips. In addition, they will provide the capability for interactive, real-time adjustment of CCD waveforms for engineering purposes. Each of these two systems is composed of three major subsystems: (1) an instrument computer and its software; (2) a data capture computer and its software; and (3) a CCD/dewar controller and its software. The instrument computer is a Unix workstation, and the functions it provides include user interfaces, the interactive real-time display of CCD images, and the recording of image and FITS header data to disk and/or tape. The data capture computer is responsible for the packaging and high-speed transfer of the CCD pixel data stream into a bulk RAM, and the subsequent transfer of this data to the instrument computer. The CCD/dewar controller generates the waveforms for clocking the CCD, digitizes the pixel data, and transmits it via high-speed link to the data capture computer. It is also responsible for monitoring and controlling the dewar temperature and cryogen levels. Given the number of different types of processors and high-speed data links employed in both systems, a major emphasis of this paper will be on the various forms of interprocessor communications utilized for data transfer and distributed process synchronization.

Kibrick, R. I.↗

A Novel Technique for Distributed Measuring of Strain in the Vadose Zone

Characterizing water content and pressure changes in the vadose zone is important to understanding a variety of geologic processes, ranging from permeability, to evapotranspiration, and aquifer recharge. Changes in water content, or pressure, cause strain in the solid porous medium of the vadose zone, and it is possible that measuring those strains could be used a characterization tool. The Coherence-length-gated Microwave Photonics Interferometry (CMPI) technique measures strain at high resolution along many intervals defined by pairs of reflectors distributed along an optical fiber. This technique has recently been developed at Clemson University, and it has the spatial, and temporal resolution to characterize the strains that are expected to occur with hydrologic changes in the vadose zone. However, the technique has never been used to measure strain in porous media, so its capabilities remain uncertain. The objective of this thesis is to evaluate the ability of using CMPI to measure strain changes in the vadose zone. The research approach consists of conducting laboratory tests using a column filled with sand that was subjected to changes in water content and pressure. An optical fiber with CMPI reflectors was deployed in a high surface area ribbon and used to measure strain along the axis of the column. The column was made from 8-inch, Schedule 40 PVC pipe (75 cm tall, 20 cm inner diameter) and filled with medium-grained sand (K = 2.5x10-6 m/s, porosity = 0.28, Coefficient of Uniformity = 2, van Genuchten (n = 4.34 and  = 0.00039 m-1), Young’s Modulus = 18 – 47 MPa). Five pressure and four temperature sensors attached to probes inserted into the wall. The optical fiber includes five reflectors spaced 10 cm apart on the inside and outside of the column along the vertical axis. Strain measurements are made between pairs of reflectors. The reflectors are created in 250-micron-diameter acrylate-coated single mode Corning SMF28e+ optical fiber using a femtosecond laser. The optical fiber was laminated between two pieces of polyester film creating a large surface area to transfer strain from the porous media to the optical fiber. The experiments were conducted by filling the column from the bottom or infiltrating water from the top. The water was allowed to equilibrate to room temperature prior to each test in order to limit thermoelastic strain. Five injection tests with a rate of 250 ml/min and three infiltration tests at varying rates were conducted, and the results show patterns of strain and pressure are generally similar. Hydrologic conditions define three zones based on the pressure magnitude and distribution. 1.) Ambient Zone where the pressure heads are quasi-static and the pressure gradient is roughly unity (head gradient of zero). This is the uppermost zone and is characterized by negative pressures. 2.) Transition Zone where the pressures increase from ambient to zero, are changing relatively rapidly and the pressure gradient is relatively steep (pressure head gradients of 2). The Transition zone is 10 to 15 cm thick. 3.) Positive Pressure Zone where the pressure is positive, the rate of change is slower than in the transition zone and the gradient is flatter (pressure head gradient 1.1 to 1.2). This is the lowest zone in the column. Injection of water causes the pressure to increase and the three zones to move upward. The Transition zone moves at a rate of approximately 0.0002 m/s +/- .00005, according to analyses of pressure profiles. This is the same as the average velocity of the water calculated as volumetric flux/effective porosity =5.4x10-5 m/s/0.28. Strain signals in the range of 10s  were observed with a noise level of generally less than 0.1  (signal to noise ratio of greater than 100) during injection and drainage. The spatial and temporal distributions of strain caused by injection depend on the location of the moving pressure zones. The locations of the different pressure zones were determined from pressure profiles at different times and these data were transferred to strain time series. This showed that strain in the Ambient pressure zone is either unchanged or slightly compressive, whereas strain in Transition zone is tensile and roughly proportional to the pressure change, and strain ranges from tensile to compressive in the Positive Pressure zone. In general, the strain is variable at the top of the Positive Pressure zone, but it appears to be consistently tensile lower in the zone. The spatial distribution of strain causes three distinct stages in the strain times series measured between pairs of CMPI reflectors. The strain is unchanged or slightly compressive during Stage 1, it increases (positive strain is tensile) approximately linearly at a rate of 0.02 to 0.03 μ/s during the Stage 2, and then flattens out in Stage 3. The strain increases again during Stage 4 of the time series. The pressure also changes in stages. It is unchanged during Stage 1 and then increases relatively rapidly at a rate of between 4 and 10 Pa/s during Stage 2 and continues to increase during Stages 3 and 4, but at a rate slightly slower than during Stage 2 (from 2.5 to 3.5 Pa/s). The rates of pressure and strain change are consistent with basic analyses. The rate of pressurization is similar to the calculated velocity of flow in the saturated zone, and this is similar to the measured velocity of the strain increase. The ratio of the pressurization rate and strain rate during Stage 2 (2.5 (Pa/s) /0.03 (μ/s)) is 80 MPa, which is approximately equal to the upper range expected considering the uniaxial compression and the Young’s Modulus of the sand. An interesting effect occurs when the upper surface of the sand becomes saturated. Significant compression (several 10s of ) occurs as the pressure and saturation increase at the upper surface. Compression occurs throughout the column, but the effect is greatest at the top of the column. The rate of compression is fastest slightly before ponding occurs, but it slows markedly and nearly stops when water starts to accumulate at the surface (ponding). This effect reverses when the surface of the sand is drained, resulting in tension throughout the column. The compression caused by this effect can be as large or larger than the tensile strain that accumulated during filling. This effect was unexpected because increasing pressure is normally associated with tensile strain. Nevertheless, this effect was observed consistently in all tests where the pressure changes at the upper surface, including tests where water was injected from below or infiltrated from above. This effect behaves as if the pore pressure at the upper surface of the sand exerts a normal force on the boundary (increasing pore pressure exerts a downward compression on the boundary). The results of these laboratory experiments indicate that CMPI can measure strain caused by fluid pressure changes in the vadose zone with a signal to noise ratio of 100 or more. Injection and drainage cause a strain signal that is complex, but repeatable. The magnitudes of the strain signal are consistent with magnitudes that are expected based on poroelastic calculations using independently measured properties of the sand. These results indicate that the CMPI technique with an optical fiber laminated in a polyester ribbon generates data that represent the strain distribution during hydrologic processes of imbibition and drainage in variably saturated sand. This suggests that distributed strain measurements using CMPI could be a viable approach for evaluating processes in the vadose zone, laying the groundwork for future field implementation.

47 OTHER INSTRUMENTATION↗

Federated Learning with Frequency Estimation for Smart Meter Systems

Federated learning (FL) is a powerful framework that enables multiple distributed clients to collaborate without the need to transfer their data to a central server. However, FL does not inherently guarantee the level of privacy that clients often require. In our review of recent studies on privacy-enhancing techniques in FL, we found that frequency estimation (FE) methods remain underexplored. To address this gap, we developed and integrated FE techniques on the client side, further examining the effects of incorporating an adaptive range and a shuffled model. We also analyzed the impact of varying hyper-parameters on privacy preservation. Our results provide clear guidance on the algorithms and configurations that are most effective for enhancing privacy in FL, particularly when using long short-term memory (LSTM) architectures.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗

First STAX detector installation at the National Institute for Radioelements (IRE)

The Source Term Analysis of Xenon (STAX) project has been installing stack detectors at medical isotope production facilities to measure radioxenon emissions to investigate the effect of radioxenon releases on nuclear explosion monitoring. This paper outlines the installation of the first STAX detection system at the National Institute for Radioelements (IRE) in Fleurus, Belgium which has been operating for over three years and transferring collected data to the STAX repository. Information about the equipment installed, the data flow established, and calculations for determination of radioxenon releases from the facility are presented. Further. data quality was investigated to confirm values reported by STAX automated data processing and in a comparison of collected STAX data with data collected by IRE for regulatory reporting.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Real-Time Monitoring of Gas-Phase and Dissolved CO 2 Using a Mixed-Matrix Composite Integrated Fiber Optic Sensor for Carbon Storage Application

Novel chemical sensors that improve detection and quantification of CO 2 are critical to ensuring safe and cost-effective monitoring of carbon storage sites. Fiber optic (FO) based chemical sensor systems are promising field-deployable systems for real-time monitoring of CO 2 in geological formations for long-range distributed sensing. Here, a mixed-matrix composite integrated FO sensor system was developed with a purely optical readout that reliably operates as a detector for gas-phase and dissolved CO 2 . A mixed-matrix composite sensor coating consisting of plasmonic nanocrystals and hydrophobic zeolite embedded in a polymer matrix was integrated on the FO sensor. The mixed-matrix composite FO sensor showed excellent reversibility/stability in a high humidity environment and sensitivity to gas-phase CO 2 over a large concentration range. This remarkable sensing performance was enabled by using plasmonic nanocrystals to significantly enhance the sensitivity and a hydrophobic zeolite to effectively mitigate interference from water vapor. The sensor exhibited the ability to sense CO 2 in the presence of other geologically relevant gases, which is of importance for applications in geological formations. A prototype FO sensor configuration which possesses a robust sensing capability for monitoring dissolved CO 2 in natural water was demonstrated. Reproducibility was confirmed over many cycles, both in a laboratory setting and in the field. More importantly, we demonstrated on-line monitoring capabilities with a wireless telemetry system, which transferred the data from the field to a website. The combination of outstanding CO 2 sensing properties and facile coating processability makes this mixed-matrix composite FO sensor a good candidate suitable for practical carbon storage applications.

54 ENVIRONMENTAL SCIENCES↗

Exploring Structure-Sensitive Relations for Small Species Adsorption Using Machine Learning

Accurate prediction of adsorption energies on heterogeneous catalyst surfaces is crucial to predicting reactivity and screening materials. Adsorption linear scaling relations have been developed extensively but often lack accuracy and apply to one adsorbate and a single binding site type at a time. These facts undermine their ability to predict structure sensitivity and optimal catalyst structure. Using machine learning on nearly 300 density functional theory calculations, we demonstrate that generalized coordination number scaling relations hold well for oxygen- and high-valency carbon-binding species but fail for others. Here we reveal that the valency and the electronic coupling of a species with the surface, along with the site type and its coordination environment, are critical for small species adsorption. The model simultaneously predicts the adsorption energy and preferred site and significantly outperforms linear scalings in accuracy. It can expose the structure sensitivity of chemical reactions and enable enhanced catalyst activity via engineering particle shape and facet defects. The generality of our methodology is validated by training the model with transition metal data and transferring it to predict adsorption energies on single-atom alloys.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deep learning-based predictive models for laser direct drive at the Omega Laser Facility

The rich and complex physics of inertial confinement fusion provides a unique and challenging space for high-fidelity first-principles modeling. Consequently, simulation codes that are used to design experiments are computationally expensive and lack the predictive capability required for extensive parameter exploration in search of a high-performing design for laser direct drive. In this article, we present two deep-learning-based predictive models intended to address these difficulties. The first model (TL DNN) acts as a fast emulator of simulations as well as experiments at the Omega Laser Facility. This model is trained on a simulation database and subsequently calibrated on experimental data using transfer learning. To facilitate the development of this model, an autoencoder is developed to reduce the dimensionality of the input space by compressing the laser pulse input. The model predicts key experimental scalar observables of Omega experiments with high accuracy and minimal computational cost. This deep neural net enables rapid exploration of a high-dimensional input parameter space for an optimal implosion design. The second model (DNN SM+) aims to extend the statistical modeling work of Lees et al. [Phys. Rev. Lett. 127, 105001 (2021)], by increasing the complexity of the model space and allowing for coupling between degradation terms. Since the model capacity of DNN SM+ is higher than the model of Lees et al., DNN SM+ can potentially provide an improvement in predictive capability, and we use this model to provide insight into complicated degradation dependencies.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Regression Analysis with the Directed Infusion of Data

Integrating artificial intelligence and machine learning tools into industry necessitates large-scale collaborative efforts that ensure the robust and accurate execution of downstream analytics such as time series prediction, uncertainty quantification, grid optimization, and condition monitoring. However, concerns related to data privacy pervade the nuclear industry due to the proprietary nature of its data and the possibility of data leakage. Legacy techniques such as encryption often require the explicit transmission of data to trustworthy parties, thereby inviting data leakage concerns. The ideal collaboration scenario avoids the explicit dissemination of data/code while maintaining experimental fidelity, which is currently accomplished using various techniques such as trusted execution environments, homomorphic encryption, differential privacy, and multimatrix masking. These techniques, however, often necessitate a trade-off between trust, efficiency, and utility. This article extends a previously proposed technique called the directed infusion of data (DIOD) that ensures data privacy, allows for scalable obfuscation, and combats the risk of data leakage without compromising utility. The experiments discussed in this article examine a regression-type scenario using DIOD with the goal of preserving the inferential link between two variables. Using the point-kinetics equations, regression experiments compare the performance of a model trained using the original data to that of a model trained using the obfuscated data, which produced identical results. Our claim is further strengthened by an information theoretic proof and experiment, which showed that the inferential content between variables remains the same after obfuscation, thereby avoiding the required communication of the proprietary data.

47 - OTHER INSTRUMENTATION↗

The Artificial Scientist: in-Transit Machine Learning of Plasma Simulations

Large-scale simulations or scientific experiments produce petabytes of data per run. This poses massive challenges for I/O and storage when scientific analysis workflows are run manually offline. Unsupervised deep learning-based techniques to extract patterns and non-linear relations from these large amounts of data provide a way to build scientific understanding from raw data, reducing the need for manual pre-selection of analysis steps, but require exascale compute and memory to process the full dataset available. In this paper, we demonstrate a heterogeneous streaming workflow in which plasma simulation data is streamed directly to a Machine Learning (ML) application training a model on the simulation data in-transit, completely circumventing the capacity-constrained filesystem bottleneck. This workflow employs openPMD to provide a high level interface to describe scientific data and also uses ADIOS2, to transfer volumes of data that exceed the capabilities of the filesystem. We employ experience replay to avoid catastrophic forgetting in learning from this non-steady state process in a continual manner and adapt it to improve model convergence while learning in-transit. As a proof-of-concept, we approach the ill-posed inverse problem of predicting particle dynamics from radiation in a particle-incell (PIConGPU) simulation of the Kelvin-Helmholtz instability (KHI). We detail hardware-software co-design challenges as we scale PIConGPU to full Frontier, the Top-1 system as of June 2024 Top500 list.

Kelling, Jeffrey [Helmholtz-Zentrum Dresden Rossen↗