Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data linking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59

A Novel Sparse Compositional Technique Reveals Microbial Perturbations

ABSTRACT The central aims of many host or environmental microbiome studies are to elucidate factors associated with microbial community compositions and to relate microbial features to outcomes. However, these aims are often complicated by difficulties stemming from high-dimensionality, non-normality, sparsity, and the compositional nature of microbiome data sets. A key tool in microbiome analysis is beta diversity, defined by the distances between microbial samples. Many different distance metrics have been proposed, all with varying discriminatory power on data with differing characteristics. Here, we propose a compositional beta diversity metric rooted in a centered log-ratio transformation and matrix completion called robust Aitchison PCA. We demonstrate the benefits of compositional transformations upstream of beta diversity calculations through simulations. Additionally, we demonstrate improved effect size, classification accuracy, and robustness to sequencing depth over the current methods on several decreased sample subsets of real microbiome data sets. Finally, we highlight the ability of this new beta diversity metric to retain the feature loadings linked to sample ordinations revealing salient intercommunity niche feature importance. IMPORTANCE By accounting for the sparse compositional nature of microbiome data sets, robust Aitchison PCA can yield high discriminatory power and salient feature ranking between microbial niches. The software to perform this analysis is available under an open-source license and can be obtained at https://github.com/biocore/DEICODE ; additionally, a QIIME 2 plugin is provided to perform this analysis at https://library.qiime2.org/plugins/q2-deicode .

59 BASIC BIOLOGICAL SCIENCES↗

Reducing uncertainty of high-latitude ecosystem models through identification of key parameters

Abstract Climate change is having significant impacts on Earth’s ecosystems and carbon budgets, and in the Arctic may drive a shift from an historic carbon sink to a source. Large uncertainties in terrestrial biosphere models (TBMs) used to forecast Arctic changes demonstrate the challenges of determining the timing and extent of this possible switch. This spread in model predictions can limit the ability of TBMs to guide management and policy decisions. One of the most influential sources of model uncertainty is model parameterization. Parameter uncertainty results in part from a mismatch between available data in databases and model needs. We identify that mismatch for three TBMs, DVM-DOS-TEM, SIPNET and ED2, and four databases with information on Arctic and boreal above- and belowground traits that may be applied to model parametrization. However, focusing solely on such data gaps can introduce biases towards simple models and ignores structural model uncertainty, another main source for model uncertainty. Therefore, we develop a causal loop diagram (CLD) of the Arctic and boreal ecosystem that includes unquantified, and thus unmodeled, processes. We map model parameters to processes in the CLD and assess parameter vulnerability via the internal network structure. One important substructure, feed forward loops (FFLs), describe processes that are linked both directly and indirectly. When the model parameters are data-informed, these indirect processes might be implicitly included in the model, but if not, they have the potential to introduce significant model uncertainty. We find that the parameters describing the impact of local temperature on microbial activity are associated with a particularly high number of FFLs but are not constrained well by existing data. By employing ecological models of varying complexity, databases, and network methods, we identify the key parameters responsible for limited model accuracy. They should be prioritized for future data sampling to reduce model uncertainty.

54 ENVIRONMENTAL SCIENCES↗

UAE6 - Wind Tunnel Tests Data - UAE6 - Sequence S - Raw Data

Sequences S, T, and U: Upwind, No Probes (F); Upwind 2° Pitch (F); Upwind 4° Pitch (F) This test sequence used an upwind, rigid turbine with a 0° cone angle. The wind speed ranged from 5 m/s to 25 m/s. Yaw angles of 0° to 180° were achieved for Sequence S, but the yaw angle remained at 0° for Sequences T and U. The blade tip pitch was 3° for Sequence S, 2° for Sequence T, and 4° for Sequence U. These three sequences were interleaved during testing because the pitch angle change was easily made by the turbine operator. The rotor rotated at 72 RPM. Blade pressure measurements were collected. The five-hole probes were removed and the plugs were installed. Plastic tape 0.03-mm-thick was used to smooth the interface between the plugs and the blade. The teeter dampers were replaced with rigid links, and these two channels were flagged as not applicable by setting the measured values in the data file to -99999.99 Nm. The teeter link load cell was pre-tensioned to 40,000 N. During post-processing, the probe channels were set to read -99999.99. In addition to the standard 30-second campaigns, yaw sweeps were done at 7 m/s and 10 m/s for the Sequence S configuration. These 6-minute campaigns were collected while the yaw drive rotated the turbine 360° at a rate of 1°/s. The file names for these campaigns use the letter designation, followed by two digits for wind speed, followed by YSU, followed by 00.

17 WIND ENERGY↗

UAE6 - Wind Tunnel Tests Data - UAE6 - Sequence T - Raw Data

Sequences S, T, and U: Upwind, No Probes (F); Upwind 2° Pitch (F); Upwind 4° Pitch (F) This test sequence used an upwind, rigid turbine with a 0° cone angle. The wind speed ranged from 5 m/s to 25 m/s. Yaw angles of 0° to 180° were achieved for Sequence S, but the yaw angle remained at 0° for Sequences T and U. The blade tip pitch was 3° for Sequence S, 2° for Sequence T, and 4° for Sequence U. These three sequences were interleaved during testing because the pitch angle change was easily made by the turbine operator. The rotor rotated at 72 RPM. Blade pressure measurements were collected. The five-hole probes were removed and the plugs were installed. Plastic tape 0.03-mm-thick was used to smooth the interface between the plugs and the blade. The teeter dampers were replaced with rigid links, and these two channels were flagged as not applicable by setting the measured values in the data file to -99999.99 Nm. The teeter link load cell was pre-tensioned to 40,000 N. During post-processing, the probe channels were set to read -99999.99. In addition to the standard 30-second campaigns, yaw sweeps were done at 7 m/s and 10 m/s for the Sequence S configuration. These 6-minute campaigns were collected while the yaw drive rotated the turbine 360° at a rate of 1°/s. The file names for these campaigns use the letter designation, followed by two digits for wind speed, followed by YSU, followed by 00.

17 WIND ENERGY↗

UAE6 - Wind Tunnel Tests Data - UAE6 - Sequence U - Raw Data

Sequences S, T, and U: Upwind, No Probes (F); Upwind 2° Pitch (F); Upwind 4° Pitch (F) This test sequence used an upwind, rigid turbine with a 0° cone angle. The wind speed ranged from 5 m/s to 25 m/s. Yaw angles of 0° to 180° were achieved for Sequence S, but the yaw angle remained at 0° for Sequences T and U. The blade tip pitch was 3° for Sequence S, 2° for Sequence T, and 4° for Sequence U. These three sequences were interleaved during testing because the pitch angle change was easily made by the turbine operator. The rotor rotated at 72 RPM. Blade pressure measurements were collected. The five-hole probes were removed and the plugs were installed. Plastic tape 0.03-mm-thick was used to smooth the interface between the plugs and the blade. The teeter dampers were replaced with rigid links, and these two channels were flagged as not applicable by setting the measured values in the data file to -99999.99 Nm. The teeter link load cell was pre-tensioned to 40,000 N. During post-processing, the probe channels were set to read -99999.99. In addition to the standard 30-second campaigns, yaw sweeps were done at 7 m/s and 10 m/s for the Sequence S configuration. These 6-minute campaigns were collected while the yaw drive rotated the turbine 360° at a rate of 1°/s. The file names for these campaigns use the letter designation, followed by two digits for wind speed, followed by YSU, followed by 00.

17 WIND ENERGY↗

Production and tests of the ADDC for the Micromegas detector of the ATLAS New Small Wheel

The Large Hadron Collider (LHC) at CERN will perform a series of upgrades to allow luminosity increases during the next physics runs expected from 2022 on. These will also significantly increase the trigger rates of all the detectors. As part of the ATLAS Phase-I upgrade, the current Small Wheel muon detectors will be replaced with state-of-the-art New Small Wheel (NSW) detectors to cope with the increased luminosity of the LHC. The Address in Real Time Data Driver Card (ADDC) is designed to transmit the trigger data from the resistive Micro-Mesh Gaseous Structure (Micromegas) detectors of the NSW. The ART ASIC on the ADDC is a customdesigned chip to receive theAddress in Real Time (ART) signals from theVMMfront-end ASIC and perform hit-selection processing. The processed trigger data is then sent to the Trigger Processor (TP) through optical links. A total of 512 ADDCs will be installed on the detector close to the front-end boards. Therefore, those cards must be able to work properly in a high radiation and magnetic field environment. Furthermore, after four rounds of prototyping, the ADDC production was launched in January of 2019. 600 ADDC boards have been produced and tested with an automated test stand. This test setup can simulate the front-end signals and provide "Level-1 Data Driver Card" (L1DDC) functions as well as part of the TP functions. Thus, the ADDC functionality and stability can be verified without the remaining NSW electronics. This paper describes the ADDC hardware prototype development, radiation and integration testing, and the ADDC automated production test procedure.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Acoustic Energy Release During the Laboratory Seismic Cycle: Insights on Laboratory Earthquake Precursors and Prediction

Abstract Machine learning can predict the timing and magnitude of laboratory earthquakes using statistics of acoustic emissions. The evolution of acoustic energy is critical for lab earthquake prediction; however, the connections between acoustic energy and fault zone processes leading to failure are poorly understood. Here, we document in detail the temporal evolution of acoustic energy during the laboratory seismic cycle. We report on friction experiments for a range of shearing velocities, normal stresses, and granular particle sizes. Acoustic emission data are recorded continuously throughout shear using broadband piezo‐ceramic sensors. The coseismic acoustic energy release scales directly with stress drop and is consistent with concepts of frictional contact mechanics and time‐dependent fault healing. Experiments conducted with larger grains (10.5 μm) show that the temporal evolution of acoustic energy scales directly with fault slip rate. In particular, the acoustic energy is low when the fault is locked and increases to a maximum during coseismic failure. Data from traditional slide‐hold‐slide friction tests confirm that acoustic energy release is closely linked to fault slip rate. Furthermore, variations in the true contact area of fault zone particles play a key role in the generation of acoustic energy. Our data show that acoustic radiation is related primarily to breaking/sliding of frictional contact junctions, which suggests that machine learning‐based laboratory earthquake prediction derives from frictional weakening processes that begin very early in the seismic cycle and well before macroscopic failure.

58 GEOSCIENCES↗

In Situ Inference for Earth System Predictability

An understanding of future evolution in precipitation extremes is critical to numerous DOE mission questions. Extreme events are by nature short time-scale events that are difficult to diagnose in available model data. Accurate modeling of extreme events necessarily requires high spatial resolution at the storm scale locally. However, the environment in which storms grow is dependent on global, remote, processes. These complex spatiotemporal relationships are impossible to diagnose at resolutions required to accurately model storms responsible for extreme precipitation. At exascale, climate simulations will produce results at fine enough resolution to investigate these relationships. However, the resulting data from these simulations will be far too large to save for post-simulation analysis. We advocate for fitting statistical models inside the simulations as they run, a context known as in situ, which will facilitate scientific investigations using the full fine-scale data stream. Figure 1 shows an example of the type of model we could consider, a Bayesian hierarchical spatial regression model. Precipitation extremes at each grid cell are modeled using extreme value distributions. Since extremes are rare, fitting models to individual grid cells can result in high variance and poor estimates. Instead, the model can be made more robust by smoothing the parameters of the extreme value model across space. Additionally, the parameters themselves can be functionally linked to other variables elsewhere in the simulation. Thus, we can use the fine-scale data to build more robust models for extremes that link extreme behavior to other climate patterns.

54 ENVIRONMENTAL SCIENCES↗

In Situ Inference for Earth System Predictability

Focal Area: Focal Area 3: Insight gleaned from complex simulated data using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI. Science Challenge: An understanding of future evolution in precipitation extremes is critical to numerous DOE mission questions. Extreme events are by nature short time-scale events that are difficult to diagnose in available model data. Accurate modeling of extreme events necessarily requires high spatial resolution at the storm scale locally. However, the environment in which storms grow is dependent on global, remote, processes. These complex spatiotemporal relationships are impossible to diagnose at resolutions required to accurately model storms responsible for extreme precipitation. At exascale, climate simulations will produce results at fine enough resolution to investigate these relationships. However, the resulting data from these simulations will be far too large to save for post-simulation analysis. We advocate for fitting statistical models inside the simulations as they run, a context known as in situ, which will facilitate scientific investigations using the full fine-scale data stream. Figure 1 shows an example of the type of model we could consider, a Bayesian hierarchical spatial regression model. Precipitation extremes at each grid cell are modeled using extreme value distributions. Since extremes are rare, fitting models to individual grid cells can result in high variance and poor estimates. Instead, the model can be made more robust by smoothing the parameters of the extreme value model across space. Additionally, the parameters themselves can be functionally linked to other variables elsewhere in the simulation. Thus, we can use the fine-scale data to build more robust models for extremes that link extreme behavior to other climate patterns.

54 ENVIRONMENTAL SCIENCES↗

Knowledge catalysts

A computer implemented method integrates data from remote disparate data sources by processing a non-transitory media. The non-transitory media stores instructions for detecting data sets in different formats hosted in a plurality of heterogeneous databases that are accessible through a distributed network. The method extracts schema data from the plurality of heterogeneous databases and identifies related fields in two or more of the heterogeneous databases. The method links the related fields in the two or more of the plurality of heterogeneous databases and makes the data accessible through a virtual warehouse. As schemas change, as new data sources and analysis artifacts are created, the computer implemented method and system can act as a meta-data store, a provenance tracking device, and/or a knowledge management service.

97 MATHEMATICS AND COMPUTING↗

A GATE evaluation of the sources of error in quantitative {sup 90}Y PET

Purpose: Accurate reconstruction of the dose delivered by {sup 90}Y microspheres using a postembolization PET scan would permit the establishment of more accurate dose–response relationships for treatment of hepatocellular carcinoma with {sup 90}Y. However, the quality of the PET data obtained is compromised by several factors, including poor count statistics and a very high random fraction. This work uses Monte Carlo simulations to investigate what impact factors other than low count statistics have on the quantification of {sup 90}Y PET. Methods: PET acquisitions of two phantoms—a NEMA PET phantom and the NEMA IEC PET body phantom-containing either {sup 90}Y or {sup 18}F were simulated using GATE. Simulated projections were created with subsets of the simulation data allowing the contributions of random, scatter, and LSO background to be independently evaluated. The simulated projections were reconstructed using the commercial software for the simulated scanner, and the quantitative accuracy of the reconstruction and the contrast recovery of the reconstructed images were evaluated. Results: The quantitative accuracy of the {sup 90}Y reconstructions were not strongly influenced by the high random fraction present in the projection data, and the activity concentration was recovered to within 5% of the known value. The contrast recovery measured for simulated {sup 90}Y data was slightly poorer than that for simulated {sup 18}F data with similar count statistics. However, the degradation was not strongly linked to any particular factor. Using a more restricted energy range to reduce the random fraction in the projections had no significant effect. Conclusions: Simulations of {sup 90}Y PET confirm that quantitative {sup 90}Y is achievable with the same approach as that used for {sup 18}F, and that there is likely very little margin for improvement by attempting to model aspects unique to {sup 90}Y, such as the much higher random fraction or the presence of bremsstrahlung in the singles data.

61 RADIATION PROTECTION AND DOSIMETRY↗

Wireless link and microelectronics design for retinal prostheses

This project focuses on delivering power and data to the artificial retinal implant inside the eye and the implant microstimulator electronics which delivers the current pulses to stimulate the retinal layer to elicit visual perception. Since the use of invasive means such as tethering wires to transmit power and data results in discomfort to the patients which could eventually cause infection due to the abrasion caused by the wire and contact of the internals of the eye to the external environment, a completely wireless approach is used to transfer both power and data. Power is required inside the eye for the microelectronic implant which uses a dual voltage supply scheme (positive and negative) to deliver biphasic (anodic and cathodic) current pulses. Data in the form of digital bits from the data transmitter external to the eye, carries information about the amplitude, phase width, interphase delay, stimulation sequence for each implant electrode. The data receiver unit decodes the digital stream and the microstimulator unit generates the appropriate current stimuli. Since the external unit consisting of the power transmitter can experience coupling a variation with the power receiver due to the patient’s movements, a closed loop approach is used which varies the transmitted power dynamically to automatically compensate for such movements. This report presents the salient features of this research activities and results.

42 ENGINEERING↗

A fussy revisitation of antiprotons as a tool for Dark Matter searches

Antiprotons are regarded as a powerful probe for Dark Matter (DM) indirect detection and indeed current data from PAMELA have been shown to lead to stringent constraints. However, in order to exploit their constraining/discovery power properly and especially in anticipation of the exquisite accuracy of upcoming data from AMS-02, great attention must be put into effects (linked to their propagation in the Galaxy) which may be perceived as subleading but actually prove to be quite relevant. Using a semi-analytic code for rapidity, we revisit the computation of the astrophysical background and of the DM antiproton fluxes. Like in the fully numerical standard calculations, we include the effects of: diffusive reacceleration, energy losses including tertiary component and solar modulation (in a force field approximation). We show that their inclusion can somewhat modify the current bounds, even at large DM masses, and that a wrong interpretation of the data may arise if they are not taken into account. At the present level of accuracy of the data from PAMELA, the inclusion of the above effects amounts to changing the constraints, with respect to the case in which they are neglected, of up to about 40% at a DM mass of 1 TeV and 30% at 10 TeV . When the AMS-02 level of precision is reached, including them would strengthen (lessen) the bounds on the annihilation cross section by up to a factor of 15 below (above) a DM mass of 300 GeV. The numerical results for the astrophysical background are provided in terms of fit functions; the results for Dark Matter are incorporated in the new release of the PPPC4DMID.

79 ASTRONOMY AND ASTROPHYSICS↗

Exploring the link between C iv outflow kinematics and sublimation-temperature dust in quasars

ABSTRACT Using data from SDSS, UKIDSS, and WISE, we investigate the properties of the high-frequency cutoff to the infrared emission in ≃5000 carefully selected luminous (Lbol ∼ 1047) type 1 quasars. The strength of ≃2 μm emission, corresponding to emission from the hottest ($T\gt 1200\rm \, K$) dust in the sublimation zone surrounding the central continuum source, is observed to correlate with the blueshift of the C iv λ1550 emission line. We therefore find that objects with stronger signatures of nuclear outflows tend to have a larger covering fraction of sublimation-temperature dust. When controlling for the observed outflow strength, the hot dust covering fraction does not vary significantly across our sample as a function of luminosity, black hole mass, or Eddington fraction. The correlation between the hot dust and the C iv line blueshifts, together with the lack of correlation between the hot dust and other parameters, therefore provides evidence of a link between the properties of the broad emission line region and the infrared-emitting dusty regions in quasars.

Temple, Matthew J.↗

Phases and interfaces from real space atomically resolved data: Physics-based deep data image analysis

Advances in electron and scanning probe microscopies have led to a wealth of atomically resolved structural and electronic data, often with ~1–10 pm precision. However, knowledge generation from such data requires the development of a physics-based robust framework to link the observed structures to macroscopic chemical and physical descriptors, including single phase regions, order parameter fields, interfaces, and structural and topological defects. Here, we develop an approach based on a synergy of sliding window Fourier transform to capture the local analog of traditional structure factors combined with blind linear unmixing of the resultant 4D data set. This deep data analysis is ideally matched to the underlying physics of the problem and allows reconstruction of the a priori unknown structure factors of individual components and their spatial localization. We demonstrate the principles of this approach using a synthetic data set and further apply it for extracting chemical and physically relevant information from electron and scanning tunneling microscopy data. Furthermore, this method promises to dramatically speed up crystallographic analysis in atomically resolved data, paving the road toward automatic local structure–property determinations in crystalline and quasi-ordered systems, as well as systems with competing structural and electronic order parameters.

36 MATERIALS SCIENCE↗

WHONDRS Summer 2019 Sampling Campaign: Global River Corridor Surface Water FTICR-MS, NPOC, TN, Anions, Stable Isotopes, Bacterial Abundance, and Dissolved Inorganic Carbon (v8)

The WHONDRS Summer 2019 Sampling (S19S) Campaign collected samples in 97 globally distributed river corridor systems between July and September 2019. Surficial streambed sediments were collected at three locations within each site (upstream, midstream, and downstream). Surface water was collected at the downstream site. This dataset includes a portion of the data types produced from the surface water samples and does not include any results from the sediment. Sediment data can be found at https://data.ess-dive.lbl.gov/view/doi:10.15485/1729719. The S19S campaign was designed with the science community to ask questions associated with links among core/transient metabolomes, microbial metabolism, biogeochemical function, and physical properties of watershed and river corridor systems. This data package was originally published in March 2020. It was updated in October 2020 (v2, new and modified files), July 2021 (v3, new and modified files), January 2023 (v4, new and modified files), February 2023 (v5, new and modified files), August 2023 (v6, new and modified files), July 2025 (v7,new and modified files), and August 2026 (v8, new and modified files). Version 8 includes major restructuring of the files and their formats. See the change history section in readme for more details. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset contains (1) readme, (2) data dictionary, (3) file level metadata, (4) sampling protocol, (5) a data checks report, (6) field metadata, (7) international generic sample number (IGSN) mapping file, (8) joint genome institute (JGI) mapping file, and (9) a folder of sample data. The sample data folder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) and FTICR methods, (2) dissolved inorganic carbon (DIC), (3) stable water isotopes (O and H), (4) specific conductivity, (5) total dissolved nitrogen (TN), (6) anions, (7) flow cytometry bacterial abundance data, (8) laboratory methods, and (9) a folder containing unaligned, peak-picked high resolution characterization of dissolved organic matter from sediment via 12 Tesla Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) through the Environmental Molecular Sciences Laboratory (EMSL, https://www.pnnl.gov/environmental-molecular-sciences-laboratory), and (12) a folder containing processed FTICR-MS data. All files are .msa, .txt, .spx, .xml, .csv, .pdf, .txt, .html, .jpg, or .jpeg.

18-O↗

WHONDRS Summer 2019 Sampling Campaign: Global River Corridor Sediment FTICR-MS, Dissolved Organic Carbon, Aerobic Respiration, Elemental Composition, Grain Size, Total Nitrogen and Organic Carbon Content, Bacterial Abundance, and Stable Isotopes (v10)

The WHONDRS Summer 2019 Sampling (S19S) Campaign collected samples in 97 globally distributed river corridor systems between July and September 2019. Surficial streambed sediments were collected at three locations within each site (upstream, midstream, and downstream). Surface water was collected at the downstream site. This dataset includes a portion of the data types produced from the sediment samples and does not include any results from the surface water. Surface water data can be found at https://data.ess-dive.lbl.gov/view/doi:10.15485/1603775. The S19S campaign was designed with the science community to ask questions associated with links among core/transient metabolomes, microbial metabolism, biogeochemical function, and physical properties of watershed and river corridor systems. This data package was originally published in December 2020. It was updated in April 2021 (v2; modified files), May 2022 (v3; new and modified files), November 2022 (v4; new and modified files), January 2023 (v5 and v6; new and modified files), February 2023 (v7; new and modified files), August 2023 (v8; new and modified files), July 2025 (v9; new and modified files), and June 2026 (v10; new and modified files). Version 10 includes major restructuring of the files and their formats. See the change history in the readme for details. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset contains (1) readme, (2) data dictionary, (3) file level metadata, (4) sampling protocol, (5) a data checks report, (6) field metadata, (7) international generic sample number (IGSN) mapping file, (8) joint genome institute (JGI) mapping file, (9) a folder of site photos, and (10) a folder of sample data. The sample data folder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) and FTICR methods; (2) respiration rates and normalized respiration rates calculated from laboratory sediment incubations, (3) X-ray fluorescence (XRF) elemental composition, (4) grain size distribution from the < 2 millimeter fraction of sediment, (5) total organic carbon and total nitrogen content, (6) bacterial abundance flow cytometry data, (7) sediment stable isotopes (C and N), (8) laboratory methods, (9) water and sediment mass and volume used in incubations, (10) a folder with code, inputs, and plots for the respiration rates, (11) a folder containing XRF data, (12) a folder containing unaligned, peak-picked high resolution characterization of dissolved organic matter from sediment via 12 Tesla Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) through the Environmental Molecular Sciences Laboratory (EMSL; https://www.pnnl.gov/environmental-molecular-sciences-laboratory), (12) and a folder containing processed FTICR-MS data. All files are .msa, .txt, .spx, .xml, .csv, .pdf, .txt, .jpg, or .jpeg.

13C↗

LCLS Big Data Handling – How I Learned to Stop Worrying and Love the Data Deluge

Advanced data and computing systems are vital to Linac Coherent Light Source (LCLS) operations, data interpretation and overall scientific productivity. The transition to MHz-era operation marks a fundamental change in scale that requires new infrastructure and architectures to link LCLS to the required scale of computing needed for scientific interpretation. The LCLS-II Data System meets big data challenges by implementing configurable data reduction that can adapt to multiple science areas, real-time analysis frameworks to provide visualization and fast feedback, and the ability to transfer data to local and remote computational facilities for near real time analysis at the appropriate scale. Feature extracted information generated in the data analysis pipeline - at the edge, local compute, or remote High-Performance Computing (HPC) resources - can be used to steer experiments and inform user decisions during beam time. Artificial Intelligence and Machine Learning (AI/ML) techniques present new opportunities to rapidly analyse large datasets and direct experiments, but create new challenges in scaling, adaptability, complexity, and trustworthiness. We describe how the LCLS-II Data System architecture addresses its data-driven challenges in the areas of data acquisition, data processing, data management, and workflow orchestration to decrease the overall time-to-science and provide a vision for future developments.

artificial intelligence↗