Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Comparative analysis of model-free and model-based HVAC control for residential demand response

In this paper, we present a comparative analysis of model-free reinforcement learning (RL) and model predictive control (MPC) approaches for intelligent control of heating, ventilation, and air-conditioning (HVAC). Deep-Q-network (DQN) is used as a candidate for model-free RL algorithm. The two control strategies were developed for residential demand-response (DR) HVAC system. We considered MPC as our golden standard to compare DQN's performance. The question we tried to answer through this work was, What % of MPC's performance can be achieved by model-free RL approach for intelligent HVAC control?. Based on our test result, RL achieved an average of ≈ 62% daily cost saving of MPC. Considering the pure optimization and model-based nature of MPC methods, the RL showed very promising performance. We believe that the interpretations derived from this comparative analysis provide useful insights to choose from various DR approaches and further enhance the performance of the RL-based methods for building energy managements.

Kurte, Kuldeep↗
DOE OSTI2022

Advanced Solar and Load Forecasting Incorporating HD Sky Imaging (Phase III)

Due to rapidly changing sky conditions, the available solar irradiance for energy generation is subject to wide swings in amplitude. As the market penetration of solar energy continues to increase rapidly, these variations in solar power generation are beginning to have an impact on grid stability and increased wear on power switches. Solar power forecasting plays a critical role in operations for Independent System Operators (ISOs) and utilities. Accurate forecasts help maintain grid reliability, optimize generation from renewables, and reduce operating costs. Of particular interest to the ISOs and utilities are sudden changes in solar irradiance, termed “ramp events,” due to the movement of clouds. One significant impact of ramp events on the grid is additional ancillary service requirements necessary to manage such variability. Ramp events can also cause voltage fluctuations in the distribution grids and trigger actions of automated line equipment (e.g., tap changers), leading to additional maintenance costs. In high penetration solar regions, forecasts must be made for both transmission-connected and distribution-connected resources – either behind the meter or in front of it. Particularly for distributed solar energy resources, forecasting can be a challenge due to the lack of visibility of the energy resource. Brookhaven National Laboratory (BNL) has been working towards nowcasting Global Horizontal Irradiance (GHI) using low-cost technologies for several years now. The Solar NowCasting technology being developed by BNL is a 0 – 30 min solar “nowcasting” technology applicable to a scale covering both large generating facilities and residential, distributed solar installations that relies on a network of ground-based high-definition (HD) cameras and surface pyranometers. GHI forecasts are being produced for both regions using 8 ground-based HD cameras and at least 2 surface pyranometers. When stitched together, the camera images can be used to forecast the impact of clouds on available solar irradiance over a domain of ~50 km 2 . Phase I of the project was the engineering scale up conceptual design stage. Phase II was the initial field test and demonstration, in which the technology was scaled up by a factor of 20 times and successfully demonstrated in eastern Long Island. In Phase III, an additional forecasting network was added in upstate NY and both networks are currently being operated until at least a full year of data is gathered by the Albany network to allow for the collection of sufficient data to evaluate performance against the persistence and smart persistence models using the Solar Arbiter.

14 SOLAR ENERGY↗
DOE OSTI2023

RAPID Manufacturing Institute Final Report

The Rapid Advancement of Process Intensification Deployment (RAPID) Manufacturing Institute, founded in 2017, is a public/private partnership between the U.S. Department of Energy and the American Institute of Chemical Engineers (AIChE). RAPID promotes the development, deployment and commercialization of Process Intensification (PI) and Modular Chemical Process Intensification (MCPI) technologies, enabling U.S. manufacturing to reduce energy consumption, improve process efficiencies and lower investment and operating costs. This mission was carried out through parallel work breakdown structure elements including the establishment of committees to guide the operations and technical direction of RAPID, the establishment of management practices and institute processes, education and workforce development (EWD), and six technical focus areas for the development of technologies to advance PI and MCPI. Throughout the initial six-year cooperative agreement, RAPID worked to meet performance metrics which focused on the operation and sustainment of the institute, education and workforce development and the development of PI and MCPI for the advancement of U.S. manufacturing. All these metrics were successfully met through a total of 43 projects which leveraged $\$$70M Federal with $\$$90M cost share. As a result of these efforts, 84 private and public organizations were brought together by RAPID as members to co-invest in R&D, commercialization and deployment of innovative technologies. In the research portfolio, 82% of the 38 projects achieved > 20% energy efficiency improvement. A RAPID Test Network was developed with 51 testbed facilities to enable access to resources, facilities, tools, and expertise. Eight EWD programs were also developed with over 13,000 impressions. RAPID’s efforts to research, develop, demonstrate, and deploy high-impact PI and modular process technology solutions have enabled reduced energy use, increased sustainability, and improved profitability for U.S. manufacturing.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Influences on Discharge Partitioning on a Large River Delta: Case Study of the Mississippi-Atchafalaya Diversion, 1926-1950

The modern Mississippi River (M.R.) Delta is plumbed by the Mississippi and Atchafalaya rivers, setting water and sediment dispersal pathways for Earth’s fifth-largest river system. The Atchafalaya River’s (A.R.) partial annexation of discharge from the M.R., particularly between 1926 and 1950, prompted warnings of a rapid river avulsion and the construction of the Old River Control Structure to regulate flow. Natural and anthropogenic causes of this discharge annexation are difficult to disentangle. Here, we develop and validate a hydrodynamic model of flow partitioning through the historic channel network. We then isolate how several key changes to the system affected discharge partitioning and stage at the A.R.-M.R. diversion. Simulations show that erosion of the upper A.R. can account for 73% of the water discharge increase. Dredging in the lower A.R. between 1932 and 1950 can account for 35% of the water discharge increase, and was also an important control on shear stress distribution. The lower M.R. was slightly erosional during this period, and therefore hindered the A.R. discharge increase. Significant lacustrine delta deposition in A.R. had little effect on partitioning. These findings highlight the importance of A.R. enlargement processes on avulsion dynamics at this site. Given the essential nature of this river junction to the society, transportation, and commerce of the United States, improved attribution of discharge increases may lead to future management strategies that are broadly impactful.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

AmeriFlux FLUXNET-1F PR-xLA NEON Lajas Experimental Station (LAJA)

This is the AmeriFlux Management Project (AMP) created FLUXNET-1F version of the carbon flux data for the site PR-xLA NEON Lajas Experimental Station (LAJA). This is the FLUXNET version of the carbon flux data for the site PR-xLA NEON Lajas Experimental Station (LAJA) produced by applying the standard ONEFlux (1F) software. Site Description - The NEON Lajas Experimental Station (LAJA) site is located on the southwest corner of the main island of Puerto Rico in a experimental range. This is a grassland site that is periodically grazed by cattle.

Network), NEON (National Ecological Observatory [N↗
DOE OSTI2021

Recommendations for Distributed Energy Resource Patching

While computer systems, software applications, and operational technology (OT)/Industrial Control System (ICS) devices are regularly updated through automated and manual processes, there are several unique challenges associated with distributed energy resource (DER) patching. Millions of DER devices from dozens of vendors have been deployed in home, corporate, and utility network environments that may or may not be internet-connected. These devices make up a growing portion of the electric power critical infrastructure system and are expected to operate for decades. During that operational period, it is anticipated that critical and noncritical firmware patches will be regularly created to improve DER functional capabilities or repair security deficiencies in the equipment. The SunSpec/Sandia DER Cybersecurity Workgroup created a Patching Subgroup to investigate appropriate recommendations for the DER patching, holding fortnightly meetings for more than nine months. The group focused on DER equipment, but the observations and recommendations contained in this report also apply to DERMS tools and other OT equipment used in the end-to-end DER communication environment. The group found there were many standards and guides that discuss firmware lifecycles, patch and asset management, and code-signing implementations, but did not singularly cover the needs of the DER industry. This report collates best practices from these standards organizations and establishes a set of best practices that may be used as a basis for future national or international patching guides or standards.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

MOSIQS: Persistent Memory Object Storage With Metadata Indexing and Querying for Scientific Computing

Scientific applications often require high-bandwidth shared storage to perform joint simulations and collaborative data analytics. Shared memory pools provide a chance to satisfy such needs. Recently, a high-speed network such as Gen-Z utilizing persistent memory (PM) offers an opportunity to create a shared memory pool connected to compute nodes. However, there are several challenges to use scientific applications on the shared memory pool directly such as scalability, failure-atomicity, and lack of scientific metadata-based search and query. In this paper, we propose MOSIQS, a persistent memory object storage framework with metadata indexing and querying for scientific computing. We design MOSIQS based on the key idea that memory objects on PM pool can live beyond the application lifetime and can become the sharing currency for applications and scientists. MOSIQS provides an aggregate memory pool atop an array of persistent memory devices to store and access memory objects to accelerate scientific computing. MOSIQS uses a lightweight persistent memory key-value store to manage the metadata of memory objects, which enables memory object sharing. To facilitate metadata search and query over millions of memory objects resident on memory pool, we introduce Group Split and Merge (GSM), a novel persistent index data structure designed primarily for scientific datasets. GSM splits and merges dynamically to minimize the query search space and maintains low query processing time while overcoming the index storage overhead. MOSIQS is implemented on top of PMDK. We evaluate the proposed approach on many-core server with an array of real PM devices. Experimental results show that MOSIQS gains a 100% write performance improvement and executes multi-attribute queries efficiently with 2.7× less index storage overhead offering significant potential to speed up scientific computing applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Regional inertia dynamics of U.S. interconnections: An event-based measurement approach

Power grid inertia plays a vital role in frequency stability following large disturbances, yet its distribution across the U.S. grid is highly uneven. While interconnection-wide inertia benchmarks are useful, they can mask regional variability driven by resource mix, network coupling, and geographic separation. This paper extends event-driven inertia estimation to the regional scale using field measurements from the Frequency Monitoring Network (FNET/GridEye). Starting from balancing authority and independent system operator footprints, candidate regions are refined using a composite coherency score that combines frequency-trajectory shape similarity, timing spread, and lead/lag behavior to ensure dynamic consistency. A filtered sliding difference method (FSDM) is then used to construct regional frequency trajectories, detect disturbance onset, and compute robust regional rate-of-change of frequency (RoCoF). Regional, local, and interconnection inertia are estimated by combining RoCoF with event power imbalance, and additional indicators (regional-to-system inertia ratio and inertial-support arrival time) quantify regional-to-interconnection coupling and relative regional contributions. The method is demonstrated on eleven regions across the Eastern Interconnection (EI) and the Western Electricity Coordinating Council (WECC), with the Electric Reliability Council of Texas (ERCOT) used for validation. In ERCOT, estimates compared against energy management system (EMS) values achieve a mean absolute percentage error of 17.94%. WECC exhibits consistently shorter inertial-support arrival times (0.15–0.3 s) than EI (0.7–1.1 s), highlighting contrasting coupling and disturbance-propagation behavior. Overall, the results reveal pronounced spatial heterogeneity in inertia and coupling, underscoring the value of regional monitoring for both operational decision-making and long-term system planning.

Disturbance events↗
DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗
DOE OSTI2026

AmeriFlux FLUXNET-1F PR-xGU NEON Guanica Forest (GUAN)

This is the AmeriFlux Management Project (AMP) created FLUXNET-1F version of the carbon flux data for the site PR-xGU NEON Guanica Forest (GUAN). This is the FLUXNET version of the carbon flux data for the site PR-xGU NEON Guanica Forest (GUAN) produced by applying the standard ONEFlux (1F) software. Site Description - The Guanica Forest (GUAN) site is located on the southwest corner of the main island of Puerto Rico in a protected forest with primary phenological species of Gymnanthes lucida, Pisonia albida, and Bursera simaruba. The mean canopy height is approximately 10 m.

Network), NEON (National Ecological Observatory [N↗
DOE OSTI2020

Tracking Robot Location for Non-Destructive Evaluation of Double-Shell Tanks

(1) Background: Non-destructive evaluation of double-shell nuclear-waste storage tanks at the U.S. Department of Energy’s Hanford site requires a robot to navigate a network of air slots in the confined space between primary and secondary tanks. Situational awareness, data collection, and data interpretation require continuous tracking of the robot’s location. (2) Methods: Robot location is continuously monitored using video image analysis for short distances and laser ranging for absolute location. (3) Results: The technique was demonstrated in our laboratory using a mockup of air slot and robot. (4) Conclusions: Location tracking and display provide decision support to inspectors and lay the groundwork for automated data collection.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Cybersecurity Certification Recommendations for Interconnected Grid Edge Devices and Inverter Based Resources

Escalating deployment of PV and grid-edge devices on the distribution grid has increased the sustainability and efficiency of the electric grid. However, the increasing number of distributed energy resources (DERs) deployed creates a heightened cyber-physical interdependency on the distribution grid and thus creates more vectors for cyber-attacks to exploit through information and communication technology (ICT) systems and networks. For example, control signal packets can be modified, intercepted, or corrupted due to vulnerabilities in communication protocols used by microgrid controllers and grid edge devices for power control. Therefore, to mitigate and prevent cyber-attacks on grid edge devices and the inverter-based resources connected to the distribution grid, the U.S. Department of Solar Energy Technologies Office (SETO) awarded funding to the National Renewable Energy Laboratory and Sandia National Laboratory (SNL) to research, develop, and harmonize cybersecurity standards for Photovoltaic (PV) systems and for other kinds of DERs. To help develop a standard for DER cybersecurity, NREL established certification recommendations and test cases, in consensus with the solar industry and UL, for ensuring intrinsic design security for DERs. These recommendations were developed to bolster the cybersecure functionalities such as TLS, MAC, CRL, session resumption/renegotiation, and password, system, and service security management within the DER devices. The proposed test cases verify authentication, authorization, confidentiality, and data integrity for data and communications of DERs that use Transmission Control Protocol/Internet Protocol (TCP/IP). They were also developed to protect DER communications from eavesdropping, replay, man-in-the-middle, denial of service (DoS), spoofing through security certificates, least-privilege violation, and brute-force credentials. This report, which has been validated and reviewed by UL, expands upon those test cases to provide DER cybersecurity certification recommendations which increase DER resiliency and help to mitigate cyber-attacks. UL's collaboration with NREL and approval of this document will accelerate the adoption of a UL standard for DER cybersecurity.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

A Scalable Transmission and Distribution Co-simulation Platform for IBR-heavy Power Systems

The integration of Inverter-Based Resources (IBRs) into power systems, including both transmission and distribution networks, poses challenges for studying grid dynamic behaviours under various operating conditions and system events. This paper addresses the urgent need to investigate the influence of IBRs on grid dynamics, and their potential to enhance power system reliability and resilience. We propose a flexible, scalable transmission and distribution co-simulation platform, using opensource tools only for assessing the impact of grid-following (GFL) and grid-forming (GFM) IBRs on dynamic stability at various renewable penetration levels (up to 100%). This platform enables researchers to explore different contingencies at transmission, distribution, or both, providing a comprehensive evaluation of grid status. A series of case studies, including both small and large T&D systems with varying GFL/GFM configurations and contingencies, have been conducted. The results not only robustly validate our co-simulation framework, but also provide invaluable insights for effective IBR management in the power grid.

grid management↗
DOE OSTI2023

EmSense: A High-Resolution Emulated Sensor for Experiments with the Smart Grid and Distributed Ledger Technology

This work involves the development of a device - EmSense (“Emulated Sensor”) - that emulates a high-resolution sensor for a power grid. The device collects raw current and voltage sensor data which derive from ORNL's signature library. This library is a dataset that ORNL curates from many different sources that include power systems from various utilities. The EmSense packages the data from the library in the form of IEC 61850 Sampled Value (SV) packets and then broadcasts these SV packets on the network. In another mode, EmSense can generate artificial sinusoidal data that appears as waveforms for voltage and current signals. EmSense has an internal algorithm for determining the period of a signal based on the data so that the period can be specified as a variable in the IEC 61850 packets. The purpose of EmSense is to allow for experimentation with the Dark Net Infrastructure where a variety of power line sensors must be represented along with their typical communication traffic. The EmSense device was developed in coordination with the software for receiving and processing the packets in the Distributed Ledger Technology (DLT) framework of the DarkNet Project. This receiving software must have a methodology for dealing with information of high velocity, variety, and volume. Experimenting with EmSense facilitates the development of such software. The results showed that the DLT framework and the trust-anchoring approach managed to process a large flow of traffic even with up to six instances of EmSense device broadcasting data. This was achieved without overfilling packet queues in the memory of the actual hardware of the DLT devices or causing the Central Processing Unit (CPU) of the hardware to be overwhelmed. The DLTs were also able to store the data in a compact and useful form for later analysis and archival purposes.

Werth, Aaron↗
DOE OSTI2026

AmeriFlux FLUXNET-1F US-CLF Cole Farm

This is the AmeriFlux Management Project (AMP) created FLUXNET-1F version of the carbon flux data for the site US-CLF Cole Farm. This is the FLUXNET version of the carbon flux data for the site US-CLF Cole Farm produced by applying the standard ONEFlux (1F) software. Site Description - The Cole Farm catchment (0.65km2) is located ~ 4 km southwest of the Shale Hills site, draining orthogonally to a syncline axis of the Wills Creek Formation, a calcareous shale containing interbedded siltstone, sandstone, shaly limestone, and dolomite. Even though the farm adopted no-till practices in the 1970s, the axial channel of Cole Farm flows over a thick (>2.5 m) package of sediment in the valley floor. Soils range in texture from silty clay at the ridge top to sandy loam in the valley floor. Data was collected and funded by the Critical Zone Observatory Network.

Davis, Kenneth J. [Department of Meteorology, Eart↗
DOE OSTI2020

Whose Gas is it anyway? Differentiating the Source of a Large Soil Vapor Plume beneath Two Adjacent Waste Sites - 20487

DOE contractor CH2M Hill Plateau Remediation Company is currently responsible for conducting groundwater contamination monitoring at several RCRA treatment, storage, and disposal units located on the Hanford Site in Richland, Washington State. The Nonradioactive Dangerous Waste Landfill treatment, storage, and disposal unit presents a distinct groundwater monitoring problem because of a large multi-contaminant soil vapor plume beneath it that is a likely source of low-level volatile organic compound groundwater contamination. Adjacent to Nonradioactive Dangerous Waste Landfill is the Solid Waste Landfill. Volatile organic compounds are inventory components of both the Nonradioactive Dangerous Waste Landfill and the Solid Waste Landfill. Therefore, it is possible that both sites could be contributing to the soil vapor plume. For regulatory purposes, it is important to differentiate which site is the primary contributor of volatile organic compounds to the plume. An approach was developed to identify the primary volatile organic compound source of the soil vapor plume beneath Nonradioactive Dangerous Waste Landfill and Solid Waste Landfill. The site conceptual model hypothesis of vapor-phase volatile organic compound transport to the dissolved phase in groundwater was tested by a simple mathematical model of vapor/liquid equilibrium concentrations at the groundwater/air interface. Once it was shown that vapor-phase volatile organic compound transport to groundwater was a valid conceptual model for Nonradioactive Dangerous Waste Landfill and Solid Waste Landfill, spatial and statistical methods were used to determine the primary site contributing to the majority of volatile organic compounds to the soil vapor plume. Average groundwater chloroform, tetrachloroethene, and trichloroethene concentrations from Nonradioactive Dangerous Waste Landfill and Solid Waste Landfill monitoring network wells were plotted on maps of the facilities and immediate vicinities and compared to soil vapor sampling probe locations. Principal component analysis and mixing ratios were used to identify source contributions of each treatment, storage, and disposal unit to the plume. Results of the vapor/liquid equilibrium concentrations mathematical model showed that transport phenomena outweigh steady-state equilibria. Estimated vapor/liquid equilibrium concentrations were considerably lower than soil vapor measurements. The results indicate that dynamic vadose zone and groundwater factors such as decreased vapor concentrations with depth, vapor dilution from dispersion in the vadose zone, and advective and diffusional volatile organic compound dilution in groundwater result in groundwater volatile organic compound concentrations much less than would be measured under steady-state equilibrium conditions. Site source contribution differentiation by principal component analysis and mixing ratios was inconclusive using actual soil gas data because of the similarity in concentration values in both datasets for Nonradioactive Dangerous Waste Landfill and Solid Waste Landfill. Similar data populations suggest mixing of the vapor contributions from both sites by dispersion through the soil matrix pore spaces. However, when groundwater volatile organic compound data were compared between the Nonradioactive Dangerous Waste Landfill and Solid Waste Landfill monitoring networks, Solid Waste Landfill mean concentrations were higher, suggesting more vapor-phase volatile organic compound transport to groundwater at those locations. Simulated volatile organic compound soil vapor and groundwater datasets created to test the methods developed for this study show that the method can be successful in source differentiation when significantly different datasets are compared. This paper will describe a method of testing a conceptual model for vapor-phase contaminant transport to groundwater and for differentiating site sources of contaminants comprising a mixed-constituent soil vapor plume. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

Continuous-variable quantum Boltzmann machine

Here, we propose a continuous-variable quantum Boltzmann machine (CVQBM) using a powerful energy-based neural network. It can be realized experimentally on a continuous-variable (CV) photonic quantum computer. We used a CV quantum imaginary time evolution (QITE) algorithm to prepare the essential thermal state and then designed the CVQBM to proficiently generate continuous probability distributions. We applied our method to both classical and quantum data. Using real-world classical data, such as synthetic-aperture radar (SAR) images, we generated probability distributions. For quantum data, we used the output of CV quantum circuits. We obtained high fidelity and low Kullback–Leibler (KL) divergence showing that our CVQBM learns distributions from given data well and generates data sampling from that distribution efficiently. We also discussed the experimental feasibility of our proposed CVQBM. Our method can be applied to a wide range of real-world problems by choosing an appropriate target distribution (corresponding to, e.g., SAR images, medical images, and risk management in finance). Moreover, our CVQBM is versatile and could be programmed to perform tasks beyond generation, such as anomaly detection.

SAR images↗
DOE OSTI2024

Human limits in machine learning: prediction of potato yield and disease using soil microbiome data

Abstract Background The preservation of soil health is a critical challenge in the 21st century due to its significant impact on agriculture, human health, and biodiversity. We provide one of the first comprehensive investigations into the predictive potential of machine learning models for understanding the connections between soil and biological phenotypes. We investigate an integrative framework performing accurate machine learning-based prediction of plant performance from biological, chemical, and physical properties of the soil via two models: random forest and Bayesian neural network. Results Prediction improves when we add environmental features, such as soil properties and microbial density, along with microbiome data. Different preprocessing strategies show that human decisions significantly impact predictive performance. We show that the naive total sum scaling normalization that is commonly used in microbiome research is one of the optimal strategies to maximize predictive power. Also, we find that accurately defined labels are more important than normalization, taxonomic level, or model characteristics. ML performance is limited when humans can’t classify samples accurately. Lastly, we provide domain scientists via a full model selection decision tree to identify the human choices that optimize model prediction power. Conclusions Our study highlights the importance of incorporating diverse environmental features and careful data preprocessing in enhancing the predictive power of machine learning models for soil and biological phenotype connections. This approach can significantly contribute to advancing agricultural practices and soil health management.

Aghdam, Rosa↗