Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “social analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Environmental and Social Justice Implications of a Circular Plastics Economy

A consideration of environmental justice (EJ) and social justice (SJ) is critical to minimize the impacts of technology deployment on local communities. SJ and EJ impacts occur in specific geographic locations but can cover a wide range of effects (e.g., air pollution, access to clean water, jobs, wages, and education), making it challenging to determine which metrics are appropriate to evaluate and which data are required. Thus, there is currently a gap in the analysis community's ability to provide useful and universal EJ and SJ metrics for emerging technologies. Here, we present a draft framework for evaluating the human health, local environment, and job implications of processes that are at an early or middle technology readiness level (TRL). Using a case study on enzymatic polyethylene terephthalate (PET) recycling (middle TRL), we demonstrate how to qualitatively and quantitatively assess these EJ and SJ metrics for a circular economy context and how to communicate the results in a manner beneficial to both researchers and local communities.

circular economy↗

Geospatial analysis of preterm and small-for-gestational age births in Washington D.C.

Background: This study is based on the recognition that adverse pregnancy outcomes significantly affect maternal and infant health, leading to increased morbidity and mortality. These outcomes are shaped by a complex interplay of individual-level factors—like maternal age and education—and community-level influences, including socio-economic status and access to healthcare. Understanding these determinants is crucial for developing effective public health strategies, especially for marginalized populations, by identifying high-risk areas and informing targeted interventions that address both individual and structural barriers. Methods: We utilized geospatial analysis to explore the association between individual- and community-level factors and adverse pregnancy outcomes, specifically preterm birth (PTB) and small-for-gestational-age (SGA) birthweight in Washington, D.C. We used Empirical Bayes smoothing methods to calculate rates of adverse birth outcomes from 2010 to 2018 at the U.S. Census tract–level. Spatial scan statistics were used to investigate if adverse birth outcomes clustered in specific areas. ANOVA tests were conducted for individual- and community-level factors within identified clusters. Results: Spatial analysis identified significant high-risk clusters for PTB and SGA infants primarily in southeastern Washington, D.C., particularly in Wards 7 and 8. Individuals residing within these clusters experienced a 47% increased risk of PTB (RR = 1.467) and a 56% increased risk of SGA (RR = 1.560) compared to those outside clusters. Space–time analysis revealed temporal variation, with PTB clusters persisting from 2011 to 2014 and SGA clusters extending through 2017. Compared to low-risk clusters, high-risk clusters had younger birthing individuals (mean age ~26.5 vs. ~33 years), lower maternal college degree attainment (~20% vs. ~80%), higher rates of late or no prenatal care (~16% vs. 11%), and increased prevalence of smoking and hypertension (all P < 0.001). Community-level indicators showed lower median household incomes ($\$40,000$ vs. ~$\$105,000$), greater poverty (~16% vs. ~7% below $\$10,000$/year), higher public assistance use (~32% vs. ~5%), and reduced healthcare access (greater distances to emergency and specialty care) in high-risk areas (all P < 0.001). Neighborhood deprivation indices were significantly elevated, commutes were longer, and population density was lower in these clusters. These findings highlight that adverse birth outcomes cluster in neighborhoods with pronounced socioeconomic and health disparities. Conclusion: High-risk birth clusters highlight intertwined factors: individual, socio-economic, and geographic. Addressing these requires comprehensive interventions focusing on social and structural determinants of health.

Birth outcomes↗

Integrating Spatial and Ethnographic Methods for Resilience Research: A Thick Mapping Approach for Hurricane Maria in Puerto Rico

Hurricane Maria left unprecedented impacts on Puerto Rican communities, leaving some without infrastructure services and unable to communicate with family for several months. Here, to understand the forms of community-level resilience that emerged while hard infrastructure systems took time recover, this article (1) abductively explores resilience as an emergent phenomenon of complex adaptive systems; (2) identifies subsequent forms of social capital, local adaptive capacities, and manifestations of quantifiable variables, such as infrastructure performance, in community experiences; and (3) demonstrates a framework to integrate disparate methodologies for resilience assessments via a multiplicity of mappings of space and place. We combine ethnographic and geospatial methods into an interactive GeoApp for analysis using participant-coded narratives and a series of geospatial indicators as a thick map. Thick mapping facilitates quantitative and qualitative data analysis at several scales, while enabling qualitative query of collected narratives. Results highlight local innovation, community bonding and bridging, and nuances in the role of public institutions as emergent elements of resilience. The thick map shows how top-down assessments can be augmented by thick data and how multiple framings can be anchored in the same system or place. These findings are important to inform and integrate community-oriented and technocentric solutions toward resilience-enhancing measures.

54 ENVIRONMENTAL SCIENCES↗

Marginal Damage of Methane Emissions: Ozone Impacts on Agriculture

Methane directly contributes to air pollution, as an ozone precursor, and to climate change, generating physical and economic damages to different systems, namely agriculture, vegetation, energy, human health, or biodiversity. The methane-related damages to climate, measured as the Social Cost of Methane, and to human health have been analyzed by different studies and considered by government rulemaking in the last decades, but the ozone-related damages to crop revenues associated to methane emissions have not been incorporated to policy agenda. Using a combination of the Global Change Analysis Model and the TM5-FASST Scenario Screening Tool, we estimate that global marginal agricultural damages range from ~ 423 to 556 $2010/t-CH 4 , of which 98 $2010/t-CH 4 occur in the USA, which is the most affected region due to its role as a major crop producer, followed by China, EU-15, and India. These damages would represent 39–59% of the climate damages and 28–64% of the human health damages associated with methane emissions by previous studies. The marginal damages to crop revenues calculated in this study complement the damages from methane to climate and human health, and provides valuable information to be considered in future cost-benefits analyses.

54 ENVIRONMENTAL SCIENCES↗

Where to cool off: a geospatial framework for placement of cooling centers

Indoor cooling is essential to reduce heat stress and increase passive survivability during heatwaves. Although air conditioning (AC) is recommended for maintaining indoor thermal comfort, low- and medium-income households in the U.S. often do not own an AC and/or limit AC usage to reduce energy consumption and associated costs, thereby risking their health and safety. With the frequency and intensity of heatwaves increasing, cooling centers are considered an appropriate alternative to indoor cooling and a possible mitigation strategy to prevent adverse health impacts of heat exposure. However, these centers are limited in numbers and not always accessible. This requires (i) developing a geospatial framework using physical and social factors for optimal siting of cooling centers to meet future needs and (ii) ranking of existing and potential cooling centers (schools, libraries, religious institutions) based on their accessibility among vulnerable populations and proximity to healthcare facilities. We developed and deployed a geospatial framework based on the Multi-criteria Decision Analysis approach in five U.S. cities (Los Angeles (LA), Phoenix, Austin, Atlanta, Miami) to evaluate the effectiveness of the framework in ranking cooling centers based on accessibility and population coverage. The results revealed that (i) access to cooling centers varies across cities and 32.2–50.7% of centers are within walking distance of the most vulnerable populations, (ii) vulnerable populations exposed to Urban Heat Island (UHI) effects are more likely to experience energy burden, and (iii) about 21.2–49.4% of population with high energy burden have access to these centers. Considering that more cooling centers are needed to assist energy burdened households alleviate heat exposure impacts, the framework developed herein could be adapted to incorporate other factors (e.g. health impacts, policies) to assess site suitability of existing shelters, identify potential sites for new cooling centers, and geo-target communities where energy efficient emerging technologies could be deployed to reduce heat stress.

58 GEOSCIENCES↗

Paradigm for approaching the forbidden spontaneous phase transition in the one-dimensional Ising model at a fixed finite temperature

The Ising model describes collective behaviors such as phase transitions and critical phenomena in various physical, biological, economical, and social systems. It is well known that spontaneous phase transition at finite temperature does not exist in the Ising model with short-range interactions in one dimension. Yet, little is known about whether this forbidden phase transition can be approached arbitrarily closely—at fixed finite temperature. Here I use symmetry analysis of the transfer matrix to reveal the existence of spontaneous ultranarrow phase crossover (UNPC) at finite temperature in one class of one-dimensional Ising models on decorated two-leg ladders, in which the crossover temperature T 0 is determined solely by on-rung interactions and decorations, while the crossover width 2 δ T is independently, exponentially reduced ( δ T = 0 means a genuine phase transition) by on-leg interactions and decorations. These findings establish a simple ideal paradigm for realizing an infinite number of one-dimensional Ising systems with spontaneous UNPC at desirable T 0 , which would be characterized in routine laboratory measurements as a genuine first-order phase transition with large latent heat thanks to the ultranarrow δ T (say less than one nanokelvin), paving a way to push the limit in our understanding of phase transitions and the dynamical actions of frustration arbitrarily close to the forbidden regime. Published by the American Physical Society 2024

1-dimensional systems↗

A Play-Based Exploration of CO 2 Storage in the Illinois Basin: Community Baseline Assessment for Carbon Storage Engagement Planning in the Illinois Basin

This report presents a stakeholder mapping analysis conducted as part of the Play-Based Exploration of CO 2 Storage in the Illinois Basin project (DOE Cooperative Agreement DE-FE0032366). The analysis was undertaken to systematically identify, categorize, and characterize stakeholder groups and community conditions relevant to the planning and deployment of carbon capture and storage (CCS) infrastructure across the Illinois Basin. Its purpose is to establish a contextual baseline that informs targeted engagement strategies, supports equitable project planning, and strengthens the social dimensions of CCS site screening.

01 COAL, LIGNITE, AND PEAT↗

Investigating the interconnectedness of active transportation and public transit usage as a primer for Mobility-as-a-Service adoption and deployment

With the advent of Mobility-as-a-Service packages to reduce car usage and (by extension) greenhouse gas emissions, it is crucial that researchers and practitioners consider mutual determinants and outcomes that link the adoption of multiple alternative modes. This study therefore investigates the joint usage of active travel (walking, cycling, bike-sharing) and public transit (bus, rail) modes with respect to personal and situational contexts. Online survey data (n = 826) were collected across six Midwestern U.S. states using Amazon MTurk. Respondents indicated their average weekly usage of eight travel modes across three trip purposes so that multimodality could be assessed. Several psychological constructs were extracted via (a) the stages of change framework, used to indicate willingness to adopt new behavior, and (b) confirmatory factor analysis conducted on Likert scales rooted in theories of community, identity, norms, personality, and well-being. A multiple-indicators multiple-causes structural equation model is then employed to investigate the process of adopting a modality style that incorporates active and transit modes. The model confirms that compatible physical and social contexts, as well as navigational skills and openness to learning, are key primers of multimodalism. However, a path juncture stemming from neighborhood support for mobility innovation illustrates a potential polarity in outcomes between individuals and communities. In addition, the stage of active mobility adoption is linked to identity and norm activation, offering further guidance on what influences readiness for change. The seamless integration of mobility services is critical to matching the convenience and comfort of the private vehicle; understanding potential pathways to sustainable mobility, though, requires analyses and interventions that are driven by well-being outcomes and grounded in rigorous psychological frameworks. Here, the research findings offer practical guidance for identifying intervention opportunities to be linked with MaaS enrollment while demonstrating the need to illuminate how mobility might relate to social cohesion, identity expression, and various sources of satisfaction.

42 ENGINEERING↗

Machine Intelligence to Detect, Characterise, and Defend against Influence Operations in the Information Environment

Social media has enabled a new era of manipulation in the information and cognitive domains. Deceptive content—misleading, falsified, and fabricated—is routinely created and spread in the modern social media environment with the intent to create confusion and widen political and social divides, and exploit the societal conflict exacerbated by these divides in the real-world (aka physical domain). Such disinformation campaigns demonstrate a threat to the integrity of economic, political, cultural, public health, and national security institutions around the world. In this work we overview our artificial intelligence (AI) capabilities to detect, describe, and defend against information operations on Twitter as an example social platform to understand the influence of misleading and falsified content diffusion and better enable those charged with defending against such manipulation to enable responsive parties to counter it. We first present novel linguistically-informed deep learning (DL) models for misinformation and disinformation detection, and present an in-depth linguistic analysis of psycho-linguistic markers across broad deception categories. We then demonstrate how our models perform in the multilingual and multimodal setting and categorize falsified and misleading content based on the intent to deceive. We also provide a large-scale analysis to describe user behavior and spread patterns while engaging with deceptive content and report novel findings about the immediate diffusion of deceptive content by characterizing the vulnerable sub-populations and their demographics, and explicitly measuring speed and scale of deception spread to uncover who shares deceptive content, how quickly, how much, and how evenly. In addition, we measure audience reactions to misinformation and disinformation at scale, distinguishing the reactions of users identified as bots versus humans. Finally, we take advantage of deep translation and generation models to create unique solutions for real-time defense against digital deception and discuss how to apply causal inference to prescribe and intervene into strategic communications jointly across information, cognitive, and physical domains.

artificial intelligence, deep learning, neural lan↗

Geospatial assessment of the economic opportunity for reforestation in Maryland, USA

Afforestation and reforestation have the potential to provide effective climate mitigation through forest carbon sequestration. Strategic reforestation activities, which account for both carbon sequestration potential (CSP) and economic opportunity, can provide attractive options for policymakers who must manage competing social and environmental goals. In particular, forest carbon pricing can incentivize reforestation on private land, but this may require landholders to forego other profits. Here, we utilize an ambitious geospatial approach to quantify economic opportunities for reforestation in the state of Maryland (USA) based on high-resolution remoting sensing, ecosystem modeling, and economic analysis. Our results identify spatially-explicit areas of economic opportunity where the potential revenue from forest carbon outcompetes the expected profit of existing cropland at the hectare scale. Specifically, we find that under a baseline economic scenario of 20 dollars per ton of carbon (5% rental rate) and decadal average crop profitability, a transition to forest on agricultural land would be more profitable than 23.2% of cropland in Maryland under a 20-year land-use commitment. Accounting for variations in carbon and crop pricing, 5.5% to 55.4% of cropland would be immediately outcompeted by expected forest carbon revenue, with the potential for an additional 0.5% to 10.6% of outcompeted cropland within 20 years. Under the baseline economic scenario, an annual allocation of $5.8 million towards a carbon rental program could protect 6.93 Tg C (2.2% of the state’s total CSP) on reforested croplands. This moderate yearly cost is equal to 9.7% of Maryland’s average annual auction proceeds from participation in the Regional Greenhouse Gas Initiative (between 2014-2018), and 19.3% of the average annual subsidy payments for corn, soy, and wheat allocated over the same period. This methodological approach may be useful for state governments, not-for-profit organizations, or regional climate initiatives interested in identifying strategic areas for reforestation.

54 ENVIRONMENTAL SCIENCES↗

Biomass feedstock transport using fuel cell and battery electric trucks improves lifecycle metrics of biofuel sustainability and economy

We report the use of new vehicle technologies such as fuel cell hybrid electric and fully electric powertrains for biomass feedstock supply is an unexplored solution to reducing biofuel production cost, greenhouse gas emissions, and health impacts. These technologies have found success in light-duty vehicle applications and are in development for heavy-duty trucks. This study presents the first detailed stochastic techno-economic analysis and life-cycle assessment of biomass feedstock supply systems with diesel, fuel cell hybrid electric, and fully electric trucks and determines their impacts on biofuel production considering butanol as a representative biofuel. This study finds that fuel cell hybrid electric and fully electric trucks consume less energy relative to the diesel-powered truck regardless of the evaluated circumstances, including payloads of truck (loaded and empty), pavement types (gravel and paved), road conditions (normal and damaged), and road networks (local and highways). The use of fuel cell hybrid and fully electric trucks powered by H 2 -fuel and renewable sources of electricity, respectively, results in a large reduction in cost and carbon footprint, specifically for a long-distance hauling, and minimize other economic and environmental impacts. While the economic advantage of fuel cell hybrid electric vehicle is dependent on the price of H 2 -fuel and road conditions, use reduces the GHG emissions of biobutanol per 100 km-trucking-distance by 0.98-10.9 gCO 2e /MJ. Results show that converting to fully electric truck transport decreases the biobutanol production cost and GHG emissions per 100 km-trucking-distance by 0.4-7.3 cents/L and 0.78 to 9.1 gCO 2e /MJ, respectively. This study establishes the foundation for future investigations that will guide the development of economically, socially, and environmentally sustainable biomass feedstock supply system for cellulosic biorefineries or other goods transportation systems.

09 BIOMASS FUELS↗

Factors influencing mode choice of adults with travel-limiting disability

Introduction: Despite the plethora of research devoted to analyzing the impact of disability on travel behavior, not enough studies have investigated the varying impact of social and environmental factors on the mode choice of people with disabilities that restrict their ability to use transportation modes efficiently. This research gap can be addressed by investigating the factors influencing the mode choice behavior of people with travel-limiting disabilities, which can inform the development of accessible and sustainable transportation systems. Additionally, such studies can provide insights into the social and economic barriers faced by this population group, which can help policymakers to promote social inclusion and equity. Method: This study utilized a Random Parameters Logit model to identify the individual, trip, and environmental factors that influence mode selection among people with travel-limiting disabilities. Here, using the 2017 National Household Travel Survey data for New York State, which included information on respondents with travel-limiting disabilities, the analysis focused on a sample of 8,016 people. In addition, climate data from the National Oceanic and Atmospheric Administration were integrated as additional explanatory variables in the modeling process. Results: The results revealed that people with disabilities may be inclined to travel longer distances walking in the absence of suitable accommodation facilities for other transportation modes. Furthermore, people were less inclined to walk during summer and winter, indicating a need to consider weather conditions as a significant determinant of mode choice. Moreover, low-income people with disabilities were more likely to rely on public transport or walking. Conclusion: Based on this study’s findings, transportation agencies could design infrastructure and plan for future expansions that is more inclusive and accessible, thus catering to the mobility needs of people with travel-limiting disabilities.

99 GENERAL AND MISCELLANEOUS↗

Deer Isle and Stonington Energy Reliability Opportunities

The towns of Deer Isle and Stonington, located on the island of Deer Isle in Maine, partnered with the U.S. Department of Energy's Energy Technology Innovation Partnership Project (ETIPP) to examine options to enhance energy resilience, reduce dependence on imported fuels, and mitigate high energy costs. With technical support from the National Laboratory of the Rockies (NLR), Lawrence Berkeley National Laboratory (LBNL), and regional partner the Island Institute, the project focused on evaluation of the local electricity distribution infrastructure, examination of ongoing activities to reduce the frequency of outages, review of the applicable policy and regulatory environment, and identification of potential on-site energy solutions. Key findings revealed the island's reliance on a single distribution line, and limited hosting capacity for additional distributed energy resources. Solar photovoltaics (PV) technology, with and without battery storage, was found to be the most viable technology given economic, technical, and social considerations. While also technically feasible, wind energy faces more difficult siting and less public acceptance compared to PV. Marine energy technologies were determined to be infeasible due to technology immaturity and lengthy permitting timelines. Modeling of potential building-scale PV plus storage systems resulted in successful grant applications for installations at two local facilities. Analysis of community scale energy generation and storage options resulted the local utility, Versant, submitting a grant application for a proposed community-scale battery storage system to increase grid resilience and hosting capacity. This initiative has strengthened the communities' energy planning capabilities and laid the groundwork for future clean energy development.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Adapting the Technology Performance Level Integrated Assessment Framework to Low-TRL Technologies Within the Carbon Capture, Utilization, and Storage Industry, Part I

With the urgent need to mitigate climate change and rising global temperatures, technological solutions that reduce atmospheric CO 2 are an increasingly important part of the global solution. As a result, the nascent carbon capture, utilization, and storage (CCUS) industry is rapidly growing with a plethora of new technologies in many different sectors. There is a need to holistically evaluate these new technologies in a standardized and consistent manner to determine which technologies will be the most successful and competitive in the global marketplace to achieve decarbonization targets. Life cycle assessment (LCA) and techno-economic assessment (TEA) have been employed as rigorous methodologies for quantitatively measuring a technology's environmental impacts and techno-economic performance, respectively. However, these metrics evaluate a technology's performance in only three dimensions and do not directly incorporate stakeholder needs and values. In addition, technology developers frequently encounter trade-offs during design that increase one metric at the expense of the other. The technology performance level (TPL) combined indicator provides a comprehensive and holistic assessment of an emerging technology's potential, which is described by its techno-economic performance, environmental impacts, social impacts, safety considerations, market/deployability opportunities, use integration impacts, and general risks. TPL incorporates TEA and LCA outputs and quantifies the trade-offs between them directly using stakeholder feedback and requirements. In this article, the TPL methodology is being adapted from the marine energy domain to the CCUS domain. Adapted metrics and definitions, a stakeholder analysis, and a detailed foundation-based application of the systems engineering approach to CCUS are presented. The TPL assessment framework is couched within the internationally standardized LCA framework to improve technical rigor and acceptance. It is demonstrated how stakeholder needs and values can be directly incorporated, how LCA and TEA metrics can be balanced, and how other dimensions (listed earlier) can be integrated into a single metric that measures a technology's potential.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Understanding the Drivers of Mobility during the COVID-19 Pandemic in Florida, USA Using a Machine Learning Approach

As of March 2021, the State of Florida, U.S.A. had accounted for approximately 6.67% of total COVID-19 (SARS-CoV-2 coronavirus disease) cases in the U.S. The main objective of this research is to analyze mobility patterns during a three month period in summer 2020, when COVID-19 case numbers were very high for three Florida counties, Miami-Dade, Broward, and Palm Beach counties. To investigate patterns, as well as drivers, related to changes in mobility across the tri-county region, a random forest regression model was built using sociodemographic, travel, and built environment factors, as well as COVID-19 positive case data. Mobility patterns declined in each county when new COVID-19 infections began to rise, beginning in mid-June 2020. While the mean number of bar and restaurant visits was lower overall due to closures, analysis showed that these visits remained a top factor that impacted mobility for all three counties, even with a rise in cases. Our modeling results suggest that there were mobility pattern differences between counties with respect to factors relating, for example, to race and ethnicity (different population groups factored differently in each county), as well as social distancing or travel-related factors (e.g., staying at home behaviors) over the two time periods prior to and after the spike of COVID-19 cases.

60 APPLIED LIFE SCIENCES↗

GPS-supported smartphone app-based integrated travel diary and time-use data collection: challenges and lessons learned

Travel behaviour and time-use data are two vital data sources for travel demand modelling. Travel behaviour is traditionally collected through household travel surveys, enhanced by using GPS-supported smartphone apps for passive location data collection. However, recruiting individuals willing to install these apps with sustained motivation to continue participation has been a critical challenge. This paper shares insights from a travel and time-use data collection procedure in Chicago and Sydney using the Fourstep app. Social media platforms were utilised as a solution to recruit participants in Chicago, where an international market research company failed to accomplish the task. This paper also discusses the challenges we faced and suggests ways to overcome them, offering valuable guidance to researchers in recruiting participants for smartphone application-based data collection. It also offers an analysis of travel, time-use, and travel-based multitasking behaviours based on the data collected from the Chicago and Sydney samples.

GPS-supported smartphone apps↗

Geographical Insights into Suicide Mortality Through Spatial Machine Learning

Suicide mortality is a leading cause of death in the United States, with an upward trend that emphasizes its significance as a public health issue. Previous research has employed global models like ordinary least squares (OLS) regression and local models such as geographically weighted regression (GWR). While local models are useful for analyzing spatial variations in suicide mortality, they share limitations with traditional global models, particularly about their inability to handle multi-collinearity and non-linear relationships. Machine learning approaches, like random forests (RF), can address some of these limitations but often fail to account for spatial variability. This gap highlights the need for spatial ML models specifically designed to tackle suicide mortality. This research seeks to fill this void by using a geographically weighted random forest model (GWRF) to examine the associations between county-level suicide mortality in the U.S. from 2010 to 2020 and various social and environmental determinants of health. A key aspect of our methodology is disciplined feature selection, which reduces the pool of explanatory variables by about 90%. This refinement enhances the explanatory power of both global (R2 improved from 0.59 to 0.67) and local (R2 improved from 0.64 to 0.67) RF models while reducing their run times. An analysis of the importance scores for these selected features reveals that the drivers of suicide mortality vary by context. Thus, to effectively address regional disparities and inform targeted public health interventions, a holistic approach that incorporates multiple county-level characteristics is essential.

Lebakula, Viswadeep [ORNL] (ORCID:0000000152935914↗

Explaining and predicting human behavior and social dynamics in simulated virtual worlds: reproducibility, generalizability, and robustness of causal discovery methods

Ground Truth program was designed to evaluate social science modeling approaches using simulation test beds with ground truth intentionally and systematically embedded to understand and model complex Human Domain systems and their dynamics Lazer et al. (Science 369:1060–1062, 2020). Our multidisciplinary team of data scientists, statisticians, experts in Artificial Intelligence (AI) and visual analytics had a unique role on the program to investigate accuracy, reproducibility, generalizability, and robustness of the state-of-the-art (SOTA) causal structure learning approaches applied to fully observed and sampled simulated data across virtual worlds. In addition, we analyzed the feasibility of using machine learning models to predict future social behavior with and without causal knowledge explicitly embedded. In this paper, we first present our causal modeling approach to discover the causal structure of four virtual worlds produced by the simulation teams—Urban Life, Financial Governance, Disaster and Geopolitical Conflict. Our approach adapts the state-of-the-art causal discovery (including ensemble models), machine learning, data analytics, and visualization techniques to allow a human-machine team to reverse-engineer the true causal relations from sampled and fully observed data. We next present our reproducibility analysis of two research methods team’s performance using a range of causal discovery models applied to both sampled and fully observed data, and analyze their effectiveness and limitations. We further investigate the generalizability and robustness to sampling of the SOTA causal discovery approaches on additional simulated datasets with known ground truth. Our results reveal the limitations of existing causal modeling approaches when applied to large-scale, noisy, high-dimensional data with unobserved variables and unknown relationships between them. We show that the SOTA causal models explored in our experiments are not designed to take advantage from vasts amounts of data and have difficulty recovering ground truth when latent confounders are present; they do not generalize well across simulation scenarios and are not robust to sampling; they are vulnerable to data and modeling assumptions, and therefore, the results are hard to reproduce. Finally, when we outline lessons learned and provide recommendations to improve models for causal discovery and prediction of human social behavior from observational data, we highlight the importance of learning data to knowledge representations or transformations to improve causal discovery and describe the benefit of causal feature selection for predictive and prescriptive modeling.

97 MATHEMATICS AND COMPUTING↗