Evaluation of G-Xgboost Model Performance on Data with Different Levels of Heterogeneity Spatial and Linearity Patterns

Abstract

Spatial data analysis often faces challenges arising from spatial heterogeneity and nonlinear relationships among variables. Conventional methods, such as Ordinary Least Squares (OLS) and Geographically Weighted Regression (GWR), are limited in capturing complex nonlinear patterns, while machine learning approaches, such as XGBoost, do not explicitly account for spatial variation. To address these limitations, Geographical-XGBoost (G-XGBoost) was developed by directly incorporating spatial information into the XGBoost learning process. This study evaluates the performance of G-XGBoost against OLS, GWR, XGBoost, and Geographically Weighted XGBoost (GW-XGBoost) through a simulation study using data generated at 119 locations on Java Island. Each observation included two predictor variables drawn from a standard normal distribution, with response variables generated through linear combinations or hyperbolic tangent (tanh) transformations of spatially varying coefficients. Four scenarios were constructed by combining two levels of relationship linearity (linear and nonlinear) and spatial heterogeneity (low and high), yielding coefficient patterns ranging from smooth gradients to highly dynamic local fluctuations. Model evaluation was based on 100 replications using independently generated training and testing datasets, assessed via R^2, RMSE, MAE, and the training–testing performance gap as a measure of generalization stability. Under linear scenarios, GWR achieved the highest R² values on the test data (up to 0.94 and 0.92, respectively) with the lowest error metrics. In nonlinear scenarios, tree-based methods outperformed regression-based approaches, with GW-XGBoost achieving the highest test R^2 values of up to 0.89 and 0.85. Although G-XGBoost did not always achieve the highest predictive accuracy, it consistently produced training–testing R² gaps centered near zero across all scenarios, with an interquartile range of approximately −0.05 to 0.05, the narrowest among all competing methods, indicating more stable generalization and greater resistance to overfitting across varying levels of linearity and spatial heterogeneity.