how does xgboost enforce monotonicity constraints

Question

I would like to know that how xgboost enforce monotonic constraints while building the tree model. So far by reading the code, I have understood that it has something to do with weights of each node but am not able to understand why this approach works. Thanks in advance for your answers

Answer 1

Here is a simple pseudocode for the same feature in LightGBM:

min_value = node.min_value
max_value = node.max_value

check(min_value <= split.left_output) 
check(min_value <= split.right_output)
check(max_value >= split.left_otput)
check(max_value >= split.right_output)
mid = (split.left_output + split.right_output) / 2;

if (split.feature is monotonic increasing) {
  check(split.left_output <= split.right_output)
  node.left_child.set_max_value(mid)
  node.right_child.set_min_value(mid)
}
if (split.feature is monotonic decreasing ) {
  check(split.left_output >= split.right_output)
  node.left_child.set_min_value(mid)
  node.right_child.set_max_value(mid)
}

Reference: https://github.com/Microsoft/LightGBM/issues/14#issuecomment-359752223.

I believe it's basically the same algorithm as the one implemented in XGBoost.

For each split candidate:

1. Check the values of both leaves against the monotonicity constraints propagated from predecessors.
2. Check the monotonicity between two leaves.
3. Reject the split if the monotonicity is broken.