// interactive walkthrough

How LightGBM trains — leaf by leaf

Same gradient/Hessian + gain math as XGBoost, but a fundamentally different growth strategy: leaf-wise (best-first) instead of level-wise. At every step, LightGBM splits the leaf that promises the highest gain — not whichever one happens to be at the current depth. This gives faster loss reduction per leaf, but produces unbalanced, deep-on-one-side trees.

Leaf-wise vs level-wise — the one slide that distinguishes LightGBM from XGBoost

XGBoost grows level-wise: split every node at depth d before any node at depth d+1. Tree depth is the natural complexity knob (max_depth).

LightGBM grows leaf-wise: maintain a priority queue of current leaves and always split the one with the largest potential gain. The natural complexity knob is the total number of leaves (num_leaves). A LightGBM tree can be very deep on one branch and very shallow on another. Watch the priority queue panel below — you can see which leaf gets chosen and why.

settings changed — model has been reset.
01

Task & objective

// what we're predicting
task
objective
02

Hyperparameters

// LightGBM-flavoured knobs · changes reset the model
03

Training controls

// one leaf-split, one tree, or all at once
ready
data & ensemble prediction —
current tree (leaf-wise) —
click ▷ Next leaf-split to start.
internal leaf last split
04

Leaf priority queue

// every current leaf, ranked by its best-possible gain — top one splits next

Start growing a tree; this queue will show every current leaf with its potential gain.

05

Why this split?

// gain math for the chosen leaf

Take a leaf-split to see the gain calculation here.

06

Tree → ensemble update

// Fnew(x) = Fold(x) + η · tree(x) — same as XGBoost

Finalize a tree (click ▶ Next tree) to see the update.

training loss —
predictions vs actuals —
07

Ensemble history

// one row per finished tree

No trees yet.