Here is a quick and dirty chart to figure out what your available modeling techniques are to apply to your dataset based on the number of observations (aka the available population for idiots like me)
| Technique | Minimum Sample Size | Scaling with Variables (k) | Key Rule of Thumb | Notes |
| Multiple Linear Regression | 50 + 8k | Linear growth | Green’s Rule: N = 50 + 8k | Widely accepted rule for regression stability. |
| Logistic Regression | 10EPV | Based on Events Per Variable (EPV) | EPV Rule: At least 10 positive outcomes per predictor variable | If only 20% of data are “events”, total N must be adjusted upward. |
| Decision Tree (Shallow) | 10k | Linear growth | 10 samples per variable | Best for small datasets and shallow depth (max depth 3-4). |
| Decision Tree (Moderate Depth) | 20k | Faster than linear | 20 samples per variable | Needed for trees with moderate depth (5-8). |
| Decision Tree (Deep) | 50k | Much faster growth | 50 samples per variable | Deep trees prone to overfitting without large datasets. |
| Random Forest | ≥10 x k x #trees | Scales with number of trees | Bootstrapping reduces per-tree data, so larger datasets needed | Common starting point: 500 trees, so total population must be high. |
| Gradient Boosted Trees (XGBoost, LightGBM, etc.) | 100k | Very high scaling | 100 samples per variable minimum | Complex ensemble methods need the largest datasets. |
| Neural Network | ≥10 x total weights | Exponential growth with complexity | Rule of thumb: 10x the number of network weights | Deep networks need massive sample sizes to generalize. |
Throwing this together helped me a ton in the beginning. Hopefully it helps you too.
/end trans
Leave a Reply