Stop guessing. Start searching. Learn how to systematically find the best settings for your machine learning models — through interactive playgrounds and live strategy comparisons.
Every machine learning model has two kinds of settings. One kind the model learns by itself. The other kind you have to choose. That choice is everything.
The model figures these out from data during training. You don't set them — gradient descent does, by minimizing the loss function step by step.
The knobs you turn before training starts. They control how the model learns — and the wrong settings can ruin everything.
A well-tuned model can be 10× more accurate than a poorly-tuned one — using the exact same architecture and data. Hyperparameters aren't a detail; they're the lever.
Train a real neural network on the classic two-moons dataset. Tune the knobs. Watch the decision boundary bend in real time. Find what works.
Adjust and train to see effects
How the model classifies the entire input space
How the error changes over epochs
Real-time training statistics
See how different learning rates affect training — your current setting is highlighted
You can't try every combination. So how do you search? Watch three classic strategies race on the same problem — same budget, same goal, very different philosophies.
Try every combination on a fixed grid. Thorough, but slow — wastes effort on bad regions.
Sample randomly. Surprisingly strong — especially in high dimensions where grid wastes budget.
Build a surrogate model of the objective. Use it to intelligently pick the next point. Sample-efficient.
Five practical lessons to carry into your own machine learning projects.
The difference between 0.001 and 0.01 matters as much as between 0.01 and 0.1. Sample logarithmically — or waste most of your budget on bad regions. This applies to regularization too.
Grid search exhausts budget on irrelevant dimensions. Random search naturally concentrates budget on what matters — the famous "Bergstra & Bengio" result. The curse of dimensionality is real.
Training a model for hours? Bayesian Optimization finds good solutions in fewer trials by learning from past evaluations. Cheap evals? Random is fine. The surrogate model pays off when each trial hurts.
Tuning on training data is cheating yourself. Use a separate validation set (or cross-validation) — otherwise you're optimizing for the wrong target and your model will fail in production.
Before optimizing, identify which hyperparameters actually affect performance. For deep learning: learning rate, regularization, and model size usually dominate. Batch size and optimizer settings matter less. Spend your budget where it counts — and remember that the best architecture often beats the best-tuned bad one. When in doubt, run a small random search first to see what matters, then focus your budget there.