Fine-tuning machine learning models (hyperparameter tuning) is crucial but tedious. Fortunately, optimization promises to automate this task. We give an overview on algorithms for this task and explain their inner workings. To help you selecting one for your project, we benchmark implementations in Python against human experts. We link to the discussion on automating machine learning in general.