How Can Data Scientists Use Parallel Processing?

Finally, my program is running! Should I go and get a coffee?

We data scientists have got powerful laptops. Laptops which have quad-core or octa-core processors and Turbo Boost technology. We routinely work with servers with even more cores and computing power. But do we really use the raw power we have at hand?

Instead of taking advantage of our resources, too often we sit around and wait for time-consuming processes to finish. Sometimes we wait for hours, even when urgent deliverables are approaching the deadline. Can we somehow do better?

In this post, I will explain how to use multiprocessing and Joblib to make your code parallel and get out some extra work out of that big machine of yours.