Big Data has become synonymous with Data engineering. But the line between Data Engineering and Data scientists is blurring day by day.

I know — Spark is sometimes frustrating to work with.

Too much data is getting generated day by day. Although sometimes we can manage our big data using tools like Rapids or Parallelization, Spark is an excellent tool to have in your repertoire if you are working with Terabytes of data.

Recently I was working on tuning hyperparameters for a huge Machine Learning model.

We as data scientists have gotten quite comfortable with Pandas or SQL or any other relational database.

I am a Mechanical engineer by education. And I started my career with a core job in the steel industry.

Big Data has become synonymous with Data engineering. But the line between Data Engineering and Data scientists is blurring day by day.

Graphs provide us with a very useful data structure. They can help us to find structure within our data.

I have been looking to create this list for a while now.

I generally have a use case for Hadoop in my daily job.