I want to make a presentation with comparison of pandas and pyspark. For this purpose I need some data and ideally a research task that will allow me to:

  1. Show how joins works in both libraries. So at least the data should consist of two datasets. Ideally data should allow and the task should require some joins/group bys to be performed to show all the spark sql features.
  2. Compare the speed of the libs.
  3. Ideally data should come with a research task/tutorial, so I could show PySpark mllib in action

Could someone please suggest a tutorial/kaggle page/anything that will fit my requirements?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.