How to delete columns in pyspark dataframe

Reading the Spark documentation I found an easier solution.

Since version 1.4 of spark there is a function drop(col) which can be used in pyspark on a dataframe.

You can use it in two ways

  1. df.drop('age')
  2. df.drop(df.age)

Pyspark Documentation – Drop

Leave a Comment