---
title: "5 Best Ways to Delete Rows in PySpark"
description: "In PySpark, delete rows from DataFrame: filter, where, na.drop, drop, SQL Expression based on criteria."
url: https://srinimf.com/2024/03/25/5-best-ways-to-delete-rows-in-pyspark/
date: 2024-03-25
modified: 2026-10-10
author: "Srini"
image: https://i0.wp.com/srinimf.com/wp-content/uploads/2024/03/pexels-photo-4039778.jpeg?fit=1880%2C1253&ssl=1
categories: ["PySpark"]
tags: ["Delete Rows in PySpark"]
type: post
lang: en
---

# 5 Best Ways to Delete Rows in PySpark

In PySpark, you can delete rows from a DataFrame using various methods depending on your criteria for deletion.

Related[PySpark Dataframe: Skipping First Rows and Counting Null Values](https://srinimf.com/2024/04/23/pyspark-interview-questions-skip-the-first-2-rows-and-count-the-nulls-of-each-column/)

![Delete rows in PySpark](https://i0.wp.com/srinimf.com/wp-content/uploads/2024/03/pexels-photo-19802194.jpeg?resize=1880%2C1254&ssl=1)*Photo by Markus Winkler on Pexels.com*

Related[Using conditions: 5 Top PySpark SQL Examples](https://srinimf.com/2023/09/15/how-to-add-conditions-in-pyspark-sql-5-examples/)

## Delete Rows in PySpark: 5 Top Methods

Here are some common approaches:

### 01. Filtering

Use the filter() function [to create](https://srinimf.com/2020/07/25/db2-user-defined-function-syntax-to-create-drop/) a new DataFrame excluding rows that meet specific conditions.

```
# Example: Delete rows where Age is null

df_filtered = df.filter(df.Age.isNotNull())
```

### 02. Where clause

Use the where() function is equivalent to filter().

```
# Example: Delete rows where Age is null

df_filtered = df.where(df.Age.isNotNull())
```

### 03. Drop rows with [null values](https://srinimf.com/2024/09/10/pyspark-dataframe-counting-null-values-in-each-column/)

Use the **na.drop()** function [to drop](https://srinimf.com/2024/10/27/how-to-drop-columns-with-high-null-values-in-pyspark/) rows containing any null or NaN values.

```
# Example: Delete rows with any null values

df_filtered = df.na.drop()
```

**Recommended Books**

- [Data Pipeline Pocket Reference](https://www.amazon.in/Data-Pipelines-Pocket-Reference-Processing/dp/8194722934/?tag=srinimf-21)
- [Querying Databricks SQL](https://www.amazon.in/dp/9355518013/?tag=srinmf-21)
- [Azure Databricks Cookbook](https://www.amazon.in/dp/1789809711/?tag=srinmf-21)

### 04. Drop rows based on specific conditions

Use the drop() function with specific conditions.

```
# Example: Delete rows where Age is less than 18

df_filtered = df.drop(df.Age < 18)
```

### 05. SQL Expression

You can use SQL expressions for more complex conditions.

```
# Example: Delete rows where Age is less than 18 using SQL expression

df.createOrReplaceTempView("data")

df_filtered = spark.sql("SELECT * FROM data WHERE Age >= 18")
```

### Conclusion

Choose the method that best suits your precise requirements for deleting rows from the DataFrame. Remember, these operations return a new DataFrame with the specified rows removed, leaving the original DataFrame unchanged. If you want to modify the original DataFrame in place, you can reassign the filtered DataFrame to the original DataFrame variable.

### Share this:

-
- [Tweet](https://twitter.com/share)
- [![](https://i0.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png)](https://www.pinterest.com/pin/create/link/?url=https%3A%2F%2Fsrinimf.com%2F2024%2F03%2F25%2F5-best-ways-to-delete-rows-in-pyspark%2F&media=https%3A%2F%2Fi0.wp.com%2Fsrinimf.com%2Fwp-content%2Fuploads%2F2024%2F03%2Fpexels-photo-4039778.jpeg%3Ffit%3D1200%252C800%26ssl%3D1&description=5%20Best%20Ways%20to%20Delete%20Rows%20in%20PySpark)
-
- [Share on Tumblr](https://www.tumblr.com/share)
- [Share on WhatsApp (Opens in new window) WhatsApp](https://srinimf.com/2024/03/25/5-best-ways-to-delete-rows-in-pyspark/?share=jetpack-whatsapp)
- [More](#)
-

- [Print (Opens in new window) Print](https://srinimf.com/2024/03/25/5-best-ways-to-delete-rows-in-pyspark/?share=print)
- [Share on Reddit (Opens in new window) Reddit](https://srinimf.com/2024/03/25/5-best-ways-to-delete-rows-in-pyspark/?share=reddit)
- [Share on Telegram (Opens in new window) Telegram](https://srinimf.com/2024/03/25/5-best-ways-to-delete-rows-in-pyspark/?share=telegram)
- [Email a link to a friend (Opens in new window) Email](mailto:?subject=%5BShared%20Post%5D%205%20Best%20Ways%20to%20Delete%20Rows%20in%20PySpark&body=https%3A%2F%2Fsrinimf.com%2F2024%2F03%2F25%2F5-best-ways-to-delete-rows-in-pyspark%2F&share=email)
-

### Like this:

Like Loading…