---
title: "PySpark When Check Numeric of Column [Tested]"
description: "The post explains how to use PySpark to check if input values are numeric using when condition."
url: https://srinimf.com/2024/07/20/pyspark-when-check-numeric-of-column-tested/
date: 2024-07-20
modified: 2026-10-10
author: "Srini"
image: https://i0.wp.com/srinimf.com/wp-content/uploads/2024/07/image-1.png?fit=1024%2C768&ssl=1
categories: ["PySpark"]
tags: ["PySpark When Numeric"]
type: post
lang: en
---

# PySpark When Check Numeric of Column [Tested]

Do you know if it is common to check input value is numeric or not in PySpark? Here’s a way we can see [how to](https://srinimf.com/2024/08/03/how-to-check-valid-date-in-pyspark-4-top-methods/) use when condition in Pyspark code.

!](https://i0.wp.com/srinimf.com/wp-content/uploads/2024/07/cur.07.447.444_erg456.jpg?resize=999%2C1536&ssl=1)

## PySpark Example Check Numeric

```
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, when, lit

# Initialize SparkSession
spark = SparkSession.builder.appName("Example").getOrCreate()

# Example DataFrame
data = [("Srini", 12345), ("Usha", 8956), ("Sailu", "234"), ("Madhu", "567")]
columns = ["name", "valid_id"]
df = spark.createDataFrame(data, columns)

# Filter DataFrame using isin function with list of values inline
filtered_df = df.withColumn("numeric_check", 
                            when(col("valid_id").cast("int").isNull(), lit("Not numeric"))
                            .when(col("valid_id").cast("string").substr(1,1) == "0", lit("First char is zero"))
                            .otherwise(lit("Numeric")))

# Show the filtered DataFrame
filtered_df.show()
```

Related[PySpark Dataframe: Skipping First Rows and Counting Null Values](https://srinimf.com/2024/04/23/pyspark-interview-questions-skip-the-first-2-rows-and-count-the-nulls-of-each-column/)

> [Data Analytics with Python](https://amzn.to/3ZRssYO) – Top Book

Related[Popular ROLLUP SQL query to apply on your data to group it quickly](https://srinimf.com/2015/06/02/rollup-or-sub-total-in-db2-v11-sql/)

## Explained

- Cast the column to the integer using the cast method, and use the isNull() method. If true, it means, it is not numeric, otherwise, it’s numeric
- The second logic checks if the first char is “0”, it puts a message. Which, I converted to a string, and using the substring function, I extracted the first digit to compare to “0”

## Output

```
df:pyspark.sql.dataframe.DataFrame = [name: string, valid_id: string]
filtered_df:pyspark.sql.dataframe.DataFrame = [name: string, valid_id: string ... 1 more field]
+-----+--------+-------------+
| name|valid_id|numeric_check|
+-----+--------+-------------+
|Srini|   12345|      Numeric|
| Usha|    8956|      Numeric|
|Sailu|     234|      Numeric|
|Madhu|     567|      Numeric|
+-----+--------+-------------+
```

### Share this:

-
- [Tweet](https://twitter.com/share)
- [![](https://i0.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png)](https://www.pinterest.com/pin/create/link/?url=https%3A%2F%2Fsrinimf.com%2F2024%2F07%2F20%2Fpyspark-when-check-numeric-of-column-tested%2F&media=https%3A%2F%2Fi0.wp.com%2Fsrinimf.com%2Fwp-content%2Fuploads%2F2024%2F07%2Fimage-1.png%3Ffit%3D1024%252C768%26ssl%3D1&description=PySpark%20When%20Check%20Numeric%20of%20Column%20%5BTested%5D)
-
- [Share on Tumblr](https://www.tumblr.com/share)
- [Share on WhatsApp (Opens in new window) WhatsApp](https://srinimf.com/2024/07/20/pyspark-when-check-numeric-of-column-tested/?share=jetpack-whatsapp)
- [More](#)
-

- [Print (Opens in new window) Print](https://srinimf.com/2024/07/20/pyspark-when-check-numeric-of-column-tested/?share=print)
- [Share on Reddit (Opens in new window) Reddit](https://srinimf.com/2024/07/20/pyspark-when-check-numeric-of-column-tested/?share=reddit)
- [Share on Telegram (Opens in new window) Telegram](https://srinimf.com/2024/07/20/pyspark-when-check-numeric-of-column-tested/?share=telegram)
- [Email a link to a friend (Opens in new window) Email](mailto:?subject=%5BShared%20Post%5D%20PySpark%20When%20Check%20Numeric%20of%20Column%20%5BTested%5D&body=https%3A%2F%2Fsrinimf.com%2F2024%2F07%2F20%2Fpyspark-when-check-numeric-of-column-tested%2F&share=email)
-

### Like this:

Like Loading…