---
title: "Step-by-Step Guide to PySpark UDFs"
description: "In PySpark, user-defined functions simplify repetitive code. Three steps include creating, registering, and applying the UDF."
url: https://srinimf.com/2024/03/11/how-to-create-udf-in-pyspark-best-example/
date: 2024-03-11
modified: 2026-10-10
author: "Srini"
image: https://i0.wp.com/srinimf.com/wp-content/uploads/2024/03/python-code-for-data-science.png?fit=1024%2C768&ssl=1
categories: ["Data Science"]
tags: ["Create", "PySpark UDF", "Register"]
type: post
lang: en
---

# Step-by-Step Guide to PySpark UDFs

We can create functions in PySpark to make our code easier. There are three [steps to](https://srinimf.com/2014/02/11/5-steps-to-positive-attitude/) create a UDF in PySpark.

1. Creating a function
2. Register the function as udf
3. Apply the function to each column

![How to Create UDF in PySpark Best Example](https://i0.wp.com/srinimf.com/wp-content/uploads/2024/03/pexels-photo-1570264.jpeg?resize=867%2C1300&ssl=1)*Photo by REAFON GATES on Pexels.com*

## Creating UDF in PySpark

Convert the first letter of first_name and last_name to uppercase in the PySpark code example.

```
from pyspark.sql import SparkSession
from pyspark.sql.functions import udf
from pyspark.sql.types import StringType

# Create a Spark session
spark = SparkSession.builder.appName("Capitalize Names UDF").getOrCreate()

# Sample data
data = [(1, "john jones"), (2, "tracey smith"), (3, "amy sanders")]
columns = ["Seqno", "Name"]

# Create a DataFrame
df = spark.createDataFrame(data, columns)

# Define a Python function to capitalize the first letter [of each](https://srinimf.com/2014/08/12/oracle-plsql-step-by-step-guide/) word
def capitalize_name(name):
    return " ".join([word.capitalize() for word in name.split()])

# Register the function as a UDF
capitalize_name_udf = udf(capitalize_name, StringType())

# Apply the UDF to the Name column
df_transformed = df.withColumn("Name", capitalize_name_udf("Name"))

# Show the result
df_transformed.show()
```

## Output

```
Before Converting
+-----+------------+
|Seqno|Name        |
+-----+------------+
|1    |john jones  |
|2    |tracey smith|
|3    |amy sanders |
+-----+------------+

After Converting
+-----+-------------+
|Seqno|Name         |
+-----+-------------+
|1    |John Jones   |
|2    |Tracey Smith |
|3    |Amy Sanders  |
+-----+-------------+
```

### Share this:

-
- [Tweet](https://twitter.com/share)
- [![](https://i0.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png)](https://www.pinterest.com/pin/create/link/?url=https%3A%2F%2Fsrinimf.com%2F2024%2F03%2F11%2Fhow-to-create-udf-in-pyspark-best-example%2F&media=https%3A%2F%2Fi0.wp.com%2Fsrinimf.com%2Fwp-content%2Fuploads%2F2024%2F03%2Fpython-code-for-data-science.png%3Ffit%3D1024%252C768%26ssl%3D1&description=Step-by-Step%20Guide%20to%20PySpark%20UDFs)
-
- [Share on Tumblr](https://www.tumblr.com/share)
- [Share on WhatsApp (Opens in new window) WhatsApp](https://srinimf.com/2024/03/11/how-to-create-udf-in-pyspark-best-example/?share=jetpack-whatsapp)
- [More](#)
-

- [Print (Opens in new window) Print](https://srinimf.com/2024/03/11/how-to-create-udf-in-pyspark-best-example/?share=print)
- [Share on Reddit (Opens in new window) Reddit](https://srinimf.com/2024/03/11/how-to-create-udf-in-pyspark-best-example/?share=reddit)
- [Share on Telegram (Opens in new window) Telegram](https://srinimf.com/2024/03/11/how-to-create-udf-in-pyspark-best-example/?share=telegram)
- [Email a link to a friend (Opens in new window) Email](mailto:?subject=%5BShared%20Post%5D%20Step-by-Step%20Guide%20to%20PySpark%20UDFs&body=https%3A%2F%2Fsrinimf.com%2F2024%2F03%2F11%2Fhow-to-create-udf-in-pyspark-best-example%2F&share=email)
-

### Like this:

Like Loading…