---
title: "AWS Glue: Essential Job Parameters Explained"
description: "AWS Glue allows customization of job execution through various parameters, including job-specific, script, context, connection, environment-specific, and execution parameters, enhancing ETL processes effectively."
url: https://srinimf.com/2024/09/24/aws-glue-essential-job-parameters-explained/
date: 2024-09-24
modified: 2026-10-10
author: "Srini"
image: https://i0.wp.com/srinimf.com/wp-content/uploads/2024/09/use-flowervase-beside-work-table.png?fit=1024%2C768&ssl=1
categories: ["AWS"]
tags: ["AWS Glue Parameters"]
type: post
lang: en
---

# AWS Glue: Essential Job Parameters Explained

In AWS Glue, you can pass parameters to your jobs to control their behavior and customize their execution. Here’s a list of common parameters that can be used with [AWS Glue job](https://srinimf.com/2025/07/13/automating-aws-glue-job-trigger-from-s3-upload-via-eventbridge-and-lambda/)s.

![aws glue job parameters](https://i0.wp.com/srinimf.com/wp-content/uploads/2024/09/pexels-photo-3303058.jpeg?resize=867%2C1300&ssl=1)*Photo by Tuu1ea5n Kiu1ec7t Jr. on Pexels.com*

## **AWS Glue Job Parameters** **Examples**

### 1. **Job-Specific Parameters**

These parameters are specific to the job configuration.

- **`--job-name`**: The name of the job.
- **`--region`**: The AWS region where the job is being executed.
- **`--temp-dir`**: The directory in S3 to use for temporary files.
- **`--enable-continuous-cloudwatch-log`**: Whether to enable continuous logging to CloudWatch
- **`--job-language`**: The programming language of the job (Python or Scala).
- **`--glue-version`**: The version of AWS Glue (e.g., 1.0, 2.0, 3.0).
- **`--max-concurrent-runs`**: The maximum number of concurrent runs for the job.

### 2. **Script Parameters**

These parameters are often used to configure your ETL logic within the script.

- **`--source-table`**: The name of the source table (could be a database table).
- **`--destination-path`**: The S3 path where the output data should be stored.
- **`--bookmark-keys`**: The keys to use for tracking processed data (used with bookmarks).
- **`--input-format`**: The input data format (e.g., CSV, JSON, Parquet).
- **`--output-format`**: The output data format.

### 3. **AWS Glue Context Parameters**

These parameters can be used to configure the Glue context.

- **`--enable-glue-datacatalog`**: Enables the AWS Glue Data Catalog.
- **`--job-bookmark-option`**: Defines the job bookmark option (e.g., `job-bookmark-enable`, `job-bookmark-disable`).
- **`--transformation-context`**: Context for transformations.

### 4. **AWS Glue Connection Parameters**

Parameters for configuring connections to data sources.

- **`--connection-name`**: The name of the connection in the Glue Data Catalog.
- **`--connection-options`**: Connection options (like user, password, etc.).
- **`--jdbc-url`**: JDBC URL for the database connection.

### 5. **Environment-Specific Parameters**

Parameters to customize the job execution environment.

- **`--temp-dir`**: Path to the temporary directory in S3.
- **`--iam-role`**: The IAM role that AWS Glue uses [to access](https://srinimf.com/2016/11/20/the-3-new-ways-how-db2-give-access-to-currently-committed-data-ideas/) resources.

### 6. **Execution Parameters**

Parameters used during the execution of the job.

- **`--retry-attempts`**: The number of retry attempts for the job.
- **`--timeout`**: Maximum execution time in minutes before the job is automatically terminated.

### **Example of Passing Parameters**

You can pass parameters to an AWS Glue job in different ways. You can do so via the AWS Management Console, AWS CLI, or SDK. Here’s an example using the AWS CLI:

```
aws glue start-job-run --job-name my-glue-job 
--arguments '{"--source-table":"my_source_table", "--destination-path":"s3://my-bucket/output/"}'
```

### **Accessing Parameters in Glue Script**

In your Glue script (e.g., Python), you can access these parameters using the `getResolvedOptions` function:

```
from awsglue.utils import getResolvedOptions

args = getResolvedOptions(sys.argv, ['JOB_NAME', 'source-table', 'destination-path'])
source_table = args['source-table']destination_path = args['destination-path']
```

AWS Glue parameters are flexible and configurable for your ETL jobs, letting you customize execution to meet your needs.

Understanding [how to](https://srinimf.com/2024/09/03/how-to-delete-source-object-after-glue-job-run-complete/) effectively use these parameters will help you build efficient and maintainable Glue jobs.

In AWS Glue, you can add additional libraries. These can include Python packages or JAR files. Use the `--additional-python-modules` parameter for Python jobs.

Use `--extra-jars` for Scala jobs to add third-party libraries not included in the standard AWS Glue environment.

### **Adding Additional Libraries**

Here’s how you can specify additional libraries for your Glue job:

#### 1. **For Python Jobs**

To include additional Python libraries, use the `--additional-python-modules` parameter. You can specify multiple modules separated by commas.

**Example CLI Command**:

```
aws glue start-job-run --job-name my-glue-job 
--arguments '{"--additional-python-modules":"pandas==1.2.3,requests==2.25.1"}'
```

**Example Python Script**: You can also access the parameters within your Glue script using `getResolvedOptions`:

```
from awsglue.utils import getResolvedOptions
import sys

args = getResolvedOptions(sys.argv, ['JOB_NAME', 'additional-python-modules'])
additional_modules = args['additional-python-modules']
```

#### 2. **For Scala Jobs**

To include additional JAR files, use the `--extra-jars` parameter.

**Example CLI Command**:

```
aws glue start-job-run --job-name my-glue-job 
--arguments '{"--extra-jars":"s3://my-bucket/path/to/my-library.jar"}'
```

### Important Considerations

- **Library Availability**: Ensure the libraries are compatible with AWS Glue Python/Scala.
- **Package Versioning**: When specifying Python libraries, you can specify versions using `==`, `>=`, etc., to avoid compatibility issues.
- **Size Limitations**: Keep the total size of added libraries in mind, as there are limits on the script and additional libraries.
- **Dependency Management**: If your additional libraries have dependencies, ensure that they are resolved and included properly.

Adding additional libraries to your AWS Glue jobs is straightforward. Use `--additional-python-modules` for Python jobs. Use `--extra-jars` for Scala jobs. This allows you to extend the functionality of your ETL processes by leveraging external libraries.

To add additional Python libraries to your AWS Glue job, you can use the `--additional-python-modules` parameter. Examples include libraries such as `pypg`. Use this parameter when starting your Glue job. This parameter allows you to specify the libraries and their versions that you want to include.

### Using `--additional-python-modules`

Here’s how you can include `pypg` along with other libraries in your AWS Glue job.

#### Example CLI Command

If you want to run a Glue job and include `pypg` along with other libraries like `pandas` and `requests`, you would use the AWS CLI as follows:

```
aws glue start-job-run --job-name my-glue-job 
--arguments '{"--additional-python-modules":"pypg==<version>,pandas==<version>,requests==<version>"}'
```

Replace `<version>` with the specific version number you want to use (e.g., `1.0.0` for `pypg`). If you want the latest version, you can omit the version specifier.

### Example with Python Script

In your Glue script, you can access these parameters using `getResolvedOptions`:

```
from awsglue.utils import getResolvedOptions
import sys

# Retrieve job parameters
args = getResolvedOptions(sys.argv, ['JOB_NAME', 'additional-python-modules'])

# If you want to see the additional modules
additional_modules = args['additional-python-modules']print(f"Additional Python Modules: {additional_modules}")

# Your ETL logic here
# For example, import the libraries if necessary
# Depending on the library, you might import it here or use it in your logic.
```

### Important Notes

- **Library Availability**: Ensure that `pypg` and any other libraries are compatible with the AWS Glue Python environment.
- **Dependencies**: If `pypg` has additional dependencies, ensure they are also included in the `--additional-python-modules` parameter.
- **Testing**: It’s a good idea to test your job in a development environment first. Then deploy it to production. This ensures that all libraries are correctly loaded.

### Conclusion

By specifying `--additional-python-modules`, you can easily extend the functionality of your AWS Glue jobs with additional libraries like `pypg`. This is useful for incorporating specific features or capabilities not included in the default Glue environment.

References

- [AWS Glue job parameters](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)

### Share this:

-
- [Tweet](https://twitter.com/share)
- [![](https://i0.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png)](https://www.pinterest.com/pin/create/link/?url=https%3A%2F%2Fsrinimf.com%2F2024%2F09%2F24%2Faws-glue-essential-job-parameters-explained%2F&media=https%3A%2F%2Fi0.wp.com%2Fsrinimf.com%2Fwp-content%2Fuploads%2F2024%2F09%2Fuse-flowervase-beside-work-table.png%3Ffit%3D1024%252C768%26ssl%3D1&description=AWS%20Glue%3A%20Essential%20Job%20Parameters%20Explained)
-
- [Share on Tumblr](https://www.tumblr.com/share)
- [Share on WhatsApp (Opens in new window) WhatsApp](https://srinimf.com/2024/09/24/aws-glue-essential-job-parameters-explained/?share=jetpack-whatsapp)
- [More](#)
-

- [Print (Opens in new window) Print](https://srinimf.com/2024/09/24/aws-glue-essential-job-parameters-explained/?share=print)
- [Share on Reddit (Opens in new window) Reddit](https://srinimf.com/2024/09/24/aws-glue-essential-job-parameters-explained/?share=reddit)
- [Share on Telegram (Opens in new window) Telegram](https://srinimf.com/2024/09/24/aws-glue-essential-job-parameters-explained/?share=telegram)
- [Email a link to a friend (Opens in new window) Email](mailto:?subject=%5BShared%20Post%5D%20AWS%20Glue%3A%20Essential%20Job%20Parameters%20Explained&body=https%3A%2F%2Fsrinimf.com%2F2024%2F09%2F24%2Faws-glue-essential-job-parameters-explained%2F&share=email)
-

### Like this:

Like Loading…