In Databricks with Delta Lake, you can retrieve and compare the old and new versions of a table after an UPDATE operation using time travel or Delta Lake history.

Time travel

Step 1: Create a Table

CREATE OR REPLACE TABLE default.emp_1 (
id INT,
name STRING,
salary INT
) USING DELTA;

INSERT INTO default.emp_1 VALUES
(1, 'Alice', 50000),
(2, 'Bob', 60000),
(3, 'Charlie', 70000);

Step 2: Update the Table

UPDATE default.emp_1 
SET salary = salary + 5000
WHERE id = 2;

Step 3: Describe History

Query

DESCRIBE HISTORY default.emp_1;

Output

Output of history

Step 4: Compare Versions

SELECT old.id, old.name, old.salary AS old_salary, new.salary AS new_salary
FROM default.emp_1 VERSION AS OF 0 old
JOIN default.emp_1 new ON old.id = new.id
WHERE old.salary <> new.salary;

Compare result

🚀 Key Takeaways

✅ You use VERSION AS OF
DESCRIBE HISTORY table_name Shows all past versions.
✅ You can time travel to see old data without renaming tables.

Fediverse reactions

Discover more from Srinimf

Subscribe now to keep reading and get access to the full archive.

Continue reading