Skip to main content
Skip table of contents

ETL+ Load Upsert Example 1 With 'Force Replace with Upsert'

Example of loading a loading table regularly serviced with Upsert with Load Replace. example. Shows how the Load Replace processes deleted source records.
🛠️ Work in progress

Load Upsert does not delete records. Load Replace does. This examples shows the results of a Upsert followed by a Replace. This example follows the Upsert example at ETL+ Load Upsert Example 1 .

NOTE: The Load type on the Job page supports periodically running Load Replace and Load All on tables. The support for Replace is the Force Replace with Upsert, default for others option on Load type.

Most Recent Load Upsert

Continued from ETL+ Load Upsert Example 1

Target Table Before Upsert

Description

Key

Watermark

row1

1

95

row2

2

90

row3

3

91

row4

4

93

Highwater mark = 93

Source Data

Description

Key

Watermark

row1

1

95

row3

3

91

row4

4

93

NOTES

  • Watermark on row1 > Highwater mark

  • Watermark on row4 = Highwater mark

  • row2 Deleted

Target Table After Upsert

Description

Key

Watermark

row1

1

95

row2

2

90

row3

3

91

row4

4

93

ETL Log: (2 rows upserted)


Periodically Refresh with Load Replace

The Force Replace with Upsert option on the Job Page supports periodically loading the Load Upsert table with Load Replace.

Target Table Before Replace

Description

Key

Watermark

row1

1

95

row2

2

90

row3

3

91

row4

4

93

ETL Log: (2 rows upserted)

Source Data

Same data loaded with the most recent Upsert example above.

Description

Key

Watermark

row1

1

95

row3

3

91

row4

4

93

NOTES

  • row2 recently deleted from the source.

Target Table After Replace

Description

Key

Watermark

row1

1

95

row3

3

91

row4

4

93

ETL Log: (3 rows replaced)

🛠️ Screenshots

Rough examples work in progress

image-20231227-040750.png

image-20231227-040519.png

image-20231227-040652.png

image-20231227-040934.png

image-20231227-041127.png

image-20231227-041159.png

image-20231227-041213.png

image-20231227-041258.png

image-20231227-041312.png

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.