You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Models of the `FULL` kind cause the dataset associated with a model to be fully refreshed (rewritten) upon each model evaluation.
@@ -392,7 +392,7 @@ TABLE db.menu_items (
392
392
393
393
### SCD Type 2 By Column
394
394
395
-
SCD Type 2 By Column supports sourcing from tables that do not have an "Updated At" timestamp defined in the table.
395
+
SCD Type 2 By Column supports sourcing from tables that do not have an "Updated At" timestamp defined in the table.
396
396
Instead, it will check the columns defined in the `columns` field to see if their value has changed and if so it will record the `valid_from` time as the execution time when the change was detected.
397
397
398
398
This example specifies a `SCD_TYPE_2_BY_COLUMN` model kind:
@@ -425,7 +425,7 @@ TABLE db.menu_items (
425
425
```
426
426
427
427
### Change Column Names
428
-
SQLMesh will automatically add the `valid_from` and `valid_to` columns to your table.
428
+
SQLMesh will automatically add the `valid_from` and `valid_to` columns to your table.
429
429
If you would like to specify the names of these columns you can do so by adding the following to your model definition:
430
430
```sql linenums="1" hl_lines="5-6"
431
431
MODEL (
@@ -470,7 +470,7 @@ When a record is added back, the new record will be inserted into the table with
470
470
* SCD_TYPE_2_BY_COLUMN: the `execution_time` when the record was detected again
471
471
472
472
One way to think about `invalidate_hard_deletes` is that, if enabled, deletes are most accurately tracked in the SCD Type 2 table since it records when the delete occurred.
473
-
As a result though, you can have gaps between records if the there is a gap of time between when it was deleted and added back.
473
+
As a result though, you can have gaps between records if the there is a gap of time between when it was deleted and added back.
474
474
If you would prefer to not have gaps, and a result consider missing records in source as still "valid", then you can set `invalidate_hard_deletes` to `false`.
475
475
476
476
### Example of SCD Type 2 By Time in Action
@@ -543,9 +543,9 @@ Target table will be updated with the following data:
**Note:**`Cheeseburger` was deleted from `2020-01-02 11:00:00` to `2020-01-03 00:00:00` meaning if you queried the table during that time range then you would not see `Cheeseburger` in the menu.
547
-
This is the most accurate representation of the menu based on the source data provided.
548
-
If `Cheeseburger` were added back to the menu with it's original updated at timestamp of `2020-01-01 00:00:00` then the `valid_from` timestamp of the new record would have been `2020-01-02 11:00:00` resulting in no period of time where the item was deleted.
546
+
**Note:**`Cheeseburger` was deleted from `2020-01-02 11:00:00` to `2020-01-03 00:00:00` meaning if you queried the table during that time range then you would not see `Cheeseburger` in the menu.
547
+
This is the most accurate representation of the menu based on the source data provided.
548
+
If `Cheeseburger` were added back to the menu with it's original updated at timestamp of `2020-01-01 00:00:00` then the `valid_from` timestamp of the new record would have been `2020-01-02 11:00:00` resulting in no period of time where the item was deleted.
549
549
Since in this case the updated at timestamp did not change it is likely the item was removed in error and this again most accurately represents the menu based on the source data.
550
550
551
551
@@ -621,8 +621,8 @@ Assuming your pipeline ran at `2020-01-03 11:00:00`, Target table will be update
**Note:**`Cheeseburger` was deleted from `2020-01-02 11:00:00` to `2020-01-03 11:00:00` meaning if you queried the table during that time range then you would not see `Cheeseburger` in the menu.
625
-
This is the most accurate representation of the menu based on the source data provided.
624
+
**Note:**`Cheeseburger` was deleted from `2020-01-02 11:00:00` to `2020-01-03 11:00:00` meaning if you queried the table during that time range then you would not see `Cheeseburger` in the menu.
625
+
This is the most accurate representation of the menu based on the source data provided.
626
626
627
627
### Shared Configuration Options
628
628
@@ -651,7 +651,7 @@ This is the most accurate representation of the menu based on the source data pr
651
651
652
652
#### Querying the current version of a record
653
653
654
-
Although SCD Type 2 models support history, it is still very easy to query for just the latest version of a record. Simply query the model as you would any other table.
654
+
Although SCD Type 2 models support history, it is still very easy to query for just the latest version of a record. Simply query the model as you would any other table.
655
655
For example, if you wanted to query the latest version of the `menu_items` table you would simply run:
0 commit comments