demos: SAS → Snowflake full migration demo — 5-act conversion, validation, and Snowsight confirmation#137
Conversation
…neering) Single linear demo thread showing Devin validating a SAS-to-Snowflake migration with programmatic confidence: orient over the SAS estate, validate one table live, catch the is_active filter divergence (271 missing rows, $773K control-total gap), fix it, then fan out across tables in parallel. Includes Snowsight confirmation and Streamlit dashboard exploration.
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| ``` | ||
| !validate-sas-to-snowflake | ||
|
|
||
| Validate the MONTHLY_AMB table migration from SAS to Snowflake. | ||
|
|
||
| - SAS table: MONTHLY_AMB (source: sample_data/MONTHLY_AMB.sas7bdat) | ||
| - Snowflake table: MONTHLY_AMB (target: sample_data/Scenario1/) | ||
| - Repo: Cognition-Partner-Workshops/uc-data-migration-sas-to-snowflake | ||
| ``` |
There was a problem hiding this comment.
🚩 External file paths referenced in prompts cannot be verified from this repo
The document references file paths on the external uc-data-migration-sas-to-snowflake repo (e.g., sample_data/MONTHLY_AMB.sas7bdat at line 130, .workshop/playbooks/sas-to-snowflake-migration.devin.md at line 13, .agents/skills/sas-to-snowflake-migration/SKILL.md at line 15, verify/reconcile.py at line 224). REVIEW.md requires "Every file path referenced in a prompt should exist on the referenced repo's main branch." These paths could not be verified from this repo alone — they depend on the external repo's current state.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Acknowledged — these paths exist on the companion repo's PR branch (uc-data-migration-sas-to-snowflake#8). Once that PR merges, all referenced paths (sample_data/MONTHLY_AMB.sas7bdat, .workshop/playbooks/..., .agents/skills/..., verify/reconcile.py) will be on main. The paths to files that already exist on main (sample_data/, lineage/, config/, app4.py, helper_functions.py) have been verified.
- Act 1: Orient over SAS estate - Act 2: Convert set-based SAS to Snowflake SQL (regulatory reporting) - Act 3: Convert procedural SAS to Snowpark Python (claims processing) - Act 4: Replace Control-M with Snowflake Tasks - Act 5: Validate with reconciliation harness (catch is_active divergence) - Snowsight confirmation: row counts, control totals, RWA, Task DAG - Fix Devin Review: MONTHLY_AMB schema RAW→STAGING
devin-ai-integration
Bot
changed the title
| ``` | ||
| Look at the BatchJobs/run_daily_banking.sas orchestrator in | ||
| ts-sas-legacy-analytics. It runs four SAS programs in sequence via | ||
| Control-M. Convert this to a Snowflake Task DAG that preserves the | ||
| same dependency chain. Write the DDL in | ||
| snowflake_sql/orchestration/tasks.sql in the | ||
| uc-data-migration-sas-to-snowflake repo. | ||
|
|
||
| Map the Control-M jobs: | ||
| BANK_DAILY_01 → load_customer_accounts (Daily 06:00) | ||
| BANK_DAILY_02 → daily_transactions (Daily 07:30, after 01) | ||
| BANK_WEEKLY_01 → credit_risk_scoring (Weekly Sun, after 02) | ||
| BANK_MONTHLY_01→ monthly_regulatory (Monthly 3rd BD, after 03) | ||
| INS_DAILY_01 → claims_processing (Daily 08:00, independent) | ||
| ``` |
There was a problem hiding this comment.
🚩 Act 4 prompt omits playbook invocation unlike Acts 2/3/5
The prompt at lines 219-233 (Act 4 — Replace Control-M with Snowflake Tasks) does not invoke the !convert-sas-to-snowflake playbook, while Acts 2, 3, and 5 all begin with that playbook command. This appears intentional: Act 4 is an orchestration task (converting batch scheduling) rather than a program conversion, so the conversion-and-validation playbook may not apply. However, it creates a slight inconsistency in the demo flow — a reviewer might want to confirm whether the orchestration conversion should also go through the playbook's validation gate.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Intentional — Act 4 is an orchestration/scheduling conversion, not a program-to-program conversion with a validation gate. The !convert-sas-to-snowflake playbook is scoped to "convert one SAS program and validate parity", which doesn't apply to Task DDL generation. The consistency difference is deliberate.
| Map the Control-M jobs: | ||
| BANK_DAILY_01 → load_customer_accounts (Daily 06:00) | ||
| BANK_DAILY_02 → daily_transactions (Daily 07:30, after 01) | ||
| BANK_WEEKLY_01 → credit_risk_scoring (Weekly Sun, after 02) | ||
| BANK_MONTHLY_01→ monthly_regulatory (Monthly 3rd BD, after 03) | ||
| INS_DAILY_01 → claims_processing (Daily 08:00, independent) | ||
| ``` | ||
|
|
||
| What Devin produces: | ||
|
|
||
| - **`snowflake_sql/orchestration/tasks.sql`** — a Task DAG: | ||
|
|
||
| ``` | ||
| TASK_DAILY_BANKING_ROOT (CRON 0 6 * * *) | ||
| └→ TASK_JOB01_LOAD_CUST_ACCOUNTS (AFTER ROOT) | ||
| └→ TASK_JOB02_DAILY_TRANSACTIONS (AFTER JOB01) | ||
| └→ TASK_JOB03_CALC_AMB (AFTER JOB02) | ||
| └→ TASK_JOB04_MONTHLY_REGULATORY (AFTER JOB03, conditional) | ||
| TASK_INS_CLAIMS_PROCESSING (CRON 0 8 * * *, Snowpark SP) | ||
| TASK_WEEKLY_CREDIT_RISK (CRON 0 2 * * SUN) | ||
| ``` |
There was a problem hiding this comment.
🚩 Act 4 prompt lists 5 Control-M jobs but produced DAG includes unlisted TASK_JOB03_CALC_AMB
The Act 4 prompt (lines 219-233) maps 5 Control-M jobs: BANK_DAILY_01, BANK_DAILY_02, BANK_WEEKLY_01, BANK_MONTHLY_01, INS_DAILY_01. The 'What Devin produces' DAG (lines 239-247) includes TASK_JOB03_CALC_AMB which has no corresponding entry in the prompt's job list. Additionally, the prompt says BANK_MONTHLY_01 depends on 'after 03', but the only candidate '03' in the prompt is BANK_WEEKLY_01 (credit risk scoring) — yet in the DAG, JOB03 is CALC_AMB (a different job). This is an internal documentation inconsistency: a user following along would see a task in the output that wasn't in their input. However, JOB03_CALC_AMB is referenced consistently elsewhere (Act 2 line 155, Act 5 line 288), so it's part of the demo's broader narrative — just not introduced in the Act 4 prompt. This pattern is common in demo docs where not every artifact is converted in a visible step.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Good catch on the inconsistency. TASK_JOB03_CALC_AMB is an inline batch step (not a named Control-M job) that sits between BANK_DAILY_02 and BANK_MONTHLY_01 in the actual SAS orchestration — it's computed as part of the daily chain but doesn't have its own Control-M entry. The "after 03" in the prompt refers to this implicit step. I'll add it to the prompt's job mapping to make it explicit and avoid the confusion.
1 participant
Summary
Presenter thread for the full SAS-to-Snowflake migration demo, expanded from validation-only (4 acts) to end-to-end migration (5 acts):
monthly_regulatory_reporting.sasbecomes Basel III risk weights, delinquency aging, capital adequacy in native Snowflake SQLclaims_processing.sas(hash lookups, multi-output, fraud routing) becomes a Snowpark stored procedureAFTERdependencies and CRON schedulesis_activefilter divergence (271 missing rows, $773K gap), fix, go green, fan out via child sessionsSnowsight confirmation section shows 5 live queries proving: row counts, control totals, risk-weighted assets, Task DAG status, and Snowpark procedure execution.
Bug fix:
MONTHLY_AMBschema corrected fromRAWtoSTAGINGin row-count verification SQL.Companion PR: Cognition-Partner-Workshops/uc-data-migration-sas-to-snowflake#8 (code artifacts)
Link to Devin session: https://partner-workshops.devinenterprise.com/sessions/6b016f528be64d3185e0d2df533860ad
Requested by: @bsmitches