Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 41 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ The main advantage of the module is the ability to publish all the Azure Data Fa
- [Stage value as full path to CSV config file](#stage-value-as-full-path-to-csv-config-file)
- [JSON format of Config file](#json-format-of-config-file)
- [Step: Deployment Plan](#step-deployment-plan)
- [Step: Stoping triggers](#step-stoping-triggers)
- [DryRun / Plan output and CI usage](#dryrun--plan-output-and-ci-usage)
- [Step: Stopping triggers](#step-stopping-triggers)
- [Step: Deployment of ADF objects](#step-deployment-of-adf-objects)
- [Step: Save deployment state](#step-save-deployment-state)
- [Step: Deleting objects not in source](#step-deleting-objects-not-in-source)
Expand Down Expand Up @@ -185,6 +186,7 @@ Publish-AdfV2FromJson
[-Option] <AdfPublishOption>
[-Method] <String>
[-DryRun] <Switch>
[-Plan] <Switch>
```

Assuming your ADF is named ```SQLPlayerDemo``` and the code is located in ```c:\GitHub\AdfName\```, replace the values for *SubscriptionName*, *ResourceGroupName*, *DataFactoryName* and run the following command using PowerShell CLI:
Expand Down Expand Up @@ -622,8 +624,45 @@ If you prefer using JSON rather than CSV for setting up configuration - JSON fil
This step identifies objects to be deployed using `Includes` and `Excludes` list provided in *Publish Options*.
Afterwards, if `IncrementalDeployment = true`, it excludes objects by comparing hashes from **Deployment State** to hashed of awaiting objects.

### DryRun / Plan output and CI usage

## Step: Stoping triggers
Use `-DryRun` or `-Plan` to preview intended changes without deploying. The process prints a terraform-like summary:

```text
Terraform-like plan (DryRun):
Plan: 0 to add, 1 to change, 1 to destroy.

~ Update
~ pipeline.PL_ExecSparkJob

- Delete
- pipeline.PL_Obsolete

= Unchanged / skipped
= dataset.DS_Json
```

The returned object contains a `DryRunPlan` property, so CI can enforce policy checks before deployment:

```powershell
$result = Publish-AdfV2FromJson `
-RootFolder $RootFolder `
-ResourceGroupName $ResourceGroupName `
-DataFactoryName $DataFactoryName `
-Location $Location `
-Option $opt `
-Plan

if ($result.DryRunPlan.Delete.Count -gt 0) {
Write-Error "Plan contains destroy actions. Review required before deployment."
exit 1
}

Write-Host "Plan accepted. Add=$($result.DryRunPlan.Create.Count); Change=$($result.DryRunPlan.Update.Count); Destroy=$($result.DryRunPlan.Delete.Count)"
```


## Step: Stopping triggers

💬 In log you'll see line: `STEP: Stopping triggers...`

Expand Down
2 changes: 1 addition & 1 deletion azure.datafactory.tools.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
RootModule = 'azure.datafactory.tools.psm1'

# Version number of this module.
ModuleVersion = '1.17.1'
ModuleVersion = '1.18.0'

# Supported PSEditions
# CompatiblePSEditions = @()
Expand Down
4 changes: 4 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [1.18.0] - 2026-07-01
### Added
* `Publish-AdfV2FromJson` now supports terraform-like planning via `-DryRun`/`-Plan`, including structured `DryRunPlan` output (Create/Update/Delete/Unchanged) #360

## [1.17.1] - 2026-07-01
### Fixed
* Further fixes of `DeleteNotInSource` with deserialization issue #480
Expand Down
4 changes: 3 additions & 1 deletion docs/ADVANCED/CMDLET_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ Publish-AdfV2FromJson `
[-Stage <String>] `
[-Option <AdfPublishOption>] `
[-Method <String>] `
[-DryRun]
[-DryRun] `
[-Plan]
```

**Parameters:**
Expand All @@ -33,6 +34,7 @@ Publish-AdfV2FromJson `
| Option | AdfPublishOption | No | Publish options object (filtering, triggers, etc.) |
| Method | String | No | Publishing method: 'AzDataFactory' or 'AzResource' (default) |
| DryRun | Switch | No | Validate without deploying |
| Plan | Switch | No | Alias behavior of DryRun with terraform-like naming |

**Returns:** AdfPublishOption (deployment result)

Expand Down
5 changes: 4 additions & 1 deletion docs/GUIDE/PUBLISHING.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,12 @@ Publish-AdfV2FromJson `
[-Stage] <String> `
[-Option] <AdfPublishOption> `
[-Method] <String> `
[-DryRun]
[-DryRun] `
[-Plan]
```

`-Plan` is an alias behavior of `-DryRun` and prints a terraform-like plan without deploying.

### Complete Example

```powershell
Expand Down
166 changes: 165 additions & 1 deletion private/ApplyExclusionOptions.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,9 @@ function ApplyExclusionOptions {

function ToBeDeployedStat {
param(
[Parameter(Mandatory=$True)] [Adf] $adf
[Parameter(Mandatory=$True)] [Adf] $adf,
[Parameter(Mandatory=$False)] $targetAdfInstance,
[switch] $TerraformStyle
)

$ToBeDeployedList = ($adf.AllObjects() | Where-Object { $_.ToBeDeployed -eq $true } | ToArray)
Expand All @@ -59,4 +61,166 @@ function ToBeDeployedStat {
Write-Host "- $($_.FullName($true))"
}

if (!$TerraformStyle) {
return $null
}

$plan = Get-DryRunPlan -adf $adf -targetAdfInstance $targetAdfInstance
Write-DryRunPlan -plan $plan
return $plan

}

function ConvertTo-CanonicalAdfType {
[CmdletBinding()]
param (
[parameter(Mandatory = $true)]
[String] $Type
)

$t = $Type.ToLowerInvariant()
switch ($t)
{
'pipeline' { return 'pipeline' }
'dataset' { return 'dataset' }
'dataflow' { return 'dataflow' }
'linkedservice' { return 'linkedService' }
'integrationruntime' { return 'integrationRuntime' }
'trigger' { return 'trigger' }
'factory' { return 'factory' }
'managedvirtualnetwork' { return 'managedVirtualNetwork' }
'managedprivateendpoint' { return 'managedPrivateEndpoint' }
'credential' { return 'credential' }
default { return $Type }
}
}

function Get-AdfObjectFullName {
[CmdletBinding()]
param (
[parameter(Mandatory = $true)]
$Object
)

$name = $Object.Name
$simtype = ''

if ($Object -is [AdfObject]) {
$simtype = Get-SimplifiedType -Type $Object.Type
if ($Object.Type -like 'Microsoft.DataFactory/factories*') {
$simtype = ConvertTo-AdfType -AzType $Object.Type
$simtype = Get-SimplifiedType -Type $simtype
}
}
else {
$simtype = Get-SimplifiedType -Type $Object.GetType().Name
}

$simtype = ConvertTo-CanonicalAdfType -Type $simtype
return "$simtype.$name"
}

function Get-DryRunPlan {
[CmdletBinding()]
param (
[parameter(Mandatory = $true)] [Adf] $adf,
[parameter(Mandatory = $false)] $targetAdfInstance
)

$create = [System.Collections.Generic.List[string]]::new()
$update = [System.Collections.Generic.List[string]]::new()
$delete = [System.Collections.Generic.List[string]]::new()
$unchanged = [System.Collections.Generic.List[string]]::new()

$sourceByName = @{}
$adf.AllObjects() | ForEach-Object {
$fullName = Get-AdfObjectFullName -Object $_
$sourceByName[$fullName] = $_
}

$targetByName = @{}
if ($null -ne $targetAdfInstance) {
$targetAdfInstance.AllObjects() | ForEach-Object {
$fullName = Get-AdfObjectFullName -Object $_
$targetByName[$fullName] = $_
}
}

$adf.AllObjects() | ForEach-Object {
$fullName = Get-AdfObjectFullName -Object $_
if ($_.ToBeDeployed -eq $false) {
$unchanged.Add($fullName) | Out-Null
}
elseif ($targetByName.ContainsKey($fullName)) {
$update.Add($fullName) | Out-Null
}
else {
$create.Add($fullName) | Out-Null
}
}

if ($null -ne $targetAdfInstance -and $adf.PublishOptions.DeleteNotInSource) {
$targetByName.Keys | ForEach-Object {
$fullName = $_
if (!$sourceByName.ContainsKey($fullName)) {
$oname = [AdfObjectName]::new($fullName)
$canDelete = $true
if ($adf.PublishOptions.DoNotDeleteExcludedObjects) {
$canDelete = !($oname.IsNameExcluded($adf.PublishOptions))
}
if ($canDelete) {
$delete.Add($fullName) | Out-Null
}
}
}
}

$create = @($create | Sort-Object)
$update = @($update | Sort-Object)
$delete = @($delete | Sort-Object)
$unchanged = @($unchanged | Sort-Object)

return [PSCustomObject]@{
Create = @($create)
Update = @($update)
Delete = @($delete)
Unchanged = @($unchanged)
}
}

function Write-DryRunPlan {
[CmdletBinding()]
param (
[parameter(Mandatory = $true)]
$plan
)

Write-Host "==================================================================================="
Write-Host "Terraform-like plan (DryRun):"
Write-Host "Plan: $($plan.Create.Count) to add, $($plan.Update.Count) to change, $($plan.Delete.Count) to destroy."

if ($plan.Create.Count -gt 0) {
Write-Host ""
Write-Host " + Create"
$plan.Create | ForEach-Object { Write-Host " + $_" }
}

if ($plan.Update.Count -gt 0) {
Write-Host ""
Write-Host " ~ Update"
$plan.Update | ForEach-Object { Write-Host " ~ $_" }
}

if ($plan.Delete.Count -gt 0) {
Write-Host ""
Write-Host " - Delete"
$plan.Delete | ForEach-Object { Write-Host " - $_" }
}

if ($plan.Unchanged.Count -gt 0) {
Write-Host ""
Write-Host " = Unchanged / skipped"
$plan.Unchanged | ForEach-Object { Write-Host " = $_" }
}

}
38 changes: 36 additions & 2 deletions public/Publish-AdfV2FromJson.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ AzResource method has been introduced due to bugs in Az.DataFactory PS module.
Optional switch parameter. When provided, process will not make any changes to target data factory but instead return the ADF object
that would be used in deployment.

.PARAMETER Plan
Optional switch parameter. Alias for DryRun with terraform-style intent. Behaves the same as DryRun.

.EXAMPLE
# Publish entire ADF
$ResourceGroupName = 'rg-devops-factory'
Expand Down Expand Up @@ -74,6 +77,10 @@ Publish-AdfV2FromJson -RootFolder "$RootFolder" -ResourceGroupName "$ResourceGro
# Execute dry run of intended publishing changes
Publish-AdfV2FromJson -RootFolder "$RootFolder" -ResourceGroupName "$ResourceGroupName" -DataFactoryName "$DataFactoryName" -Location "$Location" -Stage "UAT" -DryRun

.EXAMPLE
# Execute plan (alias of DryRun) to preview intended changes
Publish-AdfV2FromJson -RootFolder "$RootFolder" -ResourceGroupName "$ResourceGroupName" -DataFactoryName "$DataFactoryName" -Location "$Location" -Stage "UAT" -Plan

.LINK
Online version: https://github.com/SQLPlayer/azure.datafactory.tools/
#>
Expand Down Expand Up @@ -104,9 +111,16 @@ function Publish-AdfV2FromJson {
[String]$Method = 'AzResource',

[parameter(Mandatory = $false)]
[switch]$DryRun
[switch]$DryRun,

[parameter(Mandatory = $false)]
[switch]$Plan
)

if ($Plan.IsPresent) {
$DryRun = $true
}

$m = Get-Module -Name "azure.datafactory.tools"
$verStr = $m.Version.ToString(2) + "." + $m.Version.Build.ToString("000");
Write-Host "======================================================================================";
Expand All @@ -123,6 +137,7 @@ function Publish-AdfV2FromJson {
Write-Host "Options provided: $($null -ne $Option)";
Write-Host "Publishing method: $Method";
Write-Host "Is Dry Run?: $($DryRun.IsPresent)";
Write-Host "Is Plan Mode?: $($Plan.IsPresent)";
Write-Host "======================================================================================";
if ($null -ne $Option) {
Write-Host "Options:"
Expand Down Expand Up @@ -248,10 +263,29 @@ function Publish-AdfV2FromJson {
}
Write-Host "Found $unchanged_count unchanged object(s)."
}
ToBeDeployedStat -adf $adf

$dryRunPlan = $null
if ($DryRun.IsPresent) {
$targetAdfInstance = $null
Comment on lines +267 to +269
try {
Write-Host "DRY RUN: Loading target ADF objects to classify plan changes..."
$targetAdfInstance = Get-AdfFromService -FactoryName "$DataFactoryName" -ResourceGroupName "$ResourceGroupName"
}
catch {
Write-Warning "DRY RUN: Unable to load target ADF state for full plan classification. Falling back to local-only plan. Details: $_"
}

$dryRunPlan = ToBeDeployedStat -adf $adf -targetAdfInstance $targetAdfInstance -TerraformStyle
}
else {
ToBeDeployedStat -adf $adf
}

if ($DryRun.IsPresent) {
Write-Host "DRY RUN: Terminating script pre-deployment - inspect returned object to review planned changes."
if ($null -ne $dryRunPlan) {
$adf | Add-Member -MemberType NoteProperty -Name DryRunPlan -Value $dryRunPlan -Force
}
Write-Host "===================================================================================";
return $adf
}
Expand Down
Loading
Loading