Force flag for parameters file path entry

[jikuja]

Is your feature request related to a problem? Please describe.
We have configured dev environment to use SHIR on some environment and remaining are using default AzureIR.

JSON snippets from DEV:

{
    "name": "xREST",
    "properties": {
        "annotations": [],
        "type": "RestService",
        "typeProperties": {
            // snip
        },
        "connectVia": {
            "referenceName": "SharedVMSHIR",
            "type": "IntegrationRuntimeReference"
        }
    }
}

{
    "name": "other",
    "properties": {
        "annotations": [],
        "type": "RestService",
        "typeProperties": {
            // snip
        }
        // no connectVia when default AzureIR configured on portal
    }
}

I would like to set default integration runtime on test/prod on all linked services and then override setting if needed.

First try
Parameter file:

linkedService,*,"-$.properties.connectVia",

Result: ERROR: ADFT0010: Wrong path defined in config for object(path): linkedService.*(properties.connectVia)

Second try
Parameter file(not sure if this is correct alternative for absent connectVia property):

linkedService,*,"+$.properties.connectVia","{'referenceName': 'AutoResolveIntegrationRuntime','type': 'IntegrationRuntimeReference'}"

Result: ERROR: Cannot add a member with the name "connectVia" because a member with that name already exists. To overwrite the member anyway, add the Force parameter to your command.

Third try
Parameter file:

linkedService,*,"$.properties.connectVia","{'referenceName': 'AutoResolveIntegrationRuntime','type': 'IntegrationRuntimeReference'}"

Result: ERROR: ADFT0010: Wrong path defined in config for object(path): linkedService.*(properties.connectVia)

Fourth try
Parameters file:

linkedService,xREST,"-$.properties.connectVia",

Result: No errors

So it looks like I need to edit linked services one-by-one basis instead of setting default value for all.

Describe the solution you'd like
Support force modifier. E.g.

Remove all properties.connectVia properties from linked services and ignore errors if properties.connectVia does not exist:

#linkedService,*,"-*$.properties.connectVia",

Add properties.connectVia property to all linked services and ignore errors if properties.connectVia already exists:

#linkedService,*,"+*$.properties.connectVia","{'referenceName': 'AutoResolveIntegrationRuntime','type': 'IntegrationRuntimeReference'}"

Modify properties.connectVia property to all linked services and ignore errors if properties.connectVia does not exists:

linkedService,*,"*$.properties.connectVia","{'referenceName': 'AutoResolveIntegrationRuntime','type': 'IntegrationRuntimeReference'}"

Describe alternatives you've considered
AFAIK only other option is to handle resource one-by-one. That adds maintenance burden with parameter files.

For this particular use case we just probably need to edit parameter files every time we move a new linked service to use SHIR and make deployment process to remove connectVia property

Additional context
N/A

[jikuja]

Let's put this on hold.

Just after ticket creation I checked code internals and found this:

azure.datafactory.tools/private/Update-PropertiesFromFile.ps1

Lines 122 to 136 in 467cd6e

	try {
	if ($action -ne "add") {
	Invoke-Expression "`$isExist = (`$null -ne `$json.properties.$path)"
	}
	}
	catch {
	$validPath = $false
	if ($option.FailsWhenPathNotFound -eq $false) {
	Write-Warning "Wrong path defined in config for object(path): $type.$name(properties.$path), skipping..."
	} else {
	$exc = ([System.Data.DataException]::new("ADFT0010: Wrong path defined in config for object(path): $type.$name(properties.$path)"))
	Write-Error -Exception $exc
	}
	}

Never mind. This only controls of deployment fails or prints warning. Maybe Test-AdfCode should have similar control but that off-topic for this ticket.

Also looks like Test-AdfCode cmdlet is hardcoded to fail with path errors:

azure.datafactory.tools/public/Test-AdfCode.ps1

Lines 123 to 137 in 467cd6e

	$adf.PublishOptions.FailsWhenConfigItemNotFound = $True
	$adf.PublishOptions.FailsWhenPathNotFound = $True
	$files | ForEach-Object {
	try {
	$FileName = $_.FullName
	Write-Host "Checking config file: $FileName..."
	Update-PropertiesFromFile -adf $adf -stage $FileName -ErrorVariable err -ErrorAction 'Stop' -dryRun:$True
	}
	catch {
	$result.ErrorCount += 1
	Write-Host "ERROR: $($_.Exception.Message)" -ForegroundColor 'Red'
	Write-Debug -Message $_.Exception
	#$_.Exception
	}
	}

[jikuja]

My workaround

• Disabled CI pipeline failure when Test-AdfCode failed
• Added $opt.FailsWhenPathNotFound = $false into CD
• Added linkedService,*,"-$.properties.connectVia", into test-parameter file