Skip to content

Bugfix conversion of single activities in pipeline json files#49

Open
Alex-Swann wants to merge 10 commits into
Azure-Player:mainfrom
Alex-Swann:bugfix/pipeline-notebook-deployments
Open

Bugfix conversion of single activities in pipeline json files#49
Alex-Swann wants to merge 10 commits into
Azure-Player:mainfrom
Alex-Swann:bugfix/pipeline-notebook-deployments

Conversation

@Alex-Swann

@Alex-Swann Alex-Swann commented Mar 27, 2026

Copy link
Copy Markdown

Bugfix conversion of single activities in pipeline json files when handling dynamic variables.

If a pipeline has a variable that needs to be dynamic to accommodate this being utilised in different deployment environments (DEV, UAT, PROD), e.g.

{
	"name": "Run all notebooks",
	"properties": {
		"activities": [
			{
				"name": "ForEach Notebook",
				"type": "ForEach",
				"dependsOn": [],
				"userProperties": [],
				"typeProperties": {
					"items": {
						"value": "<an_array_of_string_values_exposed_via_@item()>",
						"type": "Expression"
					},
					"activities": [
						{
							"name": "Child - Pipeline",
							"type": "ExecutePipeline",
							"dependsOn": [],
							"policy": {
								"secureInput": false
							},
							"userProperties": [],
							"typeProperties": {
								"pipeline": {
									"referenceName": "Child - Pipeline",
									"type": "PipelineReference"
								},
								"waitOnCompletion": true,
								"parameters": {
									"StorageUrl": {
										"value": "@variables('StorageUrl')",
										"type": "Expression"
									}
								}
							}
						}
					]
				}
			}
		],
		"variables": {
			"StorageUrl": {
				"type": "String",
				"defaultValue": "test-url"
			}
		},
		"folder": {
			"name": "Core Code"
		},
		"annotations": []
	}
}

the serialisation and deserialisation process in Save-SynapseObjectAsFile.ps1 means the activities array, and any nested activities, are accidentally converted back into a object as opposed to an array of objects.

{
	"name": "Run all notebooks",
	"properties": {
		"activities": {
		          "name": "ForEach Notebook",
		          "type": "ForEach",
		          "dependsOn": [],
		          "userProperties": [],
		          "typeProperties": {
			  "items": {
				  "value": "<an_array_of_string_values_exposed_via_@item()>",
				  "type": "Expression"
			  },
			"activities": {
			        "name": "Child - Pipeline",
			        "type": "ExecutePipeline",
			        "dependsOn": [],
			        "policy": {
				        "secureInput": false
			        },
			        "userProperties": [],
			        "typeProperties": {
				        "pipeline": {
					        "referenceName": "Child - Pipeline",
					        "type": "PipelineReference"
				        },
				        "waitOnCompletion": true,
				        "parameters": {
					"StorageUrl": {
						"value": "@variables('StorageUrl')",
						    "type": "Expression"
					 }
				    }
			       }
		          }
		     }
		},
		"variables": {
			"StorageUrl": {
				"type": "String",
				"defaultValue": "test-url"
			}
		},
		"folder": {
			"name": "Core Code"
		},
		"annotations": []
	}
}

This fails Synapse's REST-API validations when attempting to deploy. This PR is to start the conversation around addressing it (or fix it altogether).

@Alex-Swann Alex-Swann changed the title Bugfix conversion of single activities in pipeline json files when ha… Bugfix conversion of single activities in pipeline json files Mar 27, 2026
@NowinskiK

Copy link
Copy Markdown
Member

Thank you, Alex for your contribution. Could you please add a unit test to cover the use case?

@Alex-Swann

Copy link
Copy Markdown
Author

Thank you, Alex for your contribution. Could you please add a unit test to cover the use case?

I hadn't spotted those. Thanks, I spotted an edge case too. Let me know if they align with both the fix and everything else the PS file is responsible for 👍

}
else {
$token = $SynapseToken.token
$token = $SynapseToken.Token ?? $SynapseToken.token

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid using operators supported in PS 7.x+ only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants