ADA, Databricks fundamentals and new Databricks page in Databricks fundamentals

[Lsnaathorst1]

Overview of changes

This PR is intended to cover stage 1 of several stages of changes to the AG Databricks guidance

Why are these changes being made?

These changes are being made to make the guidance more user friendly to navigate and to move all the Databricks and ADA pages into one area, with an extra page being added for Databricks in learning resources, to bring this in line with other pages (e.g., R, SQL, Git).

1. Review of the ADA page - shift stuff out of this page that isn't on ADA e.g., Databricks fundamentals to the fundamentals page, then there is material in there from What this means for existing code to existing links that can be moved out to databricks fundamentals for now (before moving to to its own page in stage 2)
2. Move all other Databricks pages to "ADA and Databricks " (rename Databricks set up guides)
3. Remove content from Databricks fundamentals where it appears on other Databricks pages i.e. Notebooks and workflows
4. Create Databricks in learning resources - adding in Databricks info from Learning resources > learning support and Databricks set up guides > Databricks - existing processes

Out of scope

This pull request will not cover creating the new page in ADA and Databricks on migration for existing processes. This will be covered in stage 2. It will also not cover adding in guidance on common errors, combining the script workflows pages or adding guidance in Python. These will be part of stage 3.

Detailed description of changes

• Moved content out of ADA page that is not ADA specific to Databricks fundamentals new page

Checklist before PR comes out of draft

• Create new Databricks page in L&R with content from learning support and existing processes
• Remove new page (this will be covered in stage 2 instead) and move other content to Databricks fundamentals for now
• Rename Databricks pages to ADA and Databricks and move all other Databricks pages to there.
• Sense check content as it now is on ADA, Databricks Fundamentals, is there anything we can remove or update
• Check for broken links: Checked databricks.qmd (last link broken)

Issue ticket number/s and link

Checklist before requesting a review

• I have checked the contributing guidelines
• I have checked for and linked any relevant issues that this may resolve
• I have checked that these changes build locally
• I understand that if merged into main, these changes will be publicly available