Currently need to run through the following to make this workable with serverless:
- Remove compute/cluster details from resource .yml files (for serverless execution)
- Manually set the workspace Environment Verison to 2 in each notebook that is configured to run as a job. This cannot be configured within DAB yaml for Notebook tasks and resolves package import errors
- Remove ‘requirements.txt’ install to avoid unnecessary dependencies
- Remove ‘dbutils.library.restartPython()’ cells
- Resolves compute connection issues
Suggest making appropriate updates to the repo so serverless mlops-stacks is more turnkey. Also update the dependencies in MLOps Stacks to the latest versions (like MLflow 3). Also consider changing the jobs by default to serverless.
Currently need to run through the following to make this workable with serverless:
Suggest making appropriate updates to the repo so serverless mlops-stacks is more turnkey. Also update the dependencies in MLOps Stacks to the latest versions (like MLflow 3). Also consider changing the jobs by default to serverless.