Remove python folder #449

svij-sc · 2026-01-21T17:33:50Z

Scope of work done

Our source code has lived in GiGL/python for a while.
Although this was fine when the repo was first created, it has since became a headache as we need to make special considerations when handling it in code, when reading bundled yamls, resolution for IDEs, building docker images, et al.

This is also making it harder to bundle static files (non source code) in wheels; thus I am opting to remove it.

Where is the documentation for this feature?: N/A

Did you add automated tests or write a test plan?

Ran testing suite in comments below; addressed fixes as needed.

Updated Changelog.md? NO

Ready for code review?: YES

svij-sc · 2026-01-21T20:53:00Z

/help

github-actions · 2026-01-21T20:53:17Z

GiGL Automation

@ 20:53:17UTC :

🤖 Available PR Commands

You can trigger the following workflows by commenting on this PR:

/help - Checkout code
/unit_test_py - Run Python Unit Tests
/unit_test_scala - Run Scala Unit Tests
/integration_test - Run Integration Tests
/e2e_test - Run E2E Tests
/notebook_tests - Run Example Notebooks Tests
/lint_test - Run Linting Tests

💡 Usage: Simply comment on this PR with any of the commands above (e.g., /unit_test)

⏱️ Note: Commands may take some time to complete. Progress updates will be posted as comments.

…lder

svij-sc · 2026-01-21T21:00:25Z

/unit_test_py

svij-sc · 2026-01-21T21:00:31Z

/integration_test

github-actions · 2026-01-21T21:01:02Z

GiGL Automation

@ 21:01:02UTC : 🔄 Python Unit Test started.

@ 21:14:56UTC : ❌ Workflow failed.
Please check the logs for more details.

github-actions · 2026-01-21T21:01:06Z

GiGL Automation

@ 21:01:06UTC : 🔄 Integration Test started.

@ 21:18:15UTC : ❌ Workflow failed.
Please check the logs for more details.

github-actions · 2026-01-21T21:01:29Z

GiGL Automation

@ 21:01:29UTC : Starting to build base images for CUDA and CPU.
This may take a while, please be patient.
The images will be pushed to the GCP Artifact Registry.
Once done, the workflow will update the dep_vars.env file with the new image names.

…names

github-actions · 2026-01-21T21:11:57Z

GiGL Automation

@ 21:11:56UTC : Built and pushed new images:

CUDA base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-cuda-base:2b270be5b9e9aa0396d9ced365407e9f6a19a262.90.1
CPU base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-cpu-base:2b270be5b9e9aa0396d9ced365407e9f6a19a262.90.1
Dataflow base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-dataflow-base:2b270be5b9e9aa0396d9ced365407e9f6a19a262.90.1
Builder image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/gigl-base-images/gigl-builder:2b270be5b9e9aa0396d9ced365407e9f6a19a262.90.1

Updated dep_vars.env with new image names.
Updated .github/cloud_builder/run_command_on_pr_cloud_build.yaml with new builder image.

svij-sc · 2026-01-22T03:27:28Z

/unit_test_py

github-actions · 2026-01-22T03:27:43Z

GiGL Automation

@ 03:27:43UTC : 🔄 Python Unit Test started.

@ 04:22:11UTC : ✅ Workflow completed successfully.

svij-sc · 2026-01-22T03:27:46Z

/integration_test

github-actions · 2026-01-22T03:27:57Z

GiGL Automation

@ 03:27:57UTC : 🔄 Integration Test started.

@ 04:20:45UTC : ❌ Workflow failed.
Please check the logs for more details.

…lder

svij-sc · 2026-01-22T06:16:13Z

/integration_test

github-actions · 2026-01-22T06:16:28Z

GiGL Automation

@ 06:16:28UTC : 🔄 Integration Test started.

@ 07:23:00UTC : ✅ Workflow completed successfully.

github-actions · 2026-01-22T06:17:56Z

GiGL Automation

@ 06:17:56UTC : Starting to build base images for CUDA and CPU.
This may take a while, please be patient.
The images will be pushed to the GCP Artifact Registry.
Once done, the workflow will update the dep_vars.env file with the new image names.

…names

github-actions · 2026-01-22T06:26:49Z

GiGL Automation

@ 06:26:49UTC : Built and pushed new images:

CUDA base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-cuda-base:b34c863a2168c8df5a6da1f6385e5d374f0175d2.91.1
CPU base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-cpu-base:b34c863a2168c8df5a6da1f6385e5d374f0175d2.91.1
Dataflow base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-dataflow-base:b34c863a2168c8df5a6da1f6385e5d374f0175d2.91.1
Builder image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/gigl-base-images/gigl-builder:b34c863a2168c8df5a6da1f6385e5d374f0175d2.91.1

Updated dep_vars.env with new image names.
Updated .github/cloud_builder/run_command_on_pr_cloud_build.yaml with new builder image.

svij-sc · 2026-01-22T16:38:56Z

/e2e_test

github-actions · 2026-01-22T16:39:32Z

GiGL Automation

@ 16:39:32UTC : 🔄 E2E Test started.

@ 18:01:50UTC : ❌ Workflow failed.
Please check the logs for more details.

kmontemayor2-sc · 2026-01-22T17:04:23Z

/unit_test

github-actions · 2026-01-22T17:04:37Z

GiGL Automation

@ 17:04:36UTC : 🔄 Scala Unit Test started.

@ 17:13:14UTC : ✅ Workflow completed successfully.

github-actions · 2026-01-22T17:04:41Z

GiGL Automation

@ 17:04:41UTC : 🔄 Python Unit Test started.

@ 18:18:19UTC : ✅ Workflow completed successfully.

kmontemayor2-sc

Can we verify this works internally too?

kmontemayor2-sc · 2026-01-22T17:06:59Z

Makefile

Do we need to update the compile_protos rule too?

https://github.com/Snapchat/GiGL/blob/main/Makefile#L316

We do, let me look into it.
Thanks for the good eye.

kmontemayor2-sc · 2026-01-22T17:09:26Z

.gitignore

 .lib/
-python/gigl/env/customer.yaml
-python/gigl/deps/
+gigl/env/customer.yaml


lol what even is this file?

not blocking just curious :)

some legacy artifact that we moved away from years ago yet is still here.

kmontemayor2-sc · 2026-01-22T17:10:47Z

containers/Dockerfile.builder

Hmmm - in the end, what do we want the docker file structure to look like? Is this it?

There's still this /gigl_deps this we use.

GiGL deps will probably stick around as its more a directory that hosts the gigl python virtual env and deps for docker base images.

The GiGL source images are rebuilt every code change but deps remain static unless there is a new dep

kmontemayor2-sc · 2026-01-22T17:11:11Z

containers/Dockerfile.dataflow.src

-
+COPY gigl gigl
+COPY snapchat snapchat
+COPY tests tests


why do we need tests here?

ditto generally.

Unsure what the consequences of removing it are, and tbh I did not block myself on testing this as my goal was to keep existing logic and just delete the folder.
I was going to add a follow up task to see if we can remove this. Let me do this now: #456

kmontemayor2-sc · 2026-01-22T17:14:03Z

gigl/src/common/constants/local_fs.py

    """Returns the parent directory of GiGL"""
    file_path = Path(__file__)
-    path = LocalUri(file_path.parents[5])
+    path = LocalUri(file_path.parents[4])


unrelated but can we explain this magic number? Something like

GiGL/gigl/experimental/knowledge_graph_embedding/lib/constants/local.py

Line 5 in 563375d

) # gigl/experimental/knowledge_graph_embedding/

kmontemayor2-sc · 2026-01-22T17:15:28Z

scala_spark35/common/src/main/scala/snapchat/research/gbml/gbml_config/GbmlConfig.scala

@@ -4067,7 +4067,7 @@ object GbmlConfig extends scalapb.GeneratedMessageCompanion[snapchat.research.gb
    *   Path to modeling task spec class path to construct model for inference. Used for the subgraph-sampling-based inference process.
    * @param inferenceBatchSize
    *   Optional. If set, will be used to batch inference samples to a specific size before call for inference is made
-    *   Defaults to setting in python/gigl/src/inference/gnn_inferencer.py
+    *   Defaults to setting in igl/src/inference/gnn_inferencer.py


Suggested change

* Defaults to setting in igl/src/inference/gnn_inferencer.py

* Defaults to setting in gigl/src/inference/gnn_inferencer.py

nit.

kmontemayor2-sc · 2026-01-22T17:15:54Z

pyproject.toml

Do we need to update this too?

https://github.com/Snapchat/GiGL/blob/main/pyproject.toml#L253-L263

kmontemayor2-sc · 2026-01-22T17:18:39Z

tests/unit/main.py

Hmmm, I think it's a bit odd we have both tests/ and testing/ now. I remember there was a reason I named it testing. #185 (comment)

But it seems like that should no longer be an issue if we move testing to tests (or vise-versa).

Probably fine to do as a follow up fwiw.

svij-sc · 2026-01-22T17:58:48Z

@kmontemayor2-sc addressed ur comments.

Re:

Can we verify this works internally too?

See: go/gigl-issue/1465

kmontemayor2-sc

Have we verified that we can still release/etc with these changes?

We should probably have a version bump with this (and update CHANGELOG)

Otherwise, LGTM but I'd prefer we can verify everything works internally first :)

kmontemayor2-sc · 2026-01-22T19:42:48Z

tests/unit/utils/__init__.py

QQ why do we need this now?

/shrug.
I didn't touch this file.

Merging artifacts?
Nonetheless, this is already a common practive to have __init__.py file in every module/submodules.

svij-sc · 2026-01-22T20:10:49Z

Have we verified that we can still release/etc with these changes?

No, mostly because release (the wheel) is broken right now and this PR is a prerequisite to me fixing the release process.

We should probably have a version bump with this (and update CHANGELOG)

I am planning to have a version bump and subsequent release when the release wheel is fixed - which I am working on right now.

svij-sc · 2026-01-22T20:16:45Z

@kmontemayor2-sc

Otherwise, LGTM but I'd prefer we can verify everything works internally first :)

As more PRs go into GiGL the burden of work here to constantly update and test this scales tremendously as this will conflict with every* PR.

You can track the progress here go/gigl-issue/1465; which should be resolved soon

mkolodner-sc

Thanks Shubham! A few remaining python references from my search: 1, 2, 3, 4, 5, 6, 7

svij-sc · 2026-01-22T20:46:56Z

Thanks for the finds Matt.

Thanks Shubham! A few remaining python references from my search: 1, 2, 3, 4, 5, 6, 7

1: is expected as its on a specific commit/blob
2-7: fixed

github-actions · 2026-01-22T20:47:13Z

GiGL Automation

@ 20:47:13UTC : 🔄 E2E Test started.

@ 22:04:38UTC : ✅ Workflow completed successfully.

mkolodner-sc

Thanks Shubham! Generally LGTM conditioned on all testing passing

svij-sc added 2 commits January 21, 2026 17:32

remove python folder

c28d519

fixes for path mangling

e243101

Merge remote-tracking branch 'origin/main' into svij/remove-python-fo…

2b270be

…lder

[AUTOMATED] Update dep.vars, and other relevant files with new image …

f5c6589

…names

fix

65ba86e

svij-sc added 6 commits January 22, 2026 03:41

readd dost+partitioner_test

a7c3740

remove added fix

3d7bb7a

revert

fcd4f0f

Merge remote-tracking branch 'origin/main' into svij/remove-python-fo…

f586a5e

…lder

move

44c3cda

fix

b34c863

[AUTOMATED] Update dep.vars, and other relevant files with new image …

563375d

…names

svij-sc marked this pull request as ready for review January 22, 2026 16:37

svij-sc requested a review from kmontemayor2-sc as a code owner January 22, 2026 16:37

svij-sc requested review from mkolodner-sc, nshah-sc, xgao4-sc, yliu2-sc and zfan3-sc as code owners January 22, 2026 16:37

svij-sc changed the title ~~[WIP] remove python folder~~ Remove python folder Jan 22, 2026

kmontemayor2-sc reviewed Jan 22, 2026

View reviewed changes

pr comments + formatter

1c2badc

svij-sc mentioned this pull request Jan 22, 2026

Combine the two tests folder tests/ and testing/ #457

Open

nit

c4d721a

kmontemayor2-sc requested changes Jan 22, 2026

View reviewed changes

update changelog

0e1eed2

mkolodner-sc reviewed Jan 22, 2026

View reviewed changes

address comments

898fff1

mkolodner-sc approved these changes Jan 22, 2026

View reviewed changes

	* Defaults to setting in igl/src/inference/gnn_inferencer.py
	* Defaults to setting in gigl/src/inference/gnn_inferencer.py

Remove python folder #449

Are you sure you want to change the base?

Remove python folder #449

Conversation

svij-sc commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

svij-sc commented Jan 21, 2026

Uh oh!

github-actions bot commented Jan 21, 2026

GiGL Automation

🤖 Available PR Commands

Uh oh!

svij-sc commented Jan 21, 2026

Uh oh!

svij-sc commented Jan 21, 2026

Uh oh!

github-actions bot commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

github-actions bot commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

github-actions bot commented Jan 21, 2026

GiGL Automation

Uh oh!

github-actions bot commented Jan 21, 2026

GiGL Automation

Uh oh!

svij-sc commented Jan 22, 2026

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

svij-sc commented Jan 22, 2026

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

svij-sc commented Jan 22, 2026

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

github-actions bot commented Jan 22, 2026

GiGL Automation

Uh oh!

github-actions bot commented Jan 22, 2026

GiGL Automation

Uh oh!

svij-sc commented Jan 22, 2026

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

kmontemayor2-sc commented Jan 22, 2026

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GiGL Automation

Uh oh!

kmontemayor2-sc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

svij-sc commented Jan 21, 2026 •

edited

Loading

github-actions bot commented Jan 21, 2026 •

edited

Loading

github-actions bot commented Jan 21, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

svij-sc Jan 22, 2026 •

edited

Loading

svij-sc commented Jan 22, 2026 •

edited

Loading

svij-sc commented Jan 22, 2026 •

edited

Loading