Make group syncing optional?

[danielcompton]

Hi there

I work for Whimsical (https://whimsical.com). We have a mutual customer that would like to use ConductorOne to manage their Whimsical users. I had a look at using baton-scim to integrate with our SCIM API.

I was able to create a config to fetch from our SCIM API, but I got an error when baton-scim tried to load groups:

{"level":"error","ts":1753761833.80499,"caller":"connectorrunner/runner.go:182","msg":"runner: error processing on-demand task","error":"runner: error processing task: rpc error: code = NotFound desc = error: listing resources failed: baton-scim: error fetching groups: error fetching SCIM groups: rpc error: code = NotFound desc = 404 Not Found","task_id":"","task_type":"sync_full"}
{"level":"error","ts":1753761833.805001,"caller":"cli/commands.go:178","msg":"error running connector","error":"runner: error processing task: rpc error: code = NotFound desc = error: listing resources failed: baton-scim: error fetching groups: error fetching SCIM groups: rpc error: code = NotFound desc = 404 Not Found"}
runner: error processing task: rpc error: code = NotFound desc = error: listing resources failed: baton-sci

Is it possible to disable group syncing? When I tried to remove the config from the file I got a validation error.

[ggreer]

Can you share the config? (Minus any sensitive info, of course.) I thought Whimsical let you fetch groups, just not provision them with SCIM. https://help.whimsical.com/security/scim#scim-basics-for-different-providers says "Group provisioning is not supported."

If group fetching is not possible, I can change the config validation so that the group section isn't required, and group syncing will be skipped if it's omitted.

[ggreer]

#13 should fix this issue. If you don't specify groups in the config, baton-scim will skip syncing groups. v0.0.4 has this change. https://github.com/ConductorOne/baton-scim/releases/tag/v0.0.4

[danielcompton]

I thought Whimsical let you fetch groups, just not provision them with SCIM. help.whimsical.com/security/scim#scim-basics-for-different-providers says "Group provisioning is not supported."

Sorry, we will clarify that, neither group fetching nor provisioning is supported.

I'm not 100% I've got all the config correct, but here's what I was using:

apiEndpoint: "https://api.whimsical.com/scim-v2"
hasScimHeader: true
auth:
  authType: "apiKey"
  apiKeyPrefix: "Bearer"
user:
  id: "id"
  userName: "userName"
  displayName: "displayName"
  firstName: "name.givenName"
  lastName: "name.familyName"
  primaryEmail: "emails[?(@.primary==true)].value"
  firstEmail: "emails[0].value"
  active: "active"
  roles:
    path: "roles"
    name: "value"
    display: "display"
group:
  id: "id"
  displayName: "displayName"
  members:
    path: "members"
    id: "value"
    displayName: "display"
pagination:
  totalResults: "totalResults"
  itemsPerPage: "itemsPerPage"
  startIndex: "startIndex"
provisioning:
  addUserRole:
    schemas: "urn:ietf:params:scim:api:messages:2.0:PatchOp"
    op: "Replace"
    path: "roles.value"
    valuePath: "value"
  removeUserRole:
    schemas: "urn:ietf:params:scim:api:messages:2.0:PatchOp"
    op: "Replace"
    path: "roles.value"
    valuePath: "value"

[danielcompton]

@ggreer thanks for that. I used this config file:

apiEndpoint: "https://api.whimsical.com/scim-v2"
hasScimHeader: true
auth:
  authType: "apiKey"
  apiKeyPrefix: "Bearer"
user:
  id: "id"
  userName: "userName"
  displayName: "displayName"
  firstName: "name.givenName"
  lastName: "name.familyName"
  primaryEmail: "emails[?(@.primary==true)].value"
  firstEmail: "emails[0].value"
  active: "active"
  roles:
    path: "roles"
    name: "value"
    display: "display"
pagination:
  totalResults: "totalResults"
  itemsPerPage: "itemsPerPage"
  startIndex: "startIndex"
provisioning:
  addUserRole:
    schemas: "urn:ietf:params:scim:api:messages:2.0:PatchOp"
    op: "Replace"
    path: "roles.value"
    valuePath: "value"
  removeUserRole:
    schemas: "urn:ietf:params:scim:api:messages:2.0:PatchOp"
    op: "Replace"
    path: "roles.value"
    valuePath: "value"

And got these logs:

Details

2025-08-21T06:49:59.149+1200    DEBUG   logging/logging.go:70   Logger created! {"log_level": "debug"}
2025-08-21T06:49:59.150+1200    DEBUG   connectorrunner/runner.go:158   runner: claiming worker
2025-08-21T06:49:59.150+1200    DEBUG   connectorrunner/runner.go:165   runner: worker claimed, checking for next task
2025-08-21T06:49:59.150+1200    DEBUG   connectorrunner/runner.go:192   runner: got task        {"task_id": "", "task_type": "sync_full"}
2025-08-21T06:49:59.150+1200    DEBUG   connectorrunner/runner.go:196   runner: one-shot mode enabled. Performing action synchronously.
2025-08-21T06:49:59.153+1200    DEBUG   connector/connector_server_unix.go:39   listener started        {"listen_port": 56279, "listener_fd": 9}
2025-08-21T06:49:59.168+1200    DEBUG   logging/logging.go:70   Logger created! {"log_level": "debug"}
2025-08-21T06:49:59.168+1200    DEBUG   cli/commands.go:397     starting grpc server
2025-08-21T06:49:59.168+1200    DEBUG   connector/connector_server_unix.go:47   starting listener with fd       {"expected_listen_port": 56279}
2025-08-21T06:49:59.168+1200    DEBUG   connector/connector_server_unix.go:59   listener fd     {"fd": 3}
2025-08-21T06:49:59.168+1200    DEBUG   connector/connector_server_unix.go:71   listener started        {"listen_port": 56279}
2025-08-21T06:49:59.168+1200    INFO    grpclog/component.go:34 [core][Server #1]Server created {"system": "grpc", "grpc_log": true}
2025-08-21T06:49:59.168+1200    INFO    grpclog/component.go:34 [core][Server #1 ListenSocket #2]ListenSocket created   {"system": "grpc", "grpc_log": true}
2025-08-21T06:49:59.171+1200    INFO    grpclog/component.go:34 [core]CPU time info is unavailable on non-linux environments.   {"system": "grpc", "grpc_log": true}
2025-08-21T06:49:59.171+1200    DEBUG   local/local.go:94       successfully loaded c1z locally {"file_path": "sync.c1z", "temp_path": "/var/folders/d3/sd0zhwhj7pg3q596qc88hsnm0000gn/T/sync-1071058882.c1z"}
2025-08-21T06:49:59.174+1200    DEBUG   ratelimit/grpc.go:100   beginning ratelimit check for request   {"method": "/c1.connector.v2.ConnectorService/Validate"}
2025-08-21T06:49:59.174+1200    DEBUG   ugrpc/logging.go:35     finished unary call with code OK        {"grpc.start_time": "2025-08-21T06:49:59+12:00", "grpc.service": "c1.ratelimit.v1.RateLimiterService", "grpc.method": "Do", "peer.address": "127.0.0.1:56280", "grpc.code": "OK", "grpc.time_ms": 0.01}
2025-08-21T06:49:59.175+1200    DEBUG   ratelimit/grpc.go:119   ratelimit ok - calling method   {"method": "/c1.connector.v2.ConnectorService/Validate"}
2025-08-21T06:49:59.175+1200    DEBUG   uhttp/wrapper.go:362    http cache miss {"grpc.start_time": "2025-08-21T06:49:59+12:00", "grpc.service": "c1.connector.v2.ConnectorService", "grpc.method": "Validate", "peer.address": "127.0.0.1:56280", "url": "https://api.whimsical.com/scim-v2/Users?count=1&startIndex=1"}
2025-08-21T06:49:59.175+1200    DEBUG   uhttp/transport.go:141  Request started {"http.method": "GET", "http.url_details.host": "api.whimsical.com", "http.url_details.path": "/scim-v2/Users", "http.url_details.query": "count=1&startIndex=1"}
2025-08-21T06:49:59.996+1200    DEBUG   uhttp/transport.go:173  Request complete        {"http.method": "GET", "http.url_details.host": "api.whimsical.com", "http.url_details.path": "/scim-v2/Users", "http.url_details.query": "count=1&startIndex=1", "http.status_code": 200, "http.headers": {}}
2025/08/21 06:49:59 DEBUG: SCIM RESPONSE Status: 200, Body: {
  "totalResults" : 2,
  "itemsPerPage" : 1,
  "startIndex" : 1,
  "schemas" : [ "urn:ietf:params:scim:api:messages:2.0:ListResponse" ],
  "Resources" : [ {
    "schemas" : [ "urn:ietf:params:scim:schemas:core:2.0:User", "urn:ietf:params:scim:schemas:extension:whimsical:2.0:User" ],
 ...
2025-08-21T06:49:59.998+1200    DEBUG   ugrpc/logging.go:35     finished unary call with code OK        {"grpc.start_time": "2025-08-21T06:49:59+12:00", "grpc.service": "c1.connector.v2.ConnectorService", "grpc.method": "Validate", "peer.address": "127.0.0.1:56280", "grpc.code": "OK", "grpc.time_ms": 822.648}
2025-08-21T06:49:59.999+1200    DEBUG   sync/syncer.go:363      resuming previous sync  {"sync_id": "31Yz8VUDTeEgUIYkMQLGYNM1vMw"}
2025-08-21T06:50:00.000+1200    INFO    sync/syncer.go:623      Syncing resource types...
2025-08-21T06:50:00.000+1200    DEBUG   ratelimit/grpc.go:100   beginning ratelimit check for request   {"method": "/c1.connector.v2.ResourceTypesService/ListResourceTypes"}
2025-08-21T06:50:00.000+1200    DEBUG   ugrpc/logging.go:35     finished unary call with code OK        {"grpc.start_time": "2025-08-21T06:50:00+12:00", "grpc.service": "c1.ratelimit.v1.RateLimiterService", "grpc.method": "Do", "peer.address": "127.0.0.1:56280", "grpc.code": "OK", "grpc.time_ms": 0.011}
2025-08-21T06:50:00.000+1200    DEBUG   ratelimit/grpc.go:119   ratelimit ok - calling method   {"method": "/c1.connector.v2.ResourceTypesService/ListResourceTypes"}
2025-08-21T06:50:00.001+1200    DEBUG   ugrpc/logging.go:35     finished unary call with code OK        {"grpc.start_time": "2025-08-21T06:50:00+12:00", "grpc.service": "c1.connector.v2.ResourceTypesService", "grpc.method": "ListResourceTypes", "peer.address": "127.0.0.1:56280", "grpc.code": "OK", "grpc.time_ms": 0.017}
2025-08-21T06:50:00.001+1200    DEBUG   ugrpc/logging.go:35     finished unary call with code OK        {"grpc.start_time": "2025-08-21T06:50:00+12:00", "grpc.service": "c1.ratelimit.v1.RateLimiterService", "grpc.method": "Report", "peer.address": "127.0.0.1:56280", "grpc.code": "OK", "grpc.time_ms": 0.005}
2025-08-21T06:50:00.001+1200    INFO    sync/syncer.go:81       Synced resource types   {"count": 2}
2025-08-21T06:50:00.001+1200    DEBUG   sync/state.go:258       finishing action        {"action": {"operation":"list-resource-types"}}
2025-08-21T06:50:00.001+1200    INFO    sync/syncer.go:783      Syncing resources...
2025-08-21T06:50:00.001+1200    DEBUG   sync/state.go:258       finishing action        {"action": {"operation":"list-resources"}}
2025-08-21T06:50:00.001+1200    DEBUG   sync/state.go:252       pushing action  {"action": {"operation":3,"resource_type_id":"user"}}
2025-08-21T06:50:00.001+1200    DEBUG   sync/state.go:252       pushing action  {"action": {"operation":3,"resource_type_id":"role"}}
2025-08-21T06:50:00.002+1200    DEBUG   sync/state.go:252       pushing action  {"action": {"operation":3,"resource_type_id":"group"}}
2025-08-21T06:50:00.002+1200    DEBUG   ratelimit/grpc.go:100   beginning ratelimit check for request   {"method": "/c1.connector.v2.ResourcesService/ListResources"}
2025-08-21T06:50:00.002+1200    DEBUG   ugrpc/logging.go:35     finished unary call with code OK        {"grpc.start_time": "2025-08-21T06:50:00+12:00", "grpc.service": "c1.ratelimit.v1.RateLimiterService", "grpc.method": "Do", "peer.address": "127.0.0.1:56280", "grpc.code": "OK", "grpc.time_ms": 0.003}
2025-08-21T06:50:00.002+1200    DEBUG   ratelimit/grpc.go:119   ratelimit ok - calling method   {"method": "/c1.connector.v2.ResourcesService/ListResources"}
2025-08-21T06:50:00.002+1200    ERROR   ugrpc/logging.go:35     finished unary call with code Unknown   {"grpc.start_time": "2025-08-21T06:50:00+12:00", "grpc.service": "c1.connector.v2.ResourcesService", "grpc.method": "ListResources", "peer.address": "127.0.0.1:56280", "error": "error: list resources with unknown resource type group", "grpc.code": "Unknown", "grpc.time_ms": 0.008}
2025-08-21T06:50:00.002+1200    DEBUG   ugrpc/logging.go:35     finished unary call with code OK        {"grpc.start_time": "2025-08-21T06:50:00+12:00", "grpc.service": "c1.ratelimit.v1.RateLimiterService", "grpc.method": "Report", "peer.address": "127.0.0.1:56280", "grpc.code": "OK", "grpc.time_ms": 0.001}
2025-08-21T06:50:00.002+1200    ERROR   ratelimit/grpc.go:134   ratelimit: error running client request {"error": "rpc error: code = Unknown desc = error: list resources with unknown resource type group"}
2025-08-21T06:50:00.004+1200    DEBUG   local/local.go:135      successfully saved c1z locally  {"file_path": "sync.c1z", "temp_path": "/var/folders/d3/sd0zhwhj7pg3q596qc88hsnm0000gn/T/sync-1071058882.c1z", "bytes": 1397}
2025-08-21T06:50:00.004+1200    ERROR   connectorrunner/runner.go:200   runner: error processing on-demand task {"error": "runner: error processing task: rpc error: code = Unknown desc = error: list resources with unknown resource type group", "task_id": "", "task_type": "sync_full"}
2025-08-21T06:50:00.004+1200    ERROR   cli/commands.go:312     error running connector {"error": "runner: error processing task: rpc error: code = Unknown desc = error: list resources with unknown resource type group"}
runner: error processing task: rpc error: code = Unknown desc = error: list resources with unknown resource type group

Even though I didn't mention groups, it looks like it is still trying to do something with groups. Is there something else I need to do here?