Main - XDCR differ

[CodeJitsu42]

Opened a new PR to main

In this PR:

• The Dockerfile was enhanced.
• A new Pod CR was created.
• The script was updated.
• The entire setup was tested locally with success

[nelio2k]

Non-amdocs folks cannot reach this builder image.

ERROR: failed to solve: docker-registry-proxy.corp.amdocs.com/redhat/ubi9:9.5: failed to resolve source metadata for docker-registry-proxy.corp.amdocs.com/redhat/ubi9:9.5: failed to do request: Head "https://docker-registry-proxy.corp.amdocs.com/v2/redhat/ubi9/manifests/9.5": dialing docker-registry-proxy.corp.amdocs.com:443 container via direct connection because  has no HTTPS proxy: connecting to docker-registry-proxy.corp.amdocs.com:443: dial tcp: lookup docker-registry-proxy.corp.amdocs.com: no such host

Can this be a generic image, like before?
https://github.com/couchbase/xdcrDiffer/pull/116/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R1
ARG BUILDER_IMAGE=docker.io/golang:1.22

[nelio2k]

I think go.mod and go.sum should be included
Otherwise, I'm getting

------
 > [builder 3/6] RUN make all:
0.169 go build -o xdcrDiffer -v
0.175 go: cannot find main module, but found .git/config in /go
0.175 	to create a module there, run:
0.175 	go mod init
0.176 make: *** [Makefile:17: build] Error 1
------

[nelio2k]

@CodeJitsu42 - please note that f8575c1 has been checked in which includes your information from PR 116 combined with my findings on MB-65665.

Please rebase and give that a try, and see what other improvements you'd like to make on top.

Thanks.

[CodeJitsu42]

Hello @nelio2k , thank you.

I rebased and tested and things work fine.
There are a couple of additions in Dockerfile the use of http_proxy arg and its use as an env variable.

Additionally there were some attempts to make the tool run with TLS.
In the pod CR i mounted the certificate signed by the same RA as the one im using in couchbase cluster. Then after updating the ca-trust i was able to securely curl the cluster from that pod using the svc and the secure port.
Lastly i made some changes in the script to take enforceTLS param and then tried to run the below :

./runDiffer.sh -u user -p password***** -h plt-cbc-cluster-0001.plt-cbc-cluster.ocp-install-mesh.svc:18091 -r plt-cbc-cluster -s nfsanitybucket -t test -o /tmp --enforceTLS

However i got the below error:

Cleaning up before run... Waiting for PID 191 to terminate... differ is run with options: Options{sourceUrl: plt-cbc-cluster-0001.plt-cbc-cluster.ocp-install-mesh-pg16.svc:18091, sourceUsername: Administrator, sourcePassword: REDACTED, sourceBucketName: , remoteClusterName: , sourceFileDir: source, targetUrl: , targetUsername: , targetPassword: REDACTED, targetBucketName: , targetFileDir: target, numberOfSourceDcpClients: 1, numberOfWorkersPerSourceDcpClient: 64, numberOfTargetDcpClients: 1, numberOfWorkersPerTargetDcpClient: 64, numberOfWorkersForFileDiffer: 30, numberOfWorkersForMutationDiffer: 30, numberOfBins: 5, numberOfFileDesc: 500, completeByDuration: 0, completeBySeqno: true, checkpointFileDir: checkpoint, oldCheckpointFileName: , newCheckpointFileName: , fileDifferDir: fileDiff, mutationDifferDir: mutationDiff, mutationDifferBatchSize: 100, mutationDifferTimeout: 30, sourceDcpHandlerChanSize: 100000, targetDcpHandlerChanSize: 100000, bucketOpTimeout: 20, maxNumOfGetStatsRetry: 10, maxNumOfSendBatchRetry: 10, getStatsRetryInterval: 2, sendBatchRetryInterval: 500, getStatsMaxBackoff: 10, sendBatchMaxBackoff: 5, delayBetweenSourceAndTarget: 2, checkpointInterval: 600, runDataGeneration: true, runFileDiffer: true, runMutationDiffer: true, **enforceTLS: true**, bucketBufferCapacity: 100000, compareType: meta, mutationDifferRetries: 0, mutationDifferRetriesWaitSecs: 60, numOfFiltersInFilterPool: 32, debugMode: false, setupTimeout: 10, fileContaingXattrKeysForNoComapre: } **2025/03/28 10:01:05 revrpc: Got error (malformed HTTP response "\x15\x03\x03\x00\x02\x02") and will retry in 1s 2025-03-28T10:01:05.013Z ERRO GOXDCR.xdcrDiffTool: Failed to set authentication to request. err=CBAuth database is stale: last reason: malformed HTTP response "\x15\x03\x03\x00\x02\x02" Error creating difftool: Failed on calling /pools, err=CBAuth database is stale: last reason: malformed HTTP response "\x15\x03\x03\x00\x02\x02", statusCode=0**