Draft: submit dummy Slurm job

[hannahle]

The goals of this PR are:

1. To test out job submission: If user submits a query for gene size >600,000bp, OSMP backend makes a call to the Slurm API to start a dummy job (echo Hello world!). Result can be seen on the working directory in the Slurm node.
2. To test that we can write a bash script to get access token and post a request with the slurm response payload to the OSMP server.
3. To test subscription: If gene size is > 600,000bp, we will open a websocket server, where the front-end and back-end communicates through the websocket. A [PubSub class](https://www.apollographql.com/docs/apollo-server/data/subscriptions/#the-pubsub-class) is created to subscribe and publish results to the front end. See documentation here on how Subscriptions are created in GraphQL Apollo.

Job Automation Flow
Before diving into the code, it is helpful to understand the high-level flow of this implementation.

• Once user submits a gene query, we calculate the gene size in getVariantResolver. If the gene size is >600,000bp, we submit a job request through the Slurm API. Currently, we are running a dummy job. The API does not actually kick-start the Nextflow job, though this will be integrated down the line.
• Once the result is outputted from the job, we are planning to have a bash script that will send the result back to OSMP. We are doing this by first obtaining a token from http://localhost:9821/auth/realms/ssmp/protocol/openid-connect/token (for local development), or https://keycloak.ccmdev.ca/auth/realms/ssmp/protocol/openid-connect/token in staging.

export TOKEN=$(curl --location --request POST 'http://localhost:9821/auth/realms/ssmp/protocol/openid-connect/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'password=secret' --data-urlencode 'username=ssmp-user' --data-urlencode 'client_id=ssmp-backend' --data-urlencode 'realm=ssmp' --data-urlencode 'grant_type=password' | jq -r '.access_token')

• When the access token is retrieved, we send a request with the slurm response payload to OSMP. An example of such payload is as follows:

curl --request POST \
  -H 'Content-Type: application/json' \
  --header "Authorization: Bearer $TOKEN" \
  --data '{"operationName":"OnSlurmResponse","variables":{ "jobId": 50881, "variants": [{"start":31655057,"end":31655057,"referenceName":6,"ref":"G","alt":"C","Consequence":"NON_SYNONYMOUS","oAA":"H","nAA":"D","FeatureID":"ENST00000395952","cDNApos":1772,"protPos":537,"nhomalt":"","an":"","af":"","filter":"","transcript":"","cdna":"","amino_acids":""},{"start":31655057,"end":31655057,"referenceName":6,"ref":"G","alt":"C","Consequence":"UPSTREAM","oAA":"","nAA":"","FeatureID":"ENST00000375863","cDNApos":"","protPos":"","nhomalt":"","an":"","af":"","filter":"","transcript":"","cdna":"","amino_acids":""},{"start":31655057,"end":31655057,"referenceName":6,"ref":"G","alt":"T","Consequence":"UPSTREAM","oAA":"","nAA":"","FeatureID":"ENST00000375863","cDNApos":"","protPos":"","nhomalt":"","an":"","af":"","filter":"","transcript":"","cdna":"","amino_acids":""},{"start":31655057,"end":31655057,"referenceName":6,"ref":"G","alt":"T","Consequence":"NON_SYNONYMOUS","oAA":"H","nAA":"N","FeatureID":"ENST00000395952","cDNApos":1772,"protPos":537,"nhomalt":"","an":"","af":"","filter":"","transcript":"","cdna":"","amino_acids":""},{"start":31655057,"end":31655057,"referenceName":6,"ref":"G","alt":"T","Consequence":"3PRIME_UTR","oAA":"","nAA":"","FeatureID":"ENST00000461287","cDNApos":2014,"protPos":"","nhomalt":"","an":"","af":"","filter":"","transcript":"","cdna":"","amino_acids":""}]}}' \
  http://localhost:5862/graphql

• The /graphql is responsible for handling all inbound requests. A subscription class is created to receive the data and publish it to the front end.
• Currently, we have a GraphQL query to return variants response. In this PR, I added a GraphQL subscription to return variant response submitted by the Slurm job.
• Depending on whether the gene size is greater or smaller than 600,000bp, the VariantQueryPage will display the result from the query or the result from the subscription. In the case of a subscription, the result returned by the query is an empty list, and the front end displays data that is pushed from the web socket server.

[hannahle]

I still need a way to somehow pass the jobId from Slurm to the front-end. Any ideas?

[kevinlul]

Didn't you make it part of OnSlurmResponse?

[hannahle]

Yeah, so the way this works is a little bit different.

When the page loads for the first time, useFetchVariantsSubscription will execute a subscription and starts listening for data from the server. If the server publishes data for the first time, useFetchVariantsSubscription will receive it. However, if the server publishes data for the second time, useFetchVariantsSubscription will actually still use the old data. The way Apollo subscription hook will "resubscribe" is when we pass in a different value in variables. That's why I'm passing a random number to the variables to trigger re-subscription. But ideally, we'd want to know the jobId in advance, which I haven't figured out how to do yet.

[jennyziyi-xu]

Note that it is strongly recommended to switch from PubSub class to PubSubEngine class in the production environment.
https://www.apollographql.com/docs/apollo-server/data/subscriptions/#the-pubsub-class