feat: Add per-service scaling & gRPC support#5934
feat: Add per-service scaling & gRPC support#5934jatin5251 wants to merge 26 commits intofeast-dev:masterfrom
Conversation
e340591 to
f7f7845
Compare
f7f7845 to
206c701
Compare
206c701 to
38da893
Compare
38da893 to
c87e4be
Compare
c87e4be to
9ff2621
Compare
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
5ff72f8 to
99b5ca7
Compare
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
|
@Srihari1192 Can we please create an image and test these operator changes ? |
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
…:jatin5251/feast into feat-feast-operator-mcp-grpc-per-service
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
| cmd = append(cmd, "--registry_ttl_sec", strconv.Itoa(int(*grpcCfg.RegistryTTLSeconds))) | ||
| } | ||
| } | ||
| if tls := feast.getTlsConfigs(OnlineGrpcFeastType); tls.IsTLS() { |
There was a problem hiding this comment.
feast listen doesn't support --key/--cert options
| }, | ||
| } | ||
| } | ||
| if feast.onlineGrpcOpenshiftTls() { |
There was a problem hiding this comment.
gRPC server has NO TLS support at all and here we are trying to auto-enable broken TLS
| } | ||
|
|
||
| func (feast *FeastServices) setContainers(podSpec *corev1.PodSpec) error { | ||
| func (feast *FeastServices) setContainers(podSpec *corev1.PodSpec, feastType FeastServiceType) error { |
There was a problem hiding this comment.
I see this is breaking change and there is no clear migration path from old deployment to new multiple deployments, single container each.
It keeps the old feast-my-store deployment. After upgrade:
- Old feast-my-store (4 containers) keeps running
- New feast-my-store-* (4-5 deployments) are created
- Both run simultaneously = conflicts, duplicate services, data corruption risks
|
@jatin5251 If possible can we divide the PR into 2 phases, mainly because this contains core architecture changes.
Thoughts? |
| func (feast *FeastServices) mountPvcConfig(podSpec *corev1.PodSpec, pvcConfig *feastdevv1.PvcConfig, feastType FeastServiceType) { | ||
| if podSpec != nil && pvcConfig != nil { | ||
| volName := feast.initPVC(feastType).Name | ||
| volName := feast.initPVC(feast.pvcFeastType(feastType)).Name |
There was a problem hiding this comment.
With separate PVCs, I think it will break file-based persistence. Each pod only has access to its own storage. While materialization with file-based (parquet files), Online pod can't access offline PVC.
There was a problem hiding this comment.
That requires a RWX PVC dynamic provisioner as a pre-requisite.
Multiple replicas / HA can be considered an advanced use case so I think that's a reasonable requirement.
Otherwise, not supporting non-distributed persistant storage should be documented.
| type GrpcServerConfigs struct { | ||
| ContainerConfigs `json:",inline"` | ||
| // Replicas sets the number of replicas for the gRPC service deployment. | ||
| Replicas *int32 `json:"replicas,omitempty"` |
There was a problem hiding this comment.
Compatibility with autoscalers like HPA or KEDA must be managed carefully.
With that PR, this new field will conflict with autoscaling updates to the deployment scale sub-resources, as described in kubernetes/kubernetes#25238.
It may be possible to work-around it using server-side apply, though I wonder whether we should make it more explicit in the API and support different scaling policies, like HPA or KEDA.
@jatin5251 I second @ntkathole it'd be helpful if you could break this PR along "atomic" functionalities. |
What this PR does / why we need it:
feature_serverconfig passthrough in the FeatureStore CRD to enable MCP support in the feature server config.Which issue(s) this PR fixes:
(If required i can create Github issue)
Misc