service

Service Declaration

Declare your service

1. Tell envoy mesh sidecar about your service/app
   • http listening port of your service/app
   • http health check path of your service/app
2. Tell envoy mesh sidecar how to announce your service on the service mesh
   • https listening port
   • http health check path

Example

  titanSideCars:
    envoy:
      clusters:
        local-myapp:
          # Settings of your local application
          port: 8080  
          healthChecks:
            path: /delta/status
        remote-myapp: # reserved keyword
          # Settings of your mesh sidecar proxy
          port: 9443 
        routes: # register your app routing path
        - match:
            prefix: /delta/ 

3. Enable ratelimit side-car on a per-service basis for services that require ratelimit

# service's values.yaml
titanSideCars:
  ratelimit:
    enabled: true

4. If service intends to ratelimit on payload attributes or token claims then it must also configure and enable OPA sidecar. See here for steps

---

Examples

Following examples guide through various ratelimit scenarios!

Example 1

Lets start with a simple example that ratelimits a matching route

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - limit: 100/minute   # supported units - second, minute, hour, day

Above configuration will limit all 'POST' requests starting with '/myapp/objects' to 100 per minute.

Example 2

Lets take a slightly more complex case where we ratelimit on each unique value of a header

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - descriptors:
          - key: x-product  # may also say header.x-myapp-header. 'header.' is default
          limit: 100/minute

In above configuration we again target POST requests starting with /myapp/objects. But now limit will be applied independently to each unique value of the header x-product.
Header x-product with values prod1 and prod2 each will be independently allowed 100 times per minute

Example 3

We can also ratelimit on an attribute from payload. Lets look at an example

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects 
        method: POST
      ratelimit:
        actions:
        - descriptors:
          - key: payload.email  # 'payload.' notation references payload attributes
          limit: 100/minute

Above config will trigger ratelimit on each unique email in the payload of POST requests starting with /myapp/objects. No ratelimiting will be triggered if payload does not contain the specified attribute.
A sample payload like {"name": "foo", "email": "foo@example.com"} will trigger above ratelimit action

Example 4

We can even ratelimit on a claim inside token as in following example

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects 
        method: POST
      ratelimit:
        actions:
        - descriptors:
          - key: token.jti      # 'token.' notation references token claims
          limit: 100/minute

With above config in place, a token with a specfic jti claim will be ratelimited to 100 requests per minute for POST requests starting with /myapp/objects

Example 5

Lets dig deeper and look at a more complex ratelimit action with multiple descriptors.

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - descriptors:
          - key: x-product
            eq: myprod
          - key: payload.email
          limit: 100/minute

In above config, the ratelimit has a single action with multiple descriptors. For an action to trigger, all its descriptors must match the incoming request.
A POST request starting with /myapp/objects and header x-product with value myprod will be ratelimited to 100 per minute for each unique value of email attribute in payload.

The order of descriptors is immaterial. Following config will have identical effect

    actions:
    - descriptors:
      - key: payload.email
      - key: x-product
        eq: myprod
      limit: 100/minute

Above may also be re-written as following with identical effect

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects/
        method: GET
        headers:
        - name: x-product
          exact: myprod
      ratelimit:
        actions:
        - descriptors:
          - key: payload.email
          limit: 100/minute

Instead of matching 'x-product' header in the descriptor it may be matched in the route defintion itself. Route defintion is implicitly part of every ratelimit action. Hence the above config has same effect as the original.

Comparision operation inside ratelimit action is more useful when dealing with attributes from token or payload as we will see in later examples. The routes[].match only allows header comparison.

Example 6

We now pick up a ratelimit config with multiple ratelimit actions

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - descriptors:
          - key: payload.email
          limit: 100/minute
        - descriptors:
          - key: payload.email
            eq: foo@example.com
          limit: 5/minute

Above targets POST requests starting with /myapp/objects and has two ratelimit actions. First action sets a limit of 100 per minute on each unique email in payload. The second action sets a limit of 5 per minute if email in payload equals foo@example.com.
Hence emails bar@example.com and car@example.com will be allowed 100 times per minute each, but foo@example.com will be restricted to 5 times per minute.

When multiple actions are configured, each action is evaluated indendendently. The most constraining action gets enforced. The order of actions is immaterial. Following config wll have identical effect.

  actions:
  - descriptors:
    - key: payload.email
      eq: foo@example.com
    limit: 5/minute
  - descriptors:
    - key: payload.email
    limit: 100/minute

Example 7

Lets look at an example where a request matches multiple route definitions

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
      ratelimit:
        actions:
        - limit: 100/minute      
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - limit: 10/minute

A POST request starting with /myapp/objects will match both the first and second route. Ratelimit actions on each route will be evaluated independently and the most constraining limit will be enforced (10 per minute in this case)

Example 8

Following example shows how to configure different limits for different environment without duplicating the ratelimit rules in each deployment

titanSideCars:
  # can be overwritten per environment
  ratelimit:
    limits: # holds custom key value pairs
      small: 10/hour
      large: 100/minute

  # configured in service's values.yaml
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - limit: small  # refers to a key inside titanSideCars.ratelimit.limits
    - match:
        prefix: /myapp/objects
        method: GET
      ratelimit:
        actions:
        - limit: large

---

Config Reference

Click to expand!

Ratelimit

Ratelimit sidecar settings. Merge over-writes global.titanSideCars.ratelimit

# titanSideCars.ratelimit.

    enabled:                    bool
    imageRegistry:              string
    imageName:                  string
    imageTag:                   string
    cpu:
      request:                  string
      limit:                    string
    memory:
      request:                  string
      limit:                    string
    ephemeralStorage:
      request:                  string
      limit:                    string
    livenessFailureThreshold:   integer
    readinessFailureThreshold:  integer

    logLevel:                   string
    redisUrl:                   string 

    # TODO other redis config items

enabled

(bool, default true) Set to false to disable ratelimit sidecar

imageRegistry

(string, optional) Docker image registry path used for ratelimit sidecar. Overrides titanSideCars.imageRegistry

imageName

(string, default ratelimit)

imageTag

(string, default latest)

cpu.request

(string, default 250m)

cpu.limit

(string, default 1)

memory.request

(string, default 256Mi)

memory.limit

(string, default 1Gi)

ephemeralStorage.request

(string, default 100Mi)

ephemeralStorage.limit

(string, default 500Mi)

livenessFailureThreshold

(integer, default 50)

readinessFailureThreshold

(integer, default 100)

logLevel

(string, default INFO)

redisUrl

(string, required)

---

PerRouteRatelimit

# titanSideCars.ingress.routes[].

  match:        RouteMatch
  ratelimit:
    enabled:    bool
    action:     []RatelimitAction

match

(RouteMatch, required)

ratelimit

(optional) Ratelimit policy for requests that match corresponding route definition.
If missing, the ratelimit template parser will totally skip processing of this route entry.

ratelimit.enabled

(bool, default true) Controls enforcement of supplied ratelimit actions

ratelimit.actions

([]RatelimitAction, required) One or more ratelimit actions on request that match corresponding route definiton. Request is matched against all actions and each matching action is evaluated independently.

---

RatelimitAction

# titanSideCars.ingress.routes[].ratelimit.actions[].

    descriptors: []RatelimitDescriptor
    limit: string

descriptors

([]RatelimitDescriptor, optional) List of descriptors that define the attributes to ratelimit on. Corresponding route definition is an implicit descriptor. Incoming request must match all descriptors to be considered for ratelimiting. The ordering of descriptors is not relevant.

limit

(string, required) Ratelimit value in <ratelimit-per-unit>/<unit> format. Supported units are second minute hour and day.

Example: 5/second, 50/minute, 100/hour, 10/day

Instead of supplying a hard-coded limit, limit can also refer to a key from titanSideCars.ratelimit.limits key-value pairs as in example below. This pattern allows for easy configuration of limits on a per environment basis.

Click to expand!

titanSideCars:
  ratelimit:
    limits:
      small: 100/day
  ingress:
    routes:
    - match: /
      ratelimit:
        actions:
        - limit: small

---

RatelimitDescriptor

# titanSideCars.ingress.routes[].ratelimit.actions[].descriptors[].

    key:    string

    # comparison operators
    eq:     string          # equals
    sw:     string          # starts-with
    ew:     string          # ends-with 
    co:     string          # contains
    lk:     string          # like
    pr:     bool            # present
    neq:    string          # not equals
    nsw:    string          # not starts-with
    new:    string          # not ends-with
    nco:    string          # not contains
    nlk:    string          # not like
    npr:    bool            # not present

key

(string, required) Attribute to rate-limit on. If no comparison operator is specified, ratelimit is performed on each unique value of the attribute. Key may refer to a header, token claim or an attribute from json payload.

A header may be referenced via header. notation. Example: header.x-request-id
A token claim may be referenced via token. notation. Example: token.sub.scope
A payload attribute may be referenced via payload. notation. Example: payload.userType

eq | sw | ew | co | lk | pr | neq | nsw | new | nco | nlk | npr

(oneof optional) Comparison operator. When specified, the ratelimit is performed on result of comparision.
For binary operators, the operator value indicates the right hand operand and should be a string.

The supported operators are

• eq/neq: Exact string match
• sw/nsw: String prefix match
• ew/new: String suffix match
• co/nco: Substring match
• lk/nlk: Regex match. Regex syntax is documented here
• pr/npr: Unary operator. Indicates if key is present or not. Only true value is used. Use pr to test presence and npr to test non presence.

---