Support for tags in InfluxDb

[mxturri]

Hi,
I wasn't able to find any way of adding tags in the data publication to influxdb. It's possible that I didn't look carefully enough.
I have two use cases that I am considering:

• define a set of static tags in the InfluxDbAgent configuration to be published for all data managed by the influx db agent
  This would be useful, for example, to distinguish between a production and a development environment

      {'agent-class': 'InfluxDBAgent',
      'instance-id': 'influxagent',
      'arguments': ['--initial-state', 'record',
                    '--host', 'influxdb',
                    '--port', 443,
                    '--protocol', 'line',
                    '--ssl', True,
                    '--verify-ssl', True,
                    '--database', 'ocs-feeds'],
      'tags': ['environment','prod']}

• add tags to the data published by individual monitoring clients
  The use case I have in mind is the following. At SLAC we are monitoring a handful of refrigeration units, all with the same set of monitoring clients, for example we have 4 separate CryomechCPAAgent instances. These agents are publishing the same data set in different buckets whose name is defined by the agent's instance-id. For example:

observatory.cryo0.Coolant_In_Temp
observatory.cryo1.Coolant_In_Temp
observatory.cryo2.Coolant_In_Temp
observatory.cryo3.Coolant_In_Temp

It would be nice if there were a way to publish all the agent's data in the same bucket, with the addition of a tag to differentiate the instance id:

observatory.cryo.Coolant_In_Temp with tag cryo='cryo0'
observatory.cryo.Coolant_In_Temp with tag cryo='cryo1'
observatory.cryo.Coolant_In_Temp with tag cryo='cryo2'
observatory.cryo.Coolant_In_Temp with tag cryo='cryo3'

Having tags in the data would allow much more flexibility when visualizing the data in grafana, having, for example, the same dashboard to visualize data from different sources though the use of drop-down variables to select the desired tags.

I realize this is not a trivial request.
Thanks for any feedback/insight/help you can provide.

[BrianJKoopman]

Hi @mxturri, apologies for the slow response here. We don't currently have any way of adding tags to the data published to InfluxDB via the ocs agent. Using tags more effectively is something I had wanted to address (#175), but once we deployed the existing scheme the inertia of using that scheme was difficult to overcome.

Our current approach defines a different 'address_root' per "platform" (similar your refrigeration units). This makes the "data buckets" unique like:

platform0.cryo.Coolant_In_Temp
platform1.cryo.Coolant_In_Temp
platform2.cryo.Coolant_In_Temp
platform3.cryo.Coolant_In_Temp

This 'address_root' is defined in ocs site config file (SCF). 'observatory' is just the default/example everywhere.

I can definitely see how adding tags as you suggest here would be useful, and do agree this isn't trivial. I'm open to adding them, we just need to be aware to preserve current functionality without the tags. I'm happy to discuss further ideas for implementation.

[BrianJKoopman]

Hi @mxturri, not sure if you're still using ocs or interested in this feature still, but I just wanted to let you know that improved tagging is in the works in #467. Comments welcome!

[mxturri]

@BrianJKoopman I just saw the pull request notification and am looking forward to trying it out.
It might take me a bit to get to it as I am currently busy with something else.
Thanks so much!

[BrianJKoopman]

Great! As written this won't change anything in existing agents. They need to be modified to publish the 'influxdb_tags' dict to their Feeds.

In practice we need to deal with updating agents that we want tags in and rewriting our relevant Grafana dashboards. So this might be kind of piecemeal at first, but the infrastructure is there for it.

[BrianJKoopman]

Tags are now officially supported in ocs v0.12.1.

No agents (other than the FakeDataAgent) currently make use of them, but the infrastructure now exists.