We have raised this issue to discuss a scenario that could arise for a Lucid SA that provides replication across servers for resilience, which means any one of a set of SA servers may be actively connected to the Lucid broker at one time but the MQTT connection could be moved to one of the other servers during a failover.
Lucid v3.0 says UP/data uses QoS 1 and Retained 0.
in section 6.1 it also says the MQTT features Clean Session/Clean Start are optional at both SA and FD.
We are assuming the SA wants to support any MQTT 3.1.1 or higher broker, therefore we cannot assume shared subscriptions are available as this feature only became mandatory in MQTT 5.
If SA does set cleanSession to false
Assuming the SA does set cleanSession to false (which is not the default) then it would mean, the active SA server driver drop its connection to the broker (non-gracefully - i.e. it does not unsubscribe to the topics) then the broker would try to queue the UP/data messages until the SA server reconnects. So in this case the Lucid driver would see any UP/data messages we may have missed whilst we lost comms with the broker.
SA Server Changeover timing complication
If the SA elects a different server to be the active (which now has the responsibility to connect to the Lucid Broker) and closes the connection from the previous active SA then the the timing between closing the previous connection and creating the new connection could risk loosing UP/data payloads. This is where FD's could publish data whist none of the SA servers are subscribed to the UP/data topics.
If SA leaves cleanSession set to true
If an SA chose to connect with cleanSession set to true (which is the default) then we think the SA would be more likely to lose data messages as above but also if the driver lost comms or restarted (without gracefully closing down the UP/data topic subscriptions).
The Lucide specification says It is up to the FD to ensure it gets the data to the Broker, but this means there is no application level acknowledgement (above MQTT layer) for the data topic.
SparkplugB
SparkplugB has seq and BDseq which provides some limited help in this area.
But as seen in the NCDM/Rebirth command SparkplugB cares more about the current (latest) value of its data points and sees the intermediate time series values as being less important.
But in the case of Lucid the Intermediate values could be alarms that need to be raise/cleared in the SA.
Proposed Workaround
This is based on the understanding that if two clients connect with the same MQTT ClientId then the later connection keeps the existing subscription and the first client is disconnected.
MQTT 3.1.1
If the ClientId represents a Client already connected to the Server then the Server MUST 642 disconnect the existing Client [MQTT-3.1.4-2].
Our current plan:
Use the same MQTT ClientID across all main/standbys.
Do NOT unsubscribe from topics when the Driver stops (at least for changeovers).
When the new Driver instance runs and connects with the same ClientId it will inherit the existing connection at the broker. Including any previously queued messages for the same ClientId.
Any other thoughts on best practice here?
We have raised this issue to discuss a scenario that could arise for a Lucid SA that provides replication across servers for resilience, which means any one of a set of SA servers may be actively connected to the Lucid broker at one time but the MQTT connection could be moved to one of the other servers during a failover.
Lucid v3.0 says UP/data uses QoS 1 and Retained 0.
in section 6.1 it also says the MQTT features Clean Session/Clean Start are optional at both SA and FD.
We are assuming the SA wants to support any MQTT 3.1.1 or higher broker, therefore we cannot assume shared subscriptions are available as this feature only became mandatory in MQTT 5.
If SA does set cleanSession to false
Assuming the SA does set cleanSession to false (which is not the default) then it would mean, the active SA server driver drop its connection to the broker (non-gracefully - i.e. it does not unsubscribe to the topics) then the broker would try to queue the UP/data messages until the SA server reconnects. So in this case the Lucid driver would see any UP/data messages we may have missed whilst we lost comms with the broker.
SA Server Changeover timing complication
If the SA elects a different server to be the active (which now has the responsibility to connect to the Lucid Broker) and closes the connection from the previous active SA then the the timing between closing the previous connection and creating the new connection could risk loosing UP/data payloads. This is where FD's could publish data whist none of the SA servers are subscribed to the UP/data topics.
If SA leaves cleanSession set to true
If an SA chose to connect with cleanSession set to true (which is the default) then we think the SA would be more likely to lose data messages as above but also if the driver lost comms or restarted (without gracefully closing down the UP/data topic subscriptions).
The Lucide specification says It is up to the FD to ensure it gets the data to the Broker, but this means there is no application level acknowledgement (above MQTT layer) for the data topic.
SparkplugB
SparkplugB has seq and BDseq which provides some limited help in this area.
But as seen in the NCDM/Rebirth command SparkplugB cares more about the current (latest) value of its data points and sees the intermediate time series values as being less important.
But in the case of Lucid the Intermediate values could be alarms that need to be raise/cleared in the SA.
Proposed Workaround
This is based on the understanding that if two clients connect with the same MQTT ClientId then the later connection keeps the existing subscription and the first client is disconnected.
MQTT 3.1.1
If the ClientId represents a Client already connected to the Server then the Server MUST 642 disconnect the existing Client [MQTT-3.1.4-2].
Our current plan:
Use the same MQTT ClientID across all main/standbys.
Do NOT unsubscribe from topics when the Driver stops (at least for changeovers).
When the new Driver instance runs and connects with the same ClientId it will inherit the existing connection at the broker. Including any previously queued messages for the same ClientId.
Any other thoughts on best practice here?