You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR adds a command to the DockerNode allows restarting of a container during simulation e.g. by a workflow.
Using this function in a workflow enables the following use cases:
Testing the behavior of other components if a container restarts
Testing the restarting behavior of the component that restarts if other nodes are up and running
Changing the image of the container during simulation e.g. to a newer version or a build with a different configuration
Changing the parameters of the container e.g. connected volumes, exposed ports, environment variables during simulation
I have to admit lines 249-253 are a bit hacky. They are a workaround for a strange behavior where I couldn't get to the root cause.
If a DockerNode is stopped by calling .stop_docker_container() the veth pair that belongs to it is deleted as well (as far as I can tell by the operating system since there is no call to interface.remove_veth_pair()). The strange thing is that it only happens if the DockerNode is the last one defined in the Scenario. If it is not the last one, the veth_pair survives but is still connected to the namespace of the old container and therefore needs to be deleted and recreated as well.
I tried to find the cause for this behavior but couldn't identify if it's something the kernel or the docker daemon does. However, the solution that is implemented in this PR worked for me for a couple of months during which is used this feature extensively.
Sure we cannot come up with a cleaner solution for 249-253? @arneboockmeyer, can the deletion of the veth pair be related to some kind of automatic clean up?
I hacked up an example scenario based on the existing "basic_example.py" to replicate the behaviour that I described here: https://gist.github.com/lhoff94/ceb72b2b16e2f82359d487b8b204a5cb
The "restart_docker_container" method is embedded into the scenario so that it works with the latest Marvis release (without the commit from this PR).
I also added the mavis log with the exception as a second file to the gist. As you can see, the first container restarts as expected but the second doesn't because the veth can't be deleted because it's not there anymore.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a command to the DockerNode allows restarting of a container during simulation e.g. by a workflow.
Using this function in a workflow enables the following use cases:
I have to admit lines 249-253 are a bit hacky. They are a workaround for a strange behavior where I couldn't get to the root cause.
If a DockerNode is stopped by calling .stop_docker_container() the veth pair that belongs to it is deleted as well (as far as I can tell by the operating system since there is no call to interface.remove_veth_pair()). The strange thing is that it only happens if the DockerNode is the last one defined in the Scenario. If it is not the last one, the veth_pair survives but is still connected to the namespace of the old container and therefore needs to be deleted and recreated as well.
I tried to find the cause for this behavior but couldn't identify if it's something the kernel or the docker daemon does. However, the solution that is implemented in this PR worked for me for a couple of months during which is used this feature extensively.