Skip to content

Randomize managed volume copy host #5789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yadvr merged 3 commits into from
Dec 30, 2021
+3 −2
Merged

Randomize managed volume copy host #5789

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1d1e128
Randomize managed volume copy host
Nov 16, 2021
0a39c33
Use String.format for ScaleIO debug message
Dec 22, 2021
919d66f
Update debug message for ScaleIO copy methods
Dec 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ...in/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -629,7 +629,7 @@ public void copyAsync(DataObject srcData, DataObject destData, Host destHost, As

private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO template to volume
LOGGER.debug("Initiating copy from PowerFlex template volume on host " + destHost != null ? destHost.getId() : "");
LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : "<not specified>"));
int primaryStorageDownloadWait = StorageManager.PRIMARY_STORAGE_DOWNLOAD_WAIT.value();
CopyCommand cmd = new CopyCommand(srcData.getTO(), destData.getTO(), primaryStorageDownloadWait, VirtualMachineManager.ExecuteInSequence.value());

Expand All @@ -648,7 +648,7 @@ private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Hos

private Answer copyVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO volume
LOGGER.debug("Initiating copy from PowerFlex volume on host " + destHost != null ? destHost.getId() : "");
LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : "<not specified>"));
String value = configDao.getValue(Config.CopyVolumeWait.key());
int copyVolumeWait = NumbersUtil.parseInt(value, Integer.parseInt(Config.CopyVolumeWait.getDefaultValue()));

Expand Down
1 change: 1 addition & 0 deletions server/src/main/java/com/cloud/storage/StorageManagerImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2136,6 +2136,7 @@ public Host findUpAndEnabledHostWithAccessToStoragePools(List<Long> poolIds) {
if (hostIds.isEmpty()) {
return null;
}
Collections.shuffle(hostIds);
Copy link
Copy Markdown
Contributor

@GutoVeronezi GutoVeronezi Dec 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could add some log to this method, about which host was selected (or not).

Copy link
Copy Markdown
Contributor Author

@mlsorensen mlsorensen Dec 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's logging elsewhere, where the returned value is used.

Copy link
Copy Markdown
Member

@GabrielBrascher GabrielBrascher Dec 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be wrong on this one, but I think that there are not many logs regarding the chosen host to migrate.
Call hierarchy that I've taken a quick check does not present logs. Assuming your comment, this would be logged later on in the call hierarchy ( I haven't checked all the way). Even if it is, you can see that many exceptions can be thrown not too late after the host ID is retrieved.

> StorageManagerImpl.findUpAndEnabledHostWithAccessToStoragePools(List<Long>)  (com.cloud.storage)
    > VolumeServiceImpl.copyManagedVolume(VolumeInfo, DataStore)  (org.apache.cloudstack.storage.volume)
        > VolumeServiceImpl.copyVolume(VolumeInfo, DataStore)  (org.apache.cloudstack.storage.volume)
            > VolumeOrchestrator.copyVolumeFromSecToPrimary(VolumeInfo, VirtualMachine, VirtualMachineTemplate, DataCenter, Pod, Long, ServiceOffering, ...)  (org.apache.cloudstack.engine.orchestration)
            ...
            > VolumeOrchestrator.migrateVolume(Volume, StoragePool)  (org.apache.cloudstack.engine.orchestration)
            ...
            > VolumeApiServiceImpl.orchestrateExtractVolume(long, long)  (com.cloud.storage)
            ...

Additionally, the canHostAccessStoragePools called inside the findUpAndEnabledHostWithAccessToStoragePools can return false. If the host cannot access, not much is logged as well.

Please, let us know in case we are missing something here.

Copy link
Copy Markdown
Member

@yadvr yadvr Dec 30, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logging is not of the randomised list but the selected host is logged by the deployment planner or during provisioning or volume operation.


for (Long hostId : hostIds) {
Host host = _hostDao.findById(hostId);
Expand Down