Skip to content

usage: create backup usage record for vmId-offeringId pair #5259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yadvr merged 1 commit into from
Sep 3, 2021
Merged

usage: create backup usage record for vmId-offeringId pair #5259

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 17 additions & 51 deletions usage/src/main/java/com/cloud/usage/parser/BackupUsageParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,13 @@

package com.cloud.usage.parser;

import java.text.DecimalFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.annotation.PostConstruct;
import javax.inject.Inject;

import org.apache.cloudstack.backup.Backup;
import org.apache.cloudstack.usage.UsageTypes;
import org.apache.log4j.Logger;
import org.springframework.stereotype.Component;
Expand Down Expand Up @@ -68,65 +66,33 @@ public static boolean parse(AccountVO account, Date startDate, Date endDate) {
return true;
}

final Map<Long, BackupInfo> vmUsageMap = new HashMap<>();
for (final UsageBackupVO usageBackup : usageBackups) {
final Long vmId = usageBackup.getVmId();
final Long zoneId = usageBackup.getZoneId();
final Long offeringId = usageBackup.getBackupOfferingId();
if (vmUsageMap.get(vmId) == null) {
vmUsageMap.put(vmId, new BackupUsageParser.BackupInfo(new Backup.Metric(0L, 0L), zoneId, vmId, offeringId));
Date createdDate = usageBackup.getCreated();
Date removedDate = usageBackup.getRemoved();
if (createdDate.before(startDate)) {
createdDate = startDate;
}
final Backup.Metric metric = vmUsageMap.get(vmId).getMetric();
metric.setBackupSize(metric.getBackupSize() + usageBackup.getSize());
metric.setDataSize(metric.getDataSize() + usageBackup.getProtectedSize());
}
if (removedDate == null || removedDate.after(endDate)) {
removedDate = endDate;
}
final long duration = (removedDate.getTime() - createdDate.getTime()) + 1;
final float usage = duration / 1000f / 60f / 60f;
DecimalFormat dFormat = new DecimalFormat("#.######");
Copy link
Copy Markdown
Member

@yadvr yadvr Sep 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this cause issue where localisation is not English?

Copy link
Copy Markdown
Member Author

@weizhouapache weizhouapache Sep 2, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rhtyd
good question ......
if it is a bug, we need to change all other usage parsers which use the same code. we have not received bug report about it until now.

String usageDisplay = dFormat.format(usage);

for (final BackupInfo backupInfo : vmUsageMap.values()) {
final Long vmId = backupInfo.getVmId();
final Long zoneId = backupInfo.getZoneId();
final Long offeringId = backupInfo.getOfferingId();
final Double rawUsage = (double) backupInfo.getMetric().getBackupSize();
final Double sizeGib = rawUsage / (1024.0 * 1024.0 * 1024.0);
final String description = String.format("Backup usage VM ID: %d", vmId);
final String usageDisplay = String.format("%.4f GiB", sizeGib);
final Double rawUsage = (double) usageBackup.getSize();
final String description = String.format("Backup usage VM ID: %d, backup offering: %d", vmId, offeringId);

final UsageVO usageRecord =
new UsageVO(zoneId, account.getAccountId(), account.getDomainId(), description, usageDisplay,
UsageTypes.BACKUP, rawUsage, vmId, null, offeringId, null, vmId,
backupInfo.getMetric().getBackupSize(), backupInfo.getMetric().getDataSize(), startDate, endDate);
new UsageVO(zoneId, account.getAccountId(), account.getDomainId(), description, usageDisplay + " Hrs",
Copy link
Copy Markdown
Member

@yadvr yadvr Sep 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not make sense, effectively a backup takes up space and not time. For example, two instances of backups can have different sizes - can you check how we do it for snapshots and volumes. Effectively backups of a VM are like snapshots for a volume.

Copy link
Copy Markdown
Member Author

@weizhouapache weizhouapache Sep 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rhtyd
I have checked the usage for snapshots and volumes - cloudstack displays running time for both. see

https://github.com/apache/cloudstack/blob/main/usage/src/main/java/com/cloud/usage/parser/VolumeUsageParser.java#L150-L153

https://github.com/apache/cloudstack/blob/main/usage/src/main/java/com/cloud/usage/parser/VMSnapshotUsageParser.java#L131-L134

UsageTypes.BACKUP, new Double(usage), vmId, null, offeringId, null, vmId,
usageBackup.getSize(), usageBackup.getProtectedSize(), startDate, endDate);
s_usageDao.persist(usageRecord);
}

return true;
}

static class BackupInfo {
Backup.Metric metric;
Long zoneId;
Long vmId;
Long offeringId;

public BackupInfo(Backup.Metric metric, Long zoneId, Long vmId, Long offeringId) {
this.metric = metric;
this.zoneId = zoneId;
this.vmId = vmId;
this.offeringId = offeringId;
}

public Backup.Metric getMetric() {
return metric;
}

public Long getZoneId() {
return zoneId;
}

public Long getVmId() {
return vmId;
}

public Long getOfferingId() {
return offeringId;
}
}
}