Allow deleting used without "force"

[MartinEmrich]

ISSUE TYPE

• Improvement Request
• Documentation Report

COMPONENT NAME

UI, Documentation

CLOUDSTACK VERSION

4.18.1

CONFIGURATION

Hypervisor: Xen/XCP-ng

OS / ENVIRONMENT

SUMMARY

When trying to delete a Template, it fails with a message like "Error Unable to delete template with id: 294 because VM instances: [VM instance {"id":2972,"instanceName":"i-39 ..." (continuing with a long list beyond the screen).

This can be overridden using the "force" delete option, leaving a bad feeling of "doing something dangerous".

STEPS TO REPRODUCE

• Create a VM template (e.g. import ISO, install VM, make Template from the root device).
• Create a new VM from this template
• Try to delete the template

EXPECTED RESULTS

A template should be deletable at any time without error, unless there is a actual reason against it. (which could then still be overridden using the "force" option).

ACTUAL RESULTS

Error message (see above). This gives me a clear feeling of "doing something bad", without any explanation.
This suggests that there is a hidden connection between a template and VMs created from it after creation, which the" force" option would just leave dangling.

Yet the documentation (http://docs.cloudstack.apache.org/en/4.18.1.0/adminguide/templates.html) explicitly states "When templates are deleted, the VMs instantiated from them will continue to run.", which is also my observation.

[DaanHoogland]

@MartinEmrich, #7731 should address your concern.

[MartinEmrich]

Hmm, reading the discussion and the code, it looks more like the default value for the force parameter (if not specified) was changed from true to false, which would actually make increase the issue for me. (By also affecting the API directly, not only the Web UI which seems to default to false already)

I tried to delete the templates using the Web GUI, leaving the force in the default "false" position.

I still fail to understand what would speak against always setting "force=true", making that flag redundant?

[DaanHoogland]

right? do you see any stacktraces in the logs during the call to delete

[MartinEmrich]

No stacktraces in the logs.

2023-10-30 12:51:32,525 DEBUG [c.c.a.ApiServlet] (qtp1444635922-101117:ctx-7a8983e0) (logid:8661feb8) ===START===  10.14.32.11 -- GET  id=f0bafad8-736a-4d15-905f-8afe957fa6a4&forced=false&zoneid=e99e796c-4e5c-4549-a0d7-ddd3d8526c06&command=deleteTemplate&response=json

This is the error message:

2023-10-30 12:51:32,575 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-36:ctx-aa4d8787 job-97867) (logid:f3b3e108) Complete async job-97867, jobStatus: FAILED, resultCode: 431, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"431","errortext":"Unable to delete template with id: 322 because VM instances: [VM instance {"id":XXXX,"instanceName":"YYYYY","type":"User","uuid":"f8a0c75b-784d-4490-8e29-7fd9f179a4ca"} (and lots more) ] are using it."}

I do not understand how running VMs are using templates. On XenServer, the ROOT volume ist copied from the template image; afterwards, I see no connection between the running VM and the template.

Maybe there are other hypervisors with such a relation (e.g. running VMs in a copy-on-write fashion laid over the active template image), but with Xen/XCP that's not the case (and would be veeeery slow from NFS secondary storage if it was the case).

[DaanHoogland]

thin provisioned disks (copy on write principle) would indeed explain this. You could make sure none of you service offering/ disk offering are using this.

what xcp version are you using?

[MartinEmrich]

As of now we're still on XCP-ng 7.6, but the upgrade to 8.2 is indeed currently underway by our team. So I cannot check stuff on the Pool for the next hours/days.

The only Secondary Storage we have is a NFS share, so am quite certain a real copy-on-write deployment is not used here currently.

Until now, thin provisioning on Xen meant allocating only the blocks containing actual data (i.e. actually allocating the storage only on the first write), not any copy-on-write scheme.

I do not know if XCP-ng 8.2 brings any new features regarding that. But still, LVM2-backed copy-on-write has to work on a blockdevice-level (local volumes, iSCSI,...), I doubt it could use a filesystem-level image (on a NFS share) as a lower layer.

[DaanHoogland]

It just came to me why this is working this way @MartinEmrich ; For a running instance you can say reinstall and for that you would need the original template. I agree that the message could be somewhat more friendly but the intention is good ;)
I am not sure if we could get away with just changing documentation for this. Returning a list of VMs using the template would be useful for an operator I'd say, to help make sure if they want to force delete.

[MartinEmrich]

Ah ok, that makes sense.

Maybe a "fix" could indeed be documentation-only: a sentence like "Deleting templates used by VMs is perfectly safe, but the VM can no longer be reinstalled from the deleted image".

And then, if the user accepts that and force-deletes old images, the Reinstall button will disappear and the restoreVirtualMachine command will obviously return some template-not-found error message.

[shwstppr]

@MartinEmrich force param is probably there to prevent wrongful deletion of template. As some operations such as VM restore rely on the availability of the template personally I feel it is not correct to right away delete the template when there are active VMs that use the template. User can always delete the template even in that condition by specifying force=true. From the content of your issue I feel we probably need to make the error message more meaningful.

[MartinEmrich]

@shwstppr yes I agree, I just would love to see it made more clear what "in use" actually means.

I use CloudStack for about 10 years now, and always thought I would actually break the running VMs because the template is "in use by them", so there's a big heap of old images.
Only now after doing some digging (resulting in this issue), I notice that keeping them makes no sense for us.

[DaanHoogland]

So now we have two issues if I understand correctly:

1. Add documentation explaining the consequences of deleting templates
2. make a UI change to disable the reinstall button if the template is deleted

1. I agree on
2. does not make sense as the VM may also be re-installed with a different template. This just isn´t the default.

@MartinEmrich can you make a PR for the documentation change?

[MartinEmrich]

Yes, I gave it a try: apache/cloudstack-documentation#355

[DaanHoogland]

@MartinEmrich can we close this issue?