Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


iwStack: How much space do snapshots take up?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

iwStack: How much space do snapshots take up?

http://iwstack.com/

If mirroring a volume / image in iwStack, does a snapshot occupy the same amount of fixed space as the image? So if the root volume or some other volume is a fixed 20 GB but only 5% of it is being used, how much space does the snapshot occupy? Is the snapshot using 1 GB or 20 GB in that case?

I tried reading here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Snapshots. But I'm still not sure after reading that.

Comments

  • Did you send in a ticket regarding this question to the appropriate parties?

  • Been a while, but I think the snapshot only consists of the data actually used, and it's compressed.

  • cirrus_cloudcirrus_cloud Member
    edited September 2016

    @teamacc said:
    Did you send in a ticket regarding this question to the appropriate parties?

    Yes, I already sent in two today, but I didn't want to spam them. Also, I found the other topics on iwStack here on LET to be helpful. I thought this could act as a reference for others.

    My question was

    If I understand correctly, a 10 GB image (whether it's 0 or 100% full) requires 10 GB of secondary storage for snapshots. A snapshot is the entire image, not just the part of the part that is being used.

    The response was

    Yes, same if it is 1 GB or 100.

    I am not entirely sure what that means, and I don't see anything explicit on the issue on the wiki for apache cloudstack.

    So if Nekki is right, then if I have an image where N=100GB (partitioned space) and M=7GB (used space), then each and every snapshot will take up just M space. If I want to have 4 backups / snapshots of an image, then it's 4 * M.

  • An image will use the entire volume, not the data inside that volume. So if it's a 10GB volume with only 1GB used then the image will still be 10GB.

    That is how I understand it anyway.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    Unless they do some sort of sparsing of the data then yes, it'll be whatever the max usage you've ever had just because those blocks are 'dirty'.

    Francisco

    Thanked by 1cirrus_cloud
  • As far as i can remember (It's months since i did that and i'm old ;), if you export and download a volume, it's compressed but only the real used data is exported. Never had snapshots... so i can only speak for exported instance volumes.

    Thanked by 1cirrus_cloud
  • @Shade said:
    As far as i can remember (It's months since i did that and i'm old ;), if you export and download a volume, it's compressed but only the real used data is exported. Never had snapshots... so i can only speak for exported instance volumes.

    Yeah, I'm not sure if it's the same if they save the snapshots.

  • Raise a ticket mate, only way you'll get an answer....unless.....

    I invoke @maounique

  • or simply try ;-)

    as other said it most probably is the full size, as they wont be able to determine from outside what parts of the image are actually used and which are just dirty blocks.

    even if they start with a sparsed image which could result in a considerably smaller snapshot, that image will grow over time by your system writing and deleting files.

    most filesystems don't use old dirty blocks after a file deletion but unused ones for new writes if available. so you will see the image growing despite not needing more space inside at all.

    if they do use some kind of compression on the snapshot it might help if you do overwrite the unused dikspace inside your container with zeros ( dd if=/dev/zero of=/tmpfile ... ) until disk is full, then remove tmpfile. this is just to get rid of the dirty blocks and have the snapshot feature hopefully compress the zeroed part.

    if sparsing works at least, you might consider having a look into using xfs, as this does overwrite/re-use dirty blocks first instead.

    Thanked by 1vimalware
  • MaouniqueMaounique Host Rep, Veteran
    edited September 2016

    No compression takes place, even at downloading, the images are 1:1.
    The problem with compression is that it takes CPU and extra I/O and most people take snapshots at the top of the hour, it would be a disaster if they all start using cpu and I/O in the same time, especially on SAN connected zones, even if some bw on the secondary storage will be saved. We upgraded the links to secondary storage so that is not an issue now.

    • The last snapshot of my 80GB VM used 26.9GB.
    • I've just zeroed the unused disk space and requested a new snapshot, and it says it will be 14.97GB.
    • The content of my VM has not changed much since the last snapshot, so the fact that zeroing the unused space reduced the size of the snapshot, along with the fact that the OS reports 24.7GB is in use by the VM, means snapshot compression must be enabled.

    And I've just realized that you're asking about iwStack. For some reason my brain read LunaNode, so that's what I was testing with...hopefully the same is true with iwStack :)

    Thanked by 2Falzo vimalware
  • Thanks for the replies. Support replied and @Maounique have confirmed that images are the same size. This is Apache cloudstack as mentioned in the original post.

Sign In or Register to comment.