On some servers, FTP in particular, there may be a limit on the number of files that can be listed. If you increase the volume size, it will again mean “fewer but larger files”. Also, the volume size can be changed after a backup has been created. Unlike the chunk size described above, it can be beneficial to both increase or decrease the volume size to fit your connection characteristics. The default size is 50mb, which is chosen as a sensible default for home users with limited upload speeds.
The remote volumes are called dblock files internally, and that is the extension used for the files. The volume size can be set in the graphical user interface, as well as on the commandline with the option -dblock-size. Encryption is applied to the volumes, which reduces the possibility of someone deducing properties about the contents inside the volume. The volumes are then compressed, which saves storage space and bandwidth. Rather than storing the chunks individually, Duplicati groups data in volumes, which reduces the number of the remote files, and calls to the remote server. The lower limit is 10kb, and there is no upper limit, but choosing values larger than 1mb should only be done after evaluating the above impacts. If there is sufficient bandwidth to the remote destination, choosing a larger chunk size is usually beneficial. With larger chunk sizes, it is also less likely that deduplication will detect any matching chunks, as the shared chunks contain more data. If there are many small changes to the files, this will generate many new blocks, which increases the required storage space as well as the required bandwidth. If a single byte is changed in a file, Duplicati will need to upload a new chunk. The downside to choosing a large chunk size, is that change detection and deduplication covers a larger area. When restoring, this is also a benefit, as more data can be streamed into the new file, and the data will likely span fewer remote files. If you have large files, choosing a large chunk size, will also reduce the storage overhead a bit. This effect is more noticeable if the database is stored on non-ssd disks (aka spinning disks). Internally each block needs to be stored, so having fewer blocks, means smaller (and thus faster) lookup tables. It is also possible to choose a smaller chunk size, but for most cases this has a negative impact. If you choose a larger chunk size, that will obviously generate “fewer but larger blocks”, provided your files are larger than the chunk size.
Duplicati will abort the operation with an error if you attempt to change the chunk size on an existing backup. If a file is smaller than the chunk size, or the size is not evenly divisible by the block size, it will generate a block that is smaller than the chunk size.ĭue to the way blocks are referenced (by hashes), it is not possible to change the chunk size after the first backup has been made. The chunk size is set via the advanced option -block-size and is set to 100kb by default. The block sizeĪs Duplicati makes backups with blocks, aka “file chunks”, one option is to choose what size a “chunk” should be.
Duplicacy chunk size max how to#
This documents explains what these tradeoffs are and how to choose those that fit a specific backup best. Choosing these options optimally is a balance between different usage scenarios and has different tradeoffs. Some of these options are related to sizes of various elements.
All options in Duplicati are chosen to fit a wide range of users, such that as few as possible of the users need to change settings.
0 kommentar(er)
