How I Archive

One thing I discovered over the years: I generate a TON of files. A lot of these files are trash and do not need to be referenced in the future. But there are some that I feel iffy about deleting - or even just not really a good idea.

These include:

1. Past Projects whose architecture I might want to reference 2. Photos of old documents 3. Software I purchased which is in danger of just not being available on the internet anymore in the future

For these files, I archive them. And over the few years I started doing so, I optimized it.

Archiving vs Backups

Archiving is distinct from backups in that backups are for cases where you want to restore files. Archiving meanwhile are files that you kind of want to restore but are stuff that you will not touch in a long time.

Think of it like this: Backups are operational. After data loss, corruption, accidental deletion, or malware, you want to get back to work quickly. But archives are for long-term preservation.

That also means archives generally need to be self-sufficient. I do not back up absolutely everything especially if the item is easily recoverable. But an archive needs to not rely on external dependencies so in a backup, I might not back up a license key or the file behind a link. But for an archive, both are necessary since they might not exist in the future.

Compress, Encrypt, and Sign

The process for archiving something is fairly simple: Compress, Encrypt, and then Sign.

Compress: Tar + ZSTD or ZSTD always

Since these files will not be touched for years (maybe ever), there is no reason to not use compression. I opted to use zstd since it has fast compression and decompression while having good ratios. xz has been the best at compression ratios even under testing but not by much. Usually, zstd is only worse by 4-5% in my limited testing but is MUCH faster (like 50-60% if not more).

I always use .tar.zst if I am archiving a folder but for the times I am archiving a file, I just use .zst.

Encrypt: Age

I actually really want to use AES for encryption because it is hardware accelerated. But I settled with Age for a few reasons:

One is that age is built for encrypting files and uses good encryption (ChaCha20-Poly1305). Next is that I can encrypt using multiple keys. I can protect myself from myself by encrypting to multiple keys and storing them in different locations. My threat model is a random person stealing my hard drive or the cloud provider going rogue so I don’t care too much of the “decreased security”. I could do one key with splitting using Shamir’s Secret Sharing but I am not making my life harder for a minor increase in security for my threat model.

Lastly, there is just no equivalent for AES. I think the closest is OpenSSL but I am scared of messing up and randomly realizing I did not do something properly. An alternative is GPG which I don’t like since the implementations are generally a mess.

Sign: Minisign

Age does have verification, but signing helps me protect against one threat: A provider replacing a file with another file. If a provider has your public key, they can just replace the file with another file encrypted with that public key (probably with the same file size) without you knowing. If you signed the file, unless the provider knows your private key, they cannot generate one without raising alarms.

It does also protect against bitrot but age would already not allow you to decrypt the file anyway so it isn’t as needed. The main protection here is to prevent a cloud provider from replacing the file.

Cheap Cloud Providers as Locations

The main benefit of encrypting and signing files is that I can care less about which provider has my files. The only metadata they have are file size, uploaded date, and the filename (as I choose not to hide it). Thus, I generally hop from provider to provider depending on which one has the best deal.

Currently, I just use a local drive, and a storage box for my archives. These are still backed up so I do not really mind. If ever another provider has a really good deal, I can focus on the price rather than “will they be gone in five years” or “is their security sound” because I already took care of it.