The problem: Most computer users manually save versions of files with names like Presentation-2027-Jan-11, followed later by Presentation-2027-Jan-11-UPDATED and Presentation-2027-For-Sally-Only, perhaps with Presentation-panic-edits-before-event. This is manual versioning, and even people who stick strictly to a naming convention still get confused at times. Some applications append .bak or prefix ~ to create their own backups, and then occasionally an automatically-created TMP or $tmp$ file can really save the day, and so on. Then when it comes time to clean up and organise a filesystem, everyone hesitates. Am I really sure there is not useful work in each of those ten versions? Which ones do I need to keep? So, we usually keep them all. After all, storage is cheap.

A working solution: Automatic file versioning is a very handy tool for reducing user errors and confusion, but not many people know this because only one versioning filesystem has been implemented in production. This article explains how file versioning works at a user and implementation level, with the intention of understanding how to add it to open source operating systems.

The OpenVMS ↗ operating system and its versioned filesystem has been used since 1977, and in 2004 was reckoned to have ten million users or more ↗ across half a million or so VMS clusters. I used it in the late 1990s, and ever since I have wanted file versioning on my operating systems. Unfortunately, outside VMS, versioning has only ever existed in short-lived experiments but in today’s world of cloud computing, AI and mobile devices it feels more relevant than ever. (For the curious, OpenVMS is still used today, with non-commercial copies available for free so you can try it out. I don’t really miss anything from VMS except for versioning, I find it quite clunky.)

How it works

In OpenVMS, saving a file does not destroy the file it replaces. The old one is still there with the same name, with a version number after it:

$ DIR MYFILE.TXT

MYFILE.TXT;1

$ EDIT MYFILE.TXT  ! Make some changes to MYFILE.TXT
$ DIR MYFILE.TXT

MYFILE.TXT;2        MYFILE.TXT;1

The second file appears automatically when the editor closed the first one, without the editor’s knowledge. COPY behaves the same way, this time with the default directory output left in:

$ DIR *.TXT

Directory DISK$USER:[SALLYSMITH]

ANOTHERFILE.TXT;1   MYFILE.TXT;1

Total of 2 files.

$ COPY MYFILE.TXT ANOTHERFILE.TXT
$ DIR *.TXT

Directory DISK$USER:[SALLYSMITH]

ANOTHERFILE.TXT;2   ANOTHERFILE.TXT;1   MYFILE.TXT;1

Total of 3 files.

(OpenVMS users will notice that this time I did not remove the standard OpenVMS DIR output.)

Anything that opens a file for writing on OpenVMS has this effect, and accessing MYFILE.TXT without a version number will always return the highest-numbered version. TYPE MYFILE.TXT will display the contents of version 2 on the screen (like the Unix cat command), while TYPE MYFILE.TXT;1 explicitly gives the first version. At any time you can use the PURGE command to delete all versions except for the highest numbered, which retains its number because the version numbers just keep climbing. The maximum possible version number is 32,767, but most OpenVMS installations default to 30 or so, a number commonly found to be good among the massive community of OpenVMS administrators.

🕰️
What versioning means here Every save creates a new generation of the file, numbered upward from 1. A bare name always resolves to the newest. Older generations sit beside it under the same name, reachable by number, until something removes them. This works for every program on the system, because it happens underneath every program, in the layer that opens files.

Other versioning systems

In closed source operating systems, TENEX ↗ had versioning in 1969 and later TOPS-20 ↗ took it to market, both dead by 1990. Similarly for Digital Equipment Corporation (DEC)’s RSX-11 ↗ and RSTS/E ↗ operating systems (available today under hobbyist licenses). DEC remembered about versioning and kept it in OpenVMS, which is how I came to use it years later. In open source, research literature describes the Elephant Filesystem, CVFS, VersionFS ↗ and ext3cow ↗, none of which ever had a userbase.

DragonFly BSD’s HAMMER ↗, shipped and may still be in use. It retains fine-grained history and puts it in the namespace as filename@@0x016llx, reclaimed later by an explicit prune.

What has been very successful in production is whole filesystem snapshots: open source has Btrfs ↗ and ZFS ↗, with WAFL and Windows Shadow Copy also widely used. A snapshot freezes an entire volume at one moment, and that snapshot can be backed up. This useful, but not at all the same as answering what did this particular file look like three saves ago?

How it feels to use

Mostly you can ignore the versioning altogether. You know that the newest version is the one you get by default, and that you can’t accidentally overwrite data, and that PURGE tidies up.

When you do want an older version, you don’t even necessarily need the version number. As the VSI wiki ↗ explains:

NotationMeaning
omitted, or ;highest
;0highest
;-1second highest
;-2third highest
;-0lowest

So TYPE MYFILE.TXT;-1 gives you the file as it was before your last save, without your having to look anything up first.

Commands differ in what they do when you leave the version off, according to the user’s likely requirement:

CommandDefault
TYPE (Unix cat)highest only
DIRECTORYall versions
PURGEall but the highest
DELETEnone. Unsurprisingly, a version must be given explicitly, or * for all
$ DELETE MYFILE.TXT
%DELETE-E-DELVER, explicit version number or wild card required

$ DELETE MYFILE.TXT;3      ! that one
$ DELETE MYFILE.TXT;       ! the highest
$ DELETE MYFILE.TXT;*      ! the lot
$ PURGE/KEEP=2 MYFILE.TXT  ! everything below the top two

The main point here is that all commands and applications (especially editors, see below for more on that) agree on how a file should be created, opened and closed. It doesn’t matter if a VMS application knows nothing about file versions (and many of them do not), since all applications use the one route for file handling the versions just happen anyway.

Version numbers wrap around

Left alone, versions accumulate until the limit, and when a new version would exceed the limit, the lowest surviving one is first deleted:

$ CREATE/DIRECTORY/VERSION_LIMIT=3 [.THREE]
$ SET DEFAULT [.THREE]
$ ! four rounds of editing REMY.DAT later...
$ DIR REMY.DAT

REMY.DAT;4          REMY.DAT;3          REMY.DAT;2

Version 1 vanished when version 4 was created, fully explained in Raymii’s walkthrough ↗. The default limit is zero, meaning no limit. A system where every directory reports 30 or some other number means the administrator has set this limit in site policy, or, it has been set somewhere up the tree and it propagated, because a new directory inherits its parent’s default.

Versioning is maintained at the bottom layer

Something I found quite confusing when doing the literature research is that while the Record Management Services (RMS) is what a program on OpenVMS uses to open a file and what I remember interacting with, it isn’t really the VMS filesystem. RMS presents a file as a logical collection of records in one of three organisations: sequential, relative, or indexed with keys. This contrasts with Unix where you get a stream of bytes and add the structure yourself with your choice of library abstraction.

Disk block layout, protection and versioning are all below RMS, in the Files-11 filesystem. Version numbers are assigned by the system call IO$_CREATE, which RMS or any other program can invoke, and that call is also what enforces the directory’s version limit. A program that bypasses RMS and issues $QIO system calls directly still gets versioning and full Files-11 functionality, it just has to do all the work that RMS does transparently to the user.

In Files-11, each version is a separate file with its own File ID holding a full copy of the data, so FOO.DAT;4 has no shared disk blocks with FOO.DAT;5 even if they are substantially the same file. OpenVMS does not do delta storage or compression.

OpenVMS versioning documentation and code

  1. VSI OpenVMS I/O User’s Reference Manual ↗, chapter 1, the ACP-QIO interface (PDF ↗). This is the canonical, comprehensive documentation. The version-limit algorithm is given in the discussion of IO$_CREATE and IO$_ACCESS.
  2. VSI wiki: File version ↗. Brief practical document covering the number range, per-command defaults, relative versions and limits.
  3. David Deley, VMS/RMS Internal Data Structures ↗. A byte-level walk through the on-disk format, with the DUMP commands shown as he goes so you can follow along.
  4. VMS File System Internals ↗, Kirby McCoy, Digital Press 1990. The persistent structures are in chapter 2. It describes VMS 5.2, so it predates ODS-5 and isn’t quite a complete on-disk specification, but it’s close.
  5. For working code, Paul Nankervis’s ODS2 reader ↗ parses directory records to produce listings on non-VMS hosts, and Carl Lydick’s ODS-2 reader ↗ does the same thing in a single C file.

Implementation issues on any Unix

Unix is really very messy at the kernel API and userspace levels, which is what would make versioning quite difficult to turn into an everyday tool - in fact “messy” is being polite, it is a perverse, chaotic and error-prone situation.

Unix programs have never agreed on how to save a file. Some editors truncate the original and write into it, which produces a new version on every autosave unless something decides when a write has finished. Others write a temporary file and rename(2) ↗ it over the top, which POSIX defines as an atomic replacement, so the name acquires a new inode with no recorded relationship to the one it displaced. Vim does either, depending on ‘backupcopy’ ↗. cp, mv, rsync, package managers and build systems each make assumptions about file identity, atomicity, and when a file is finished being written.

All this means that anyone trying to implement a Unix/Posix system call meaning create the next generation of this file will have to work out intent from what programs do. There isn’t likely to ever be such a system call, so the next best thing is to implement a filesystem layer that does this work as best it can.

The Unix filesystem call readdir(3) ↗ has no concept of a current version, so a directory holding 30 generations of every file must either return all of them or hide them. Returning them all would cause ls, make, rsync etc to see a filesystem perhaps 30 times the size of all the most recent versions together, while hiding them would mean a backup taken with ordinary Unix tools will lose the history in the versions. That might be a defensible strategy to define filesystem backups as orthogonal to filesystem versioning, but then you’d really want a differencing/compressing filesystem that knows about files as well as blocks. It quickly gets complicated, and also quickly approaches questions like so should we just store everything in git on top of a normal filesystem? Git certainly is a kind of versioning filesytem but then we need a conversion layer between that and all application layers and we have a very similar problem.

OpenVMS has none of this trouble because versioning sits in Files-11 below RMS, as above, so every program gets the same behaviour whether it goes through RMS or issues $QIO directly, and DIRECTORY, PURGE and DELETE were written knowing that versions exist. On Unix the filesystem would be the only part of the system that does know about versions unless there is some deep kernel engineering.

Loose implementation ideas

After reviewing all the main open source filesystems, I have a design for implementing file versioning in btrfs, and I estimate it will take maybe 4000 lines and a month or so to a working prototype. In the process of examining this I noticed Linux and FreeBSD offer quite different ways in, and FreeBSD may be a lot easier.

FreeBSD has native filesystem stacking layers. mount_nullfs(8) ↗ says the null layer exists to be copied in implementations: “make a copy of the null layer, rename all files and variables, and then begin modifying the copy”. null_bypass() in null_vnops.c ↗ passes every vnode operation down to the lower layer generically, so a versioning layer would implement only the handful of operations it cares about and let everything else through untouched. Such a layer mounts over any filesystem and needs no on-disk format change and no fsck work. With OpenZFS underneath, new versions are nearly free, because block cloning ↗ is a reflink and is reachable from copy_file_range(2) ↗ with COPY_FILE_RANGE_CLONE. That is better than Files-11 managed, which as described above copies every block of every version.

Linux has overlayfs, so the shortcut there is in userspace instead of the kernel: fanotify watches a whole mount and tells a daemon when a file has been written, and ioctl(FICLONE) makes the copy free on btrfs or XFS.

This article is titled Part 1, because the second part will point to code people can try. Let me know if you want to join in.