User-union (using LD_PRELOAD)

This is the main web site for User-union, a set of programs for POSIX/Unix/Linux systems that lets you have “union mounts” without special privileges, released as open source software under the "MIT" license.

What is a user union?

In a union mount, changes in one directory (the “underlay”), including its contents, appear to happen normally — but any changes to files are actually performed in a separate parallel directory (the “overlay”) instead. Since contents in the overlay take precedence over the contents in the underlay, once a file is “written” in the underlay, reading it back later from the underlay shows the “new” contents.

Union mounts can be used for a variety of tasks. For example, they can be used to make read-only media appear to be read-write; the underlay is the read-only media, and the overlay is some other directory that can be modified. Union mounts can also redirect software installation processes so that files that appear to be written in one place are instead written elsewhere — this lets you automate DESTDIR.

Unlike many union mount systems, user-union can be used by users without any special privileges. In fact, if you do have root privileges, I suggest that you use another union mount system instead of user-union. User-union will work for root, but it uses LD_PRELOAD to implement union mounts. This implementation technique has many inherent limitations; root users can typically use other techniques that don’t have those limitations. But if you need a way to do union mounts without privileges, user-union may be the tool you are looking for.

If you have root privileges on Linux, you might want to consider FUSE-based approaches and kernel-implemented union mounts. Implementations of such things include unionfs and aufs. As of June 2011 the Linux kernel doesn't have a built-in approach for user mounts, but there's been a lot of work to implement them, and I expect that sooner or later it will have one built in.

The manual page for user-union gives the basics. You might also want to read the manual page for run-redir-union (another program in the package).

Alternatives

If you’re just trying to automate DESTDIR, an alternative is the Auto-DESTDIR package. Auto-DESTDIR is more mature software, but it can’t do many of the things that user-union can do.

Current status

User-union is currently early-adopter software, and is not yet intended for production use. But it works in a lot of cases, far better than some so-callled “finished” programs. Help in finishing it would be appreciated.

For what it does, it works well. It currently intercepts many calls, such as open() and fopen(), and implements them well-enough that for some uses it's already great.

The problem is that some important functionality has not yet been implemented. The biggest current limitation is that the opendir() and readdir() has not been done, so whenever you list the contents of a union’ed directory it won’t be quite right if files have been created or removed. (If you just try to open a file, the right thing will happen.)

It’s only been tested for Linux; it should work as-is on *BSDs, but help testing this (and fixing any problems) would be appreciated. I’ve done some work to make it ready for Apple Mac OS X, but I doubt it’ll work without a little help; patches welcome. It's known to not work yet on Cygnus (Cygnus does support LD_PRELOAD, so this is just a matter of fixing the code to work on Cygnus).

Downloading and getting involved

User-union is released as open source software (OSS) under the MIT license. User-union is hosted on SourceForge.net. You can get the latest release version or the latest development version, report issues, submit patches, discuss it, and so on. Please post pull requests as issues, so they'll be tracked. You can also download my website's copy of the source code for the latest release version of user-union (version 0.14).

The code is posted on SourceForge using git, so if you want access to the development version, the usual commands work.

You can get a read-only copy using:

 git clone git://user-union.git.sourceforge.net/gitroot/user-union/user-union

If you have the right privileges, you can pull a read-write copy, edit, and push back using:

  git clone ssh://USERNAME@user-union.git.sourceforge.net/gitroot/user-union/user-union
  cd user-union
  # Edit away
  git commit -a -m 'YOUR COMMIT MESSAGE'
  git push

Comments on repositories

There are lots of good hosting services for OSS available. I wanted support for git (that knocks out some), a FLOSS implementation of the hosting service (that knocks out github), and bug-tracking support (at the time, that knocked out Gitorious). Fedora hosting is nice, but it throws out less inactive projects (eek); it also has the undeserved reputation of being Fedora-specific and I expect it’ll take me a while to sign the legal forms. Sourceforge met all my criteria: git support, an OSS implementation (Allura, released under the Apache 2.0 License in February 2011), and bug-tracking. Indeed, it has a lot of features that can be enabled or not per-project. I’m also familiar with SourceForge; it’s one of the most widely-used hosting services on the planet. No hosting services is perfect; it’s possible I will change this in the future. Gitorious in particular looks promising.

Implementation approach

User-union is implemented using the LD_PRELOAD mechanism.

Anything based on the LD_PRELOAD mechanism *always* has serious weaknesses, e.g., it can't redirect setuid root programs, and there will always be calls that it doesn't redirect. Also, the GNU C library normally make it impossible to redirect "internal" calls inside the GNU C library, making the "union mount" abstraction especially leaky. You can recompile the GNU C library to enable redirection (using "--disable-hidden-plt" option), but for most users, recompiling their C library is not a practical solution. Even if you had "--disable-hidden-plt", this kind of tool will always be a leaky abstraction. At best it's a set of heuristics that try to "usually work". There will always be cases where a request to access something will end up being set to the underlay (or rarely, the overlay) when it should have been redirected, typically leading to failure to create or read the right file.

You can also view my home page.