Releasing Free/Libre/Open Source Software (FLOSS) for Source Installation

David A. Wheeler

2009-04-13 (revised 2010-03-03)

If you’ve written (or started to write) some Free/Libre/Open Source Software (FLOSS), please follow the time-tested community standards for releasing FLOSS software when you want people to be able to install it from source code. This article will briefly explain why you should follow standard release conventions, and what they are.

Why Follow Standard Release Conventions?

First, the why. The quick answer is, “because it makes it easy to install your software - including creating packages to do so”. Modern Linux and BSD systems (as well as MacOS and MS Windows) normally install programs via “packages”; on Linux and BSD systems, this automatically installs all the necessary dependencies too. These packages are much easier to create and maintain if you follow standard release practices.

I’ve written a number of FLOSS programs (such as flawfinder and SLOCCount), and participated in other development, but I didn’t really understand the importance of the release standards until I started work on open proofs. I quickly learned that the packaging concept was simple, but some programs were hideously painful to package because they hadn’t followed the usual release standards. I knew some of these practices before, but I sometimes skimped on them before I understood the nightmare I was (unintentionally) inflicting on others.

I intend to modify my future releases to better meet usual release practices, now that I’ve “learned better”... and I hope to spread the word so that others will do the same.

What are the Standard Release Conventions?

Now, the what. These documents in particular describe good release practices:

  1. The GNU Coding Standards, especially the material on the release process.
  2. Software Release Practice HOWTO by Eric Steven Raymond.
  3. Filesystem Hierarchy Standard (FHS) — this describes where files are supposed to go.

If you’re releasing FLOSS, you want it to be easy to package, so it might be a very good idea to know about typical packaging guidelines. You can look at:

At least look over the first set (the GNU coding standards’ release process, Software Release Practice HOWTO, and the Filesystem Hierarchy Standard (FHS)).

A Quick Checklist

I know that the above is a lot information, so here is a quick checklist of some of the most important guidelines (in my opinion), based on problems that I’ve had:

  1. Pick a good, simple, and unique Google-able name for the package and its executables. Limit your package name to the lower case letters a-z, digits 0-9, and the four punctuation characters “-._+”. The Software Release Practice HOWTO’s chapter on good project and archive naming practice and the Fedora naming guidelines may help. Make sure the uppercase/lowercase distinction is irrelevant; some filesystems can’t distinguish them, and package names tend to be all lowercase anyway. Try to pick a name that’s unique — at least unique among packages, and preferably, one that doesn’t have any Google hits at all (so that people can find your package easily). A good example of a bad name is Why, because it is completely un-Googleable. If you don’t pick a good name, then in the best possible case, people will rename your program, often leading to confusion. At worst, they will never find your program at all. Similarly, try to make to make the name of the executable(s) unique; they will probably be installed in /usr/bin with all other user programs.
  2. Identify the version in MAJOR.MINOR... format or by using dates in YYYYMMDD format. People and software must be able to easily determine if a version is older or newer than another one. Using version numbers (e.g., 2.2.2) or date values in YYYYMMDD format (e.g., 20090413) makes that clear. The filenames of major releases should have the form NAME-VERSION.FORMAT, e.g., “myprogram-1.1.tar.gz”, so that even after downloading there are no questions. Please use a version control system, but please make occasional “real” releases that are single-file archives; that way, people can easily know what version is ready to use, and how to get it.
  3. Use a standard, widely-used, GPL-compatible FLOSS license — and say so. One of the worst mistakes a FLOSS project can make is to use a non-standard license (especially a home-grown one) or fail to state a license at all. “Home-grown” licenses are often not even FLOSS, impede reuse of other code, can delay packaging or use by years, and will dissuade potential co-developers. Many organizations and distributions explicitly forbid using software that fails to include license statement; failing to state the license makes the software useless to many. Use one of the standard FLOSS licenses (e.g., MIT, BSD-new, LGPL, or GPL), and be sure your license is GPL-compatible. Fedora’s licensing page has more info. You do not need to use the GPL itself, but using a GPL-incompatible license is a mistake, in part because most FLOSS is GPL’ed. Do not write your own FLOSS license. Clearly state in your documents - and code header files - what the license is.
  4. Follow good distribution-making practice, in particular, make sure tarballs always unpack into a single new directory named NAME-VERSION. The Software Release Practice HOWTO’s “Good distribution-making practice” goes into this. Make sure that tarballs unpack into a subdirectory, or unwary users will end up with unwanted junk in directories they use. By doing this, you’ll fulfill people’s expectation that the first two steps to do are (1) unpack (untar) the file, and (2) “cd” into its directory.
  5. Support the standard invocation to configure, build, and install it: ./configure --prefix=/usr ; make; make install. Many people expect that, after the unpack-and-cd steps, you’ll perform the normal configure, make, make install process. Please implement configuring, building, and installing this way, because this presumption is widespread (e.g., here, here, here, and here). You do not need to use the autoconf/automake tools to implement this (though you can) - just conform to the usual external interface. See the GNU Coding Standards’ “Managing the Release Process” for more. Don't interleave the configuring, building, and installation steps; in particular, the build ("make") step should create all the files to be installed, and the install step should do nothing but install the files created by the build step. The first two steps should not require root privileges; the latter one may require privileges depending on where files are being installed. The next three bullets discuss the assumptions of each of these three steps.
  6. Support the standard ./configure options like --prefix, --exec-prefix, --bindir, --libdir, and so on. By supporting the standard “configure” options you make it easy to install the program wherever the user wants it. Supporting “prefix” is especially important; “prefix” is prepended to all filenames to be installed, and should default to “/usr/local” (for a system-wide installation that’s not part of the system). When the program is packaged for a given system, the prefix is typically changed to “/usr” instead. You should support installation by a single unprivileged user, in which case the prefix would typically be “$HOME”. There is a reason that people need control over these values, for example, never assume that “libdir” is the same as “$(prefix)/lib” — on many 64-bit systems it is “$(prefix)/lib64”. Here’s how configuration should work. It’s nice to support --srcdir, but I wouldn’t worry about that; --srcdir can be awkward to support, and is far less important (storage space is plentiful nowadays, so you can just make a copy and then build). But options like --prefix and --libdir are vital.
  7. Create a makefile that can rebuild everything and uses makefile variables (including applicable standard standard makefile variable names and targets). By default, invoking “make” should rebuild/recompile everything (by tradition this target is named “all”). Makefiles should be able to rebuild everything; many distributions forbid using pre-built binaries or libraries in packages (since they can’t then fix problems). It’s okay to include pre-built binaries/libraries in a source distribution, as long as “make clean” (or “make maintainer-clean” or “make really-clean”) can get rid of them.

    Makefiles should maximally use makefile variables so they can be easily overridden where necessary, e.g., “make CC=mygcc”. Use standard makefile variable names where they apply (e.g., prefix, exec_prefix, bindir, libdir, etc.). GNU publishes useful makefile conventions. At the very least, a default “make” should build everything, and “make install” should install the results, but it’s often helpful to support other targets like “clean” and “uninstall”.

    You don’t need to create a makefile directly; many programs use makefile generators, such as the one in Cmake and autoconf/automake. Just make sure that the generated makefile(s) support these conventions.

    Note: be sure to mark phony targets (like “all”, “clean”, and “install”) with the .PHONY mark, like this: 

         .PHONY : clean
    This is particularly important for “install”; without it, “make install” will probably do nothing on filesystems that ignore case distinctions (because the make will see that the “install” file already exists). In general, marking phony targets (targets that aren’t really files) will eliminate mysterious errors and document more clearly what is what.

    Even if you use a completely different build system that does not use make, at least include a tiny makefile that invokes the “real” make system. There’s no reason you have to use make directly if you don’t want to. For example, today’s FLOSS Java applications tend to use Ant or Maven. Cmake is a very useful cross-platform build system, with a lot of users (Cmake generates makefiles, so it automatically meets the requirement to provide a makefile). However, make sure you include a small makefile that can invoke your build and installation system. That way, people don’t need to figure out how to invoke your make system, and programs can be automatically built and installed regardless of your preferred build system.

  8. “make install” should not rebuild anything; simply copy files to locations as specified by DESTDIR, prefix, and so on. Most packaging systems, including Ubuntu, Debian, and Fedora’s, expect programs to be completely built by the steps above, and then split “installation” into two steps: An “install” to an intermediate directory (where the files will then be collected), and a final “install” done on the user’s actual system. This is easily done if your installation program is invoked using “make install” and supports a few standard conventions. The install rule should “not modify anything in the directory where the program was built, provided ‘make all’ has just been done... The commands should create all the directories in which files are to be installed, if they don’t already exist.” In most cases using “cat” in “make install” is a mistake; “make install” should simply make directories and copy already-built files into the appropriate directories, so think carefully before adding any command other than mkdir, install, cp, and ln. Please support the DESTDIR convention, that is, they should always install into the directory $(DESTDIR)$(prefix) where DESTDIR can be set by the user. (This implies that you should always create directories before installing into them; $(DESTDIR)/usr/bin might not exist!) Packaging can be very painful if you don’t support DESTDIR. Automating DESTDIR can be a pain, so it’s best if the program supports it to start with. If your package doesn’t support DESTDIR, at least make sure that the “make install” target only uses a few simple commands to place the files in their final positions (e.g., mkdir, install, cp, and ln), so that the package Auto-DESTDIR can automatically support DESTDIR. The GNU documents recommend not depending on mkdir supporting “-p”, but I think this is obsolete advice; “mkdir -p” has become very widespread, and it is even included in the Single Unix Standard version 3, so I think it’s okay to depend on it now.
  9. Do not require web access or interactivity for building, testing, or installing the software. Often these aren't permitted or possible. If you must interact with a user before installation can complete, install all the files without interaction, and detect when a user tries to start the program for the first time. Then display information on how to complete the installation, or interactively complete the installation.
  10. Document the external tools/libraries needed for building and running, and make it easy to remove them and install them separately. Document in your README or INSTALL file the external tools/libraries you need for building and running your software, including a URL for each uncommon one. Most importantly, make it easy for users to separately install and update those tools/libraries!

    It’s okay if you include a copy of an externally-developed tool or library, as a convenience for people who don’t want to download it separately, but do not require people to use these embedded versions. Over time, separate tools/libraries get updated, and whatever version is embedded in your release will be obsolete. Even if you sent the latest version at the time, external programs will be upgraded later. Try to ensure that users can use dynamic libraries to load the latest versions, if that’s practical. Unfortunately, many Java programs are especially bad at this. Many distribution guidelines forbid using bundled libraries in their packages, for example, see Fedora's "No bundled libraries" rule and Debian's policy on convenience copies.

  11. If you patch an external library/tool, get the patch upstream. If you don’t, your “local” version will stagnate while the original tool gets many improvements.
  12. Use pkg-config when finding or installing libraries. Freedesktop.org's pkg-config is relatively new but has swept the world of POSIX systems because it's an easily and simple solution to a more complicated problem. If you're compiling an application, it can tell you which flags to use. If you're creating a library, please create and install the pkg-config .pc files, so others can use pkg-config.
  13. Use standard user interfaces. For command line tools, use “-” single-letter options, “--” long-name options, and “--” by itself to signal “no more options”. For GUI tools, provide a .desktop file. Make it easy for people to start up your application. GNU standards for command line interfaces notes standard option conventions for command line interfaces, including some standard long names like “--version” and “--help”. Once installed, GUI programs should be runnable from the main menu, which requires that a .desktop entry file be created and correctly installed; see freedesktop.org’s desktop entry specification.
  14. Where sensible, make the application useable as a library or via the command line. This makes it possible to build a larger program (using libraries or scripts) that uses your program/library.
  15. Include tests (as “make check”). Users and packagers want to know if the program is working as intended, and anyone who makes an improvement to it will want to know if the program is still working. So, include easily-invoked test cases, where you send in inputs, produce outputs, and check that the outputs are correct. Tests that cover a significant amount of the program’s functionality and implementation are best, but even a cursory test (a “smoke test”) is better than nothing. Set up your test suite so that it’s easy to add new tests. Make sure the test will work on the program built even if another version of the software is installed.
  16. Include documentation. Typically you’ll have a brief README (what is this program?) and INSTALL (how to install it and any requirements for tools). It’s a good idea to have a man page and additional separate documentation, sufficient to answer the questions: What does your program do (sufficient so the user can determine if they want it)? What are some sample invocations? What are its options? What is its input format or API? What is output format or result? What are its strengths or weaknesses? What is the project’s URL?
  17. Make it portable. If your software can port to different architectures and platforms, then it’s probably flexible enough to handle likely future changes. Follow formal and informal standards, such as the Single Unix Specification. Consider using checking tools like the Linux Foundation's AppChecker.
  18. Support internationalization/locationalization (i18n/L10n). Use tools (like gettext) for any command-line or GUI interface so that people can use their native language in such interfaces. Do not assume that characters all fit in one or two bytes (they do not). I recommend using UTF-8 to encode characters in most cases, but in some cases (e.g., Java, Windows APIs) it may be easier to use a different encoding; if you do, beware of their gotchas (some glyphs take more than 2 bytes in UTF-16, and you have to worry about byte ordering and the byte-order mark).

Related pages

Here are a few related pages:

There are efforts to help create packages for many different systems simultaneously. For example, the Linux Foundation announced in April 2009 that it would host a build service to build packages for multiple systems simultaneously (based on the OpenSUSE build service). But these are much easier if you follow the guidelines above.

2009: software installation in GNU/Linux is still broken -- and a path to fixing it argues that GNU/Linux distributions should change how application software is installed. That's worth discussing, but I suspect that if there are any such changes, they will primarily work only for software that follows guidelines like the ones above.

Feel free to see my home page at http://www.dwheeler.com. You may also want to look at my paper Why OSS/FS? Look at the Numbers! and my book on how to develop secure programs.

(C) Copyright 2009 David A. Wheeler.