Introduction to the Autotools (autoconf, automake, and libtool)

This page has information on the “autotools”, a set of tools for software developers that include at least autoconf, automake, and libtool. The autotools make it easier to distribute source code that (1) portably and automatically builds, (2) follows common build conventions (such as DESTDIR), and (3) provides automated dependency generation if you’re using C or C++. They’re primarily intended for Unix-like systems, but they can be used to build programs for Microsoft Windows (this requires some additional facilities, e.g., MSYS, Cygwin, or cross-compilation libraries). The autotools are not the only way to create source code releases that are easily built and packaged. But they’re one of the most widely-used, especially for programs that use C or C++ (though they’re not limited to that).

Years ago, the autotools were hard for developers to use and they had lousy documentation. The autotools have significantly improved over the years, and better documentation is available too. Unfortunately, there’s a lot of really obsolete documentation, and a lot of obsolete complaints about autotools, and it’s a little hard to get started. I hope this page will help you get started quickly.

My video tutorial: Introduction to the Autotools



You can instead view this via YouTube as:

You can also download the webm video file.

The music in the video is from “Discordian Nights” from paniq’s Beyond Good and Evil (I’m not a follower of Nietzsche or discordianism, but good music is good music). I used clips from its front and tail. This song is also licensed under the CC-BY-SA 3.0 license.

I’ve also posted some related files. The slides are available as the PDF of “Introduction to Autotools (autoconf, automake, and libtool)” [Open Document Format]. The shell script doit executes the tutorial; I use the script to automatically do every step, to make sure I don’t forget some step. The files better-configure.ac and template-configure.ac contain versions of the “configure.ac” discussed in the tutorial.

My tutorial focuses on using the tools together. You do not have to use them that way (e.g., you could use just autoconf), but I think it is much easier to use and explain them together.

How the video was created

The entire video was created using only free/libre / open source software (FLOSS) tools and patent-free formats. I count royalty-free patent use as patent free.

The presentation itself was created using OpenOffice.org and LibreOffice. I recorded the video and audio commentary using gtk-recordmydesktop (set to 30 frames per second to create smoother animations). That tool lets me easily pause and stop, and creates a simple .ogv audiovisual file (this is Ogg Theora for video, and Ogg Vorbis for audio, in an Ogg container; all are patent free). The normal pulseaudio tool let me select my recording input (so I could switch inputs between the microphone and the music). I ran GNOME 3 on Fedora, using the GNOME terminal (using a “big” profile I created, so that even when reduced in size, people could still read it). I downloaded the music in the lossless, and patent-free, FLAC audio format.

I also processed these initial materials using FLOSS tools and patent-free formats. I used audacity to take the music file (in FLAC) and create the music samples, including the fade-in and fade-out, and had it generate the patent-free Ogg Vorbis audio format. I used ffmpeg to convert the .ogv audiovisual files to the webm format, which seems to be more widely viewable. I also used ffmpeg to create a front “poster” image for my website in jpg format (again, this is patent free). I used mkvmerge to combine the webm pieces into the final work, and mkvpropedit to set the title of that work.

I used a Plantronics Audio 655 headset with microphone; these are more expensive, but they produce better quality results than many. I got the headset tip from Randal Schwartz when I did an interview with him (#160) on FLOSS weekly. I turned up the microphone volume; I find that many online tutorials’ voices are almost inaudible, and I wanted this to be something people can actually hear. The increased volume means that the results are a little breath-y, but I thought that is a reasonable trade-off; I think it is more important to be clearly understood.

Of course, I used the autotools themselves; other tools used included the bash shell, GNU make, and the gcc C compiler. I used the git software configuration management (SCM) program (aka version control program); you do not have to use git, but I thought it was important to show how autotools can be used together with a common SCM, and git is very common.

I use the HTML5 <video> tag with webm format (webm is not encumbered with royalty-bearing patents). I intentionally did not include proprietary formats (like Flash) or patent-encumbered formats (like H.264). I tested this with Firefox (both Linux and Windows), Chrome, and Internet Explorer 9, and did have to do a few tweaks to make sure it worked in all of them. Firefox requires that the MIME types be correctly reported, which I think is silly but understandable; in any case, a simple “.htaccess” file containing “AddType video/webm .webm” solved that problem. Internet Explorer 9 (IE 9) was trickier; even after the webm code is installed, it did not always work. I had to add a special head entry for IE 9 to use its standards-compliant rendering, by using:
<meta http-equiv="X-UA-Compatible" content="IE=edge" >.

Other autotools tutorials and references

For more tutorials on how to use the autotools, see:

The GNU manuals are relatively up-to-date and they are reasonably good reference materials. Each documents separate pieces of the autotools: autoconf, automake, and libtool. However, they don’t make good tutorials by themselves. In particular, each manual focuses primarily on just that one tool instead of describing how to use the tools together. They’ve improved that some over the years, in particular, the autoconf manual does try to give the bigger picture, but I think they’re still not the best place to start to learn how to use them.

Do not start with the book or website “GNU Autoconf, Automake and Libtool” by Gary Vaughan et al., published originally by New Rider in 2000. This book, often called the “Goat book”, was a really good and useful book in 2000. There’s an online version that has had some tweaks through 2006. However, the autotools has changed substantially since then; this material is interesting historically, and might be useful to you once you more thoroughly understand autotools, but it is a terrible place to start learning today. For example, they spend a lot of time on how to invoke each tool in turn (instead of just using today’s “autoreconf” which does all that). Start with other material first.

The blog post Autoconf and Automake Tutorial tries to be a simple tutorial, but it doesn’t emphasize the “right way” to do things. For example, instead of using autoreconf, it goes through a long dance of the various specific tools. It also does echo...exit instead of the simpler AC_MSG_ERROR. On the other hand, it emphasizes AC_CHECK_LIB, which is useful but I omitted in my tutorial (I emphasized pkg-config, which is easier to use but not all libraries support it).

When you are first learning how to use the autotools, avoid anything written about the autotools before around 2006. Things are just too different today, and you will waste a lot of time. You can tell that autotools information is obsolete if it talks about creating “configure.in” (the modern conventional name is “configure.ac”), fails to emphasize using autoreconf and “autoreconf -i” (and instead talks a lot about invoking by hand various tools, such as aclocal or autoheader), or fails to talk about creating an “m4” subdirectory (and instead talks about creating an aclocal.m4 file).

Autotools tips

Here are a few tips when using the autotools, most of which is also covered in my video tutorial: a template for configure.ac, a style guide for m4, using non-recursive make, and creating an m4 subdirectory.

Template for configure.ac

The file configure-template.ac.txt is a simple template showing some suggested settings to put in your “configure.ac” file. It includes comments explaining why you should use it.

Style guide for configure.ac

The autotools (particularly autoconf) are based on the “m4” macro processor, which does create some gotchas. Some people do hate the implications of the way the autotools use m4. There are definitely cons to the way the autotools work. However, a few simple conventions can avoid a lot of problems that people complain about in practice:

  1. Bracket all macro parameters with [...] (you don’t need to bracket integers). The square brackets are needed in surprising places; it’s best to always use them (they’re not necessary for integers). If you always quote, then you won’t forget to quote when you have to. This is true even if the parameters themselves invoke macros. The reason for this rule is that figuring out when you have to is trickier than it might appear. Even if you’re an m4 guru, your future co-developers may not be. I think it’s better to follow a simple, easily-understood convention that eliminates many tricky problems.
  2. Whitespace matters. Whitespace before a macro parameter is okay (it’s ignored outside []), but do not put whitespace before the macro invocation’s “(“ nor after macro parameters (before “,” or “)”). Anything that goes through m4 is much more sensitive to whitespace than most other languages, and this includes “configure.ac” (a primary input to the autotools). The m4 macro processor implements macro calls using the form NAME(PARAM1, PARAM2, ...) which look (and act) a lot like function calls. Just like many other languages, whitespace just before parameters is ignored in m4. However, whitespace is not ignored before the opening and closing parentheses of a macro call, nor are they ignored before the commas that separate parameters. Thus, you should almost never have whitespace there when using m4. In particular, if you put whitespace before an opening parentheses, the macro expansion will silently fail!
  3. Use AS_IF & AS_CASE, not if...fi & case...esac. These tell autoconf what’s conditional.
  4. To pass a literal parameter (including something with [...]), surround it with another [...] pair. These need not be adjacent to outer [...], but it’s usually best to do so. E.G., AC_MSG_ERROR([[These are [square brackets]!!]]) AC_LANG_SOURCE([[int main() { return 0; }]])
  5. Prefer “test”, not [...], for conditions. This avoids needing those extra [...].
  6. (Not mentioned in the video) Consider using the GNUlib "manywarnings" module to maximally enable gcc warning flags. Doing this early in development will detect a lot of problems, and once you know about a warning, you'll avoid doing it in the future. You can try to enable these later, but that creates a lot of extra work; it's best to maximally enable warnings early in development.

Thus, a typical “configure.ac” file for the autotools might have lines that look like this (note the absence of whitespace before the parentheses and commas, and the square brackets around parameter constants):

  AC_INIT([myprogram], [1.01], [myname@test.com])

This example tells autoconf that we are configuring program myprogram version 1.01, and that bug reports should go to myname@test.com.

Non-recursive make in the autotools’ automake

When programs get big, it’s convenient to divide the source files into directories and subdirectories. Historically, people would create a makefile in each directory that would build the files in that directory, and have makefiles recursively call down to other makefiles. This approach is called a “recursive make”; it’s the obvious way to scale up. The autotools do support recursive make, if you want to do it this historical way.

However, the lucid “Recursive Make Considered Harmful” by Peter Miller shows that the traditional “recursive make” is actually a bad approach. Recursive make tends to produce wrong results, is slower, and is hard to maintain. It is far better to execute a single make process instead of many separate ones. This one process can read a single makefile or multiple makefiles.

Automake supports non-recursive make (in addition to recursive make). For non-recursive make, just use this in the configure.ac file:

 AM_INIT_AUTOMAKE([subdir-objects])
and include this in Makefile.am:
 ACLOCAL_AMFLAGS = -I m4 --install

From then on, refer to source filenames using their full relative name starting from the top directory (aka $(srcdir))... in other words, your filenames will typically have slashes in them. You can still split makefiles into smaller files, but I find that a single long makefile is easier to maintain; just organize it into sections, like you would a book. The Automake Mythbuster explains the basics. Erik de Castro Lopo also reports that non-recursive makes work well with automake.

Creating an m4 subdirectory

The modern autotools convention is that you should use an “m4” subdirectory for internal “m4” files, instead of the default aclocal.m4. In theory, this easy to do:

  1. First, run:
     rm aclocal.m4
     mkdir m4
    
  2. Add to configure.ac:
     AC_CONFIG_MACRO_DIR([m4])
    
  3. Add to Makefile.am:
     ACLOCAL_AMFLAGS = -I m4 --install
    

But this won’t always work as-is. Due to a bug in the autotools (which I expect will be fixed in the next release of automake), an “autoreconf -i” will fail in many cases. Also, some SCMs (such as git) won’t store empty directories, so a created directory will not “stick around”.

The solution is to do the actions above, but then work around this with:

  1. Create a dummy file in the m4 directory:
     touch m4/NOTES
    
  2. Put that dummy file under version control. For example, if you’re using git, do:
      git add m4/NOTES
    
  3. Add to Makefile.am (to force redistribution):
     EXTRA_DIST = m4/NOTES
    
  4. Now you can regenerate all needed files:
     autoreconf -i
    

Note that AC_CONFIG_MACRO_DIR did not actually do anything in older versions.

pkgconfig

"pkgconfig" is great for libraries. If you create .pc file(s) for your library, you should probably use AX_CREATE_PKGCONFIG_INFO. Some people use AC_CONFIG_FILES to generate the .pc files; this is easy, but the problem is that it will generate .pc files at the wrong time - it generates them during configuration, not during build (make). If someone changes the prefix or libdir during make (which they're allowed to do), then the .pc files will be wrong.

Are the autotools perfect?

The autotools are not perfect. There are other good tools that have some similarities, and depending on what you’re doing, they might be better (examples of other build systems include Cmake, scons, rake, Apache Ant, and Apache Maven). Note, however, that the autotools don't just build software, they detect various features to make it much easier to build software that ports to a wide variety of systems. Sure, you should use standard interfaces, but sometimes there's no standard interface or the interface has serious bugs/limitations. If you have that problem, the autotools can help. Many other tools presume that there's no portability issue, or that source code recipients will manually figure out how to get your software to compile.

Richard WM Jones has a helpful summary of the complicated view many developers have of the autotools: “I’m not going to pretend that autoconf is good. But for the Fedora cross-compiler project I had to cross-compile about a hundred packages using all sorts of different build systems, and autoconf/automake-based ones were by far the most reliable. Autoconf isn’t good, but it’s much better than the alternatives once you start needing to compile on many different platforms, cross compile, or integrate with Linux distro packaging systems.”

Still, the autotools are better than they used to be.

Some problems in learning to use the autotools stem from the basic issue that autoconf is built on the m4 pre-processor, which is a tool not widely used otherwise. That is definitely an issue, and it’s too baked into autoconf to change. The good news is that simple style conventions (e.g., with brackets and whitespace) can make it easier to use.

Using autotools used to involve repeatedly working around their limitations and bugs. There are still some annoying bugs and limitations that make them harder to use than necessary (e.g., the autoreconf in autoconf version 2.68 fails to create an “m4” directory, leading to the work-around above). But it is far better than the mess it used to be. A look at the circa-2000 Goat book shows how far things have come; that book spends endless pages describing how to work around autotools bugs or limitations of the time (just look at the number of references to a “future release”, “future version”, and so on). A lot of that has been fixed; the process was painful, but the result is a more consistent tool than the past.

Some of the autotools issues are fundamental to their goals. Their goal is to make it easy to build a system, using the native tools, and there are variations in the native tools. In the longer term, forcing native tools to get fixed (so that people don’t have to work around their bugs) and getting their functionality updated (so that people don’t have to work around their limitations) would help best in the long term.

My goal in explaining the autotools is that I want useful packages to be released by their developers in a format that’s easy to install. This doesn’t apply to client users downloading Javascript (for example), but even for Javascript, you need to install software on the server so that it’s easily managed. For example, I really want packages to support DESTDIR, since that greatly simplifies package management. If understanding the autotools helps in that goal, then that’s a good thing.

In particular, the autotools implements a large set of very useful conventions for end-users (e.g., DESTDIR, cross-compilation, lots of standard options), it can directly use basic tools of the underlying system (instead of replacing them), and it can handle a remarkable variety of systems — so for end-users who must compile software for a variety of systems, there are advantages to the autotools.


You might also want to read Releasing Free/Libre/Open Source Software (FLOSS) for Source Installation