David A. Wheeler's Blog

The magic cookie parable

Sun, 15 Apr 2012 09:18 GMT

In some presentations I include the “magic cookie parable”. Here is the parable, for those who have not heard it (I usually hold a cookie in my hand when I present it). Anyway…

I have in my hand… a magic cookie! Just one cookie will supply all your food needs for a whole year. What is more, the first one is only $1. Imagine how much money you will save! Imagine how much time you will save!

Ah, but there’s a catch. Once you eat the magic cookie, you can only eat magic cookies, as all other food will become poisonous to you. What’s more, there is only one manufacturer of magic cookies.

Do you think the cookie will be $1 next year? How about for the rest of your life? Are you as eager to eat the cookie?

Is that a silly parable? It should be. Yet many people accept information technology (IT), for themselves or on behalf of their organizations, that are fundamentally magic cookies. Too many are blinded into accepting technology that makes them, or their organization, completely at the mercy of a single supplier. You can call dependence on single supplier a security problem, or a supply chain problem, or a support problem, or many other things. But no matter what you call it, it is a serious problem.

Now please do not hear what I am not saying. I am not here to attack any particular supplier. In fact, we all need suppliers, and I am grateful for suppliers! The problem is not the existence of suppliers; the problem is excessive dependency on any one supplier.

There are only a few information technology (IT) strategies that counter sole-supplier dependency that I know of:

Build and control it yourself. In a few cases this is reasonable, but in most cases, that is too expensive and it risks obsolescence.
Open systems/open standards. Here, you ensure that your system is made of modular parts with key interfaces covered by standards; that way, you can later switch to a different product. This can work, but suppliers may create proprietary extensions that (if you are not careful) lock you in anyway.
A Combination. That is, a combination of the above.

Before getting locked into a single supplier, count the true cost over the entire time it will occur. Sure, in some cases, it may be worth it anyway. But you may find that this true cost is far higher than you are willing to pay. (The cookie image is by Bob Smith, released under the CC Attribution 2.5 license. Thank you!)

DoD Open Source Software (OSS) Pages Moved

Thu, 05 Apr 2012 16:33 GMT

The US Department of Defense (DoD) has changed the URLs for some of its information on Open Source Software (OSS). Unfortunately, there are currently no redirects, and that makes them hard to find (sigh). Here are new links, if you want them.

A good place to start is the Department of Defense (DoD) Free Open Source Software (FOSS) Community of Interest page, hosted by the DoD Chief Information Officer (CIO).

From that page, you can reach:

Clarifying Guidance Regarding Open Source Software (OSS) Memorandum by David M. Wennergren, 16 October 2009 - this is the overall DoD policy on OSS
Open Technology Development (OTD): Lessons Learned & Best Practices for Military Software - OSD Report, May 2011 - this is guidance on how to develop and use OSS
DoD Open Source Software (OSS) Frequently Asked Questions (FAQ) - this has lots of answers to questions about OSS, focusing on DoD issues
Use of Free and Open-Source Software (FOSS) in the U.S. Department of Defense - 2003 Study by MITRE Corporation performed for DoD - a report by MITRE’s Terry Bollinger showing that the DoD was already using OSS widely in 2002-2003. Funny fact: In the Acknowledgements page, I am listed second.

If you are interested in the topic of DoD and OSS, you might also be interested in the Military Open Source Software (Mil-OSS) group, which is not a government organization, but is an active community.

Insecure open source software libraries?

Tue, 03 Apr 2012 14:00 GMT

The news is abuzz about a new report, “The Unfortunate Reality of Insecure Libraries” (by Aspect Security, in partnership with Sonatype). Some news articles about it, like Open source code libraries seen as rife with vulnerabilities (Network World) make it sound like open source software (OSS) is especially bad. (To be fair, they do not literally say that, but many readers might infer it.)

However, if you look at the report, you see something quite different. The report directly states that, “This paper is not a critique of open source libraries, and we caution against interpreting this analysis as such.” They only examined open source Java libraries, but their “experience in evaluating the security of hundreds of custom applications indicates that the findings are likely to apply to closed-source and commercial libraries as well.”

This is a valuable report, because it points out a general problem not specific to OSS.

The problem is that software libraries (OSS or not) are not being adequately managed, leading to a vast number of vulnerabilities. For example, the report states that “The data show that most organizations do not appear to have a strong process in place for ensuring that the libraries they rely upon are up-to-date and free from known vulnerabilities.” They point out that “development teams readily acknowledge, often with some level of embarrassment, that they make no efforts to keep their libraries up-to-date.” They also note that “Organizations download many old versions of libraries… If people were updating their libraries, we would have expected the popularity of older libraries to drop to zero within the first two years. However, the data clearly show popularity extending back over six years…. The continuing popularity of libraries for extended months suggests that incremental releases of legacy applications are not being updated to use the latest versions of libraries but are continuing to use older versions.” They recommend that software development organizations inventory, analyze, control, and monitor their libraries, and give details on each point.

I should note that I’ve been saying some of these things for years. For years I have said that you should evaluate OSS before you use it… some software is better than others. Back in 2008 I also urged developers to use system libraries, at least as an option; embedding libraries often leads over time to the use of old (and vulnerable) libraries. An advantage of OSS is that many people can review the software, find problems (including vulnerabilities), and fix them… but this advantage is lost if the fixed versions are not used! And of course, if you develop software, you need to learn how to develop secure software. As the report notes, tools can be useful (I give away flawfinder), but tools cannot replace human knowledge and human review.

For more information, you should see their actual report, “The Unfortunate Reality of Insecure Libraries” (by Aspect Security).

Software patents may silence little girl

Mon, 26 Mar 2012 01:22 GMT

Software patents are hurting the world, but the damage they do is often hard to explain and see.

But Dana Nieder’s post “Goliath v. David, AAC style” has put a face on the invisible scourge of software patents. As she puts it, a software patent has put her “daughter’s voice on the line. Literally. My daughter, Maya, will turn four in May and she can’t speak.” After many tries, the parents found a solution: A simple iPad application called “Speak for Yourself” that implements “augmentative and alternative communication” (AAC). Dana Nieder said, “My kid is learning how to ‘talk.’ It’s breathtaking.”

But now Speak for Yourself is being sued by a big company, Semantic Compaction Systems and Prentke Romich Company (SCS/PRC), who claims that the smaller Speak for Yourself is infringing SCS/PRC’s patents. If SCS/PRC wins their case, the likely outcome is that these small apps will completely disappear, eliminating the voice of countless children. The reason is simple: Money. SCS/PRC can make $9,000 by selling their one of their devices, so they have every incentive to eliminate software applications that cost only a few hundred dollars. Maya cannot even use the $9,000 device, and even if she could, it would be an incredible hardship on a Bronx family with income from a single 6th grade math teacher. In short, if SCS/PRC wins, they will take away the voice of this little girl, who is not yet even four, as well as countless others.

I took a quick look at the complaint, Semantic Compaction Systems, Inc. and Prentke Romich Company, v. Speak for Yourself LLC; Renee Collender, an individual; and Heidi Lostracco, an individual, and it is horrifying at several levels. Point 16 says that the key “invention” is this misleadingly complicated paragraph: “A dynamic keyboard includes a plurality of keys, each with an associated symbol, which are dynamically redefinable to provide access to higher level keyboards. Based on sequenced symbols of keys sequentially activated, certain dynamic categories and subcategories can be accessed and keys corresponding thereto dynamically redefined. Dynamically redefined keys can include embellished symbols and/or newly displayed symbols. These dynamically redefined keys can then provide the user with the ability to easily access both core and fringe vocabulary words in a speech synthesis system.”

Strip away the gobbledygook, and this is a patent for using pictures as menus and sub-menus. This is breathtakingly obvious, and was obvious long before this was patented. Indeed, it would have been obvious to most non-computer people. But this is the problem with many software patents; once software patents were allowed (for many years they were not, and they are still not allowed in many countries), it’s hard to figure out where to end.

One slight hope is that there is finally some effort to curb the worst abuses of the patent system. The Supreme Court decided on March 20, 2012, in Mayo v. Prometheus, that a patent must do more than simply state some law of nature and add the words “apply it.” This was a unanimous decision by the U.S. Supreme Court, remarkable and unusual in itself. You would think this would be obvious, but believe it or not, the lower court actually thought this was fine. We’ve gone through years where just about anything can be patented. By allowing software patents and business patents, the patent and trade office has become swamped with patent applications, often for obvious or already-implemented ideas. Other countries do not allow such abuse, by simply not allowing these kinds of patents in the first place, giving them time to review the rest. See my discussion about software patents for more.

My hope is that these patents are struck down, so that this 3-year-old girl will be allowed to keep her voice. Even better, let’s strike down all the software patents; that would give voice to millions.

Introduction to the autotools (autoconf, automake, libtool)

Mon, 19 Mar 2012 00:01 GMT

I’ve recently posted a video titled “Introduction to the autotools (autoconf, automake, and libtool)”. If you develop software, you might find this video useful. So, here’s a little background on it, for those who are interested.

The “autotools” are a set of programs for software developers that include at least autoconf, automake, and libtool. The autotools make it easier to create or distribute source code that (1) portably and automatically builds, (2) follows common build conventions (such as DESTDIR), and (3) provides automated dependency generation if you’re using C or C++. They’re primarily intended for Unix-like systems, but they can be used to build programs for Microsoft Windows too.

The autotools are not the only way to create source code releases that are easily built and packaged. Common and reasonable alternatives, depending on your circumstances, include Cmake, Apache Ant, and Apache Maven. But the autotools are one of the most widely-used such tools, especially for programs that use C or C++ (though they’re not limited to that). Even if you choose to not use them for projects you control, if you are a software developer, you are likely to encounter the autotools in programs you use or might want to modify.

Years ago, the autotools were hard for developers to use and they had lousy documentation. The autotools have significantly improved over the years. Unfortunately, there’s a lot of really obsolete documentation, along with a lot of obsolete complaints about autotools, and it’s a little hard to get started with them (in part due to all this obsolete documentation).

So, I have created a little video introduction at http://www.dwheeler.com/autotools that I hope will give people a hand. You can also view the video via YouTube (I had to split it into parts) as Introduction to the autotools, part 1, Introduction to the autotools, part 2, and Introduction to the autotools, part 3.

The entire video was created using free/libre / open source software (FLOSS) tools. I am releasing it in the royalty-free webm video format, under the Creative Commons CC-BY-SA license. I am posting it to my personal site using the HTML5 video tag, which should make it easy to use. Firefox and Chrome users can see it immediately; IE9 users can see it once they install a free webm driver. I tried to make sure that the audio was more than loud enough to hear, the terminal text was large enough to read, and that the quality of both is high; a video that cannot be seen or heard is rediculous.

This video tutorial emphasizes how to use the various autotools pieces together, instead of treating them as independent components, since that’s how most people will want to use them. I used a combination of slides (with some animations) and the command line to help make it clear. I even walk through some examples, showing how to do some things step by step (including using git with the autotools). This tutorial gives simple quoting rules that will prevent lots of mistakes, explains how to correctly create the “m4” subdirectory (which is recommended but not fully explained in many places), and discusses why and how to use a non-recursive make. It is merely an introduction, but hopefully it will be enough to help people get started if they want to use the autotools.

Debian GNU/Linux = $19 billion

Tue, 14 Feb 2012 17:53 GMT

Debian developer James Bromberger recently posted the interesting ”Debian Wheezy: US$19 Billion. Your price… FREE!”, where he explains why the newest Debian distribution (“Wheezy”) would have taken $19 billion U.S. dollars to develop if it had been developed as proprietary software. This post was picked up in the news article ”Perth coder finds new Debian ‘worth’ $18 billion” (by Liam Tung, IT News, February 14, 2012).

You can view this as an update of my More than a Gigabuck: Estimating GNU/Linux’s Size, since it uses my approach and even uses my tool sloccount. Anyone who says “open source software can’t scale to large systems” clearly isn’t paying attention.

New Hampshire: Open source, open standards, open data

Mon, 06 Feb 2012 01:05 GMT

The U.S. state of New Hampshire just passed act HB418 (2012), which requires state agencies to consider open source software, promotes the use of open data formats, and requires the commissioner of information technology (IT) to develop an open government data policy. Slashdot has a posted discussion about it. This looks really great, and it looks like a bill that other states might want to emulate. My congrats go to Seth Cohn (the primary author) and the many others who made this happen. In this post I’ll walk through some of its key points on open source software, open standards for data formats, and open government data.

First, here’s what it says about open source software (OSS): “For all software acquisitions, each state agency… shall… Consider whether proprietary or open source software offers the most cost effective software solution for the agency, based on consideration of all associated acquisition, support, maintenance, and training costs…”. Notice that this law does not mandate that the state government must always use OSS. Instead, it simply requires government agencies to consider OSS. You’d think this would be useless, but you’d be wrong. Fairly considering OSS is still remarkably hard to do in many government agencies, so having a law or regulation clearly declare this is very valuable. Yes, closed-minded people can claim they “considered” OSS and paper over their biases, but laws like this make it easier for OSS to get a fair hearing. The law defines “open source software” (OSS) in a way consistent with its usual technical definition, indeed, this law’s definition looks a lot like the free software definition. That’s a good thing; the impact of laws and regulations is often controlled by their definitions, so having good definitions (like this one for OSS) is really important. Here’s the New Hampshire definition of OSS, which I think is a good one:

”Unrestricted use of the software for any purpose;
Unrestricted access to the respective source code;
Exhaustive inspection of the working mechanisms of the software;
Use of the internal mechanisms and arbitrary portions of the software, to adapt them to the needs of the user;
Freedom to make and distribute copies of the software; and
Modification of the software and freedom to distribute modifications of the new resulting software, under the same license as the original software.”

The material on open standards for data says, “The commissioner shall assist state agencies in the purchase or creation of data processing devices or systems that comply with open standards for the accessing, storing, or transferring of data…” The definition is interesting, too; it defines an “open standard” as a specification “for the encoding and transfer of computer data” that meets a long list of requirements, including that it is “Is free for all to implement and use in perpetuity, with no royalty or fee” and that it “Has no restrictions on the use of data stored in the format”. The list is actually much longer; it’s clear that the authors were trying to counter common vendor tricks who try to create “open” standards that really aren’t. I think it would have been great if they had adopted the more stringent Digistan definition of open standard, but this is still a great step forward.

Finally, it talks about open government data, e.g., it requires that “The commissioner shall develop a statewide information policy based on the following principles of open government data”. This may be one of the most important parts of the bill, because it establishes these as the open data principles:

”Complete. All public data is made available, unless subject to valid privacy, security, or privilege limitations.
Primary. Data is collected at the source, with the highest possible level of granularity, rather than in aggregate or modified forms.
Timely. Data is made available as quickly as necessary to preserve the value of the data.
Accessible. Data is available to the widest range of users for the widest range of purposes.
Machine processable. Data is reasonably structured to allow automated processing.
Nondiscriminatory. Data is available to anyone, with no requirement of registration.
Nonproprietary. Data is available in a format over which no entity has exclusive control, with the exception of national or international published standards.
License-free. Data is not subject to any copyright, patent, trademark, or trade secret regulation. Reasonable privacy, security, and privilege restrictions may be allowed.”

The official motto of the U.S. state of New Hampshire is “Live Free or Die”. Looks like they truly do mean to live free.

Website back up

Fri, 20 Jan 2012 18:27 GMT

This website (www.dwheeler.com) was down part of the day yesterday due to a mistake made by my web hosting company. Sorry about that. It’s back up, obviously.

For those who are curious what happened, here’s the scoop. My hosting provider (WebHostGiant) moved my site to a new improved computer. By itself, that’s great. That new computer has a different IP address (the old one was 207.55.250.19, the new one is 208.86.184.80). That’d be fine too, except they didn’t tell me that they were changing my site’s IP address, nor did they forward the old IP address. The mistake is that the web hosting company should have notified me of this change, ahead of time, but they failed to do so. As a result, I didn’t change my site’s DNS entries (which I control) to point to its new location; I didn’t even know that I should, or what the new values would be. My provider didn’t even warn me ahead of time that anything like this was going to happen… if they had, I could have at least changed the DNS timeouts so the changeover would have been quick.

Now to their credit, once I put in a trouble ticket (#350465), Alex Prokhorenko (of WebhostGIANT Support Services) responded promptly, and explained what happened so clearly that it was easy for me to fix things. I appreciate that they’re upgrading the server hardware, I understand that IP addresses sometimes much change, and I appreciate their low prices. In fact, I’ve been generally happy with them.

But if you’re a hosting provider, you need to tell the customer if some change you make will make your customer’s entire site unavailable without the customer taking some action! A simple email ahead-of-time would have eliminated the whole problem.

Grumble grumble.

I did post a rant against SOPA and PIPA the day before, but I’m quite confident that this outage was unrelated.

Anyway, I’m back up.

Stop SOPA and PIPA

Wed, 18 Jan 2012 10:06 GMT

Please protest the proposed STOP (Stop Online Piracy Act) and PIPA (PROTECT IP Act). The English Wikipedia is blacked out today, and many other websites (like Google) are trying to awareness of these hideous proposed laws. The EFF has more information about PIPA and SOPA. Yes, the U.S. House has temporarily suspended its work, but that is just temporary; it needs to be clear that such egregious laws must never be accepted.

Wikimedia Foundation board member Kat Walsh puts it very well: “We [the Wikimedia Foundation and its project participants] depend on a legal infrastructure that makes it possible for us to operate. And we depend on a legal infrastructure that also allows other sites to host user-contributed material, both information and expression. For the most part, Wikimedia projects are organizing and summarizing and collecting the world’s knowledge. We’re putting it in context, and showing people how to make sense of it. But that knowledge has to be published somewhere for anyone to find and use it. Where it can be censored without due process, it hurts the speaker, the public, and Wikimedia. Where you can only speak if you have sufficient resources to fight legal challenges, or, if your views are pre-approved by someone who does, the same narrow set of ideas already popular will continue to be all anyone has meaningful access to.”

U.S. Department of Defense Removes Open Source Software Roadblocks (AppDev STIG)

Wed, 21 Dec 2011 19:33 GMT

The U.S. Department of Defense (DoD) has changed one of its key software development documents, making it even clearer that it’s okay to use open source software (OSS) in the DoD. This is good news beyond the DoD; if the US DoD can widely accept OSS, then maybe other organizations (that you deal with) can too.

That key document has the long title “Application Security & Development (AppDev) Security Technical Implementation Guide (STIG),” aka the AppDev STIG. The AppDev STIG includes some guidelines for how to write secure software, and a checklist for use before you can deploy custom software in certain cases. In the past, many people thought that using OSS in the DoD required special permission, because they misunderstood some of DoD’s policies, and this misunderstanding had crept into the AppDev STIG. The good news is that this has been fixed.

Here’s the basic background.

Open source software (OSS) is software where anyone can read, modify, and redistribute the source code (its “blueprints”) in original or modified form. OSS is widely used and developed in industry; some popular OSS includes the Linux kernel (the basis of Google’s Android), the Firefox web browser, and Apache (the world’s most popular web server). You can get quantitative numbers about OSS at http://www.dwheeler.com/oss_fs_why.html. There is a lot of high-quality OSS, and OSS is often very inexpensive even when you include installation, training, and so on.

Unfortunately, previous versions of the AppDev STIG were often interpreted as saying that using OSS required special permission. This document matters; DoD Directive (DoDD) 8500.01E requires that “all IA and IA-enabled IT products incorporated into DoD information systems shall be configured in accordance with DoD-approved security configuration guidelines” and tasks DISA to develop the STIGs. It’s often difficult to get systems fielded unless they meet the STIGs.

AppDev STIG version 3 revision 1 (an older version) said:

(APP2090.1: CAT II) “The Program Manager will obtain DAA acceptance of risk for all open source, public domain, shareware, freeware, and other software products/libraries with no warranty and no source code review capability, but are required for mission accomplishment.”

(APP2090.2: CAT II): “The Designer will document for DAA approval all open source, public domain, shareware, freeware, and other software products/libraries with limited or no warranty, but are required for mission accomplishment.”

Many people interpreted this as saying that any use of OSS required special permission. But where would the Defense Information Systems Agency (DISA), the author of the AppDev STIG, get that idea? Well, it turns out that this is a common misunderstanding of DoD policy. DoD Instruction 8500.2, February 6, 2003 has a control called “DCPD-1 Public Domain Software Controls” (http://www.dtic.mil/whs/directives/corres/pdf/850002p.pdf), which starts with this text:

Binary or machine executable public domain software products and other software products with limited or no warranty such as those commonly known as freeware or shareware are not used in DoD information systems unless they are necessary for mission accomplishment and there are no alternative IT solutions available.

A lot of people stopped reading there; they saw that “freeware” required special permission, and since OSS can often be downloaded for free, they presumed that all OSS was “freeware.” They should have kept reading, because it then goes on to make it clear that OSS is not freeware:

Such products are assessed for information assurance impacts, and approved for use by the DAA. The assessment addresses the fact that such software products are difficult or impossible to review, repair, or extend, given that the Government does not have access to the original source code and there is no owner who could make such repairs on behalf of the Government…

This latter part makes it clear that software only requires special treatment if the government cannot review, repair, or extend the software. If the government can do these things, there’s no problem, and by definition OSS provides these rights. But a lot of people didn’t understand this.

This was such a common misunderstanding that in October 2009, the DoD CIO’s memo “Clarifying Guidance Regarding Open Source Software (OSS)” specifically stated (in Attachment 2, 2c) that this was a misunderstanding (http://dodcio.defense.gov/sites/oss/2009OSS.pdf). The DoD CIO later instructed DISA to update the AppDev STIG so this misunderstanding would be removed.

The latest AppDev STIG (Version 3, Release 4) has just fixed this (http://iase.disa.mil/stigs/app_security/app_sec/app_sec.html). The new STIG:

Refers to the DoD OSS policy of 2009, instead of the old one.
Has better definitions for software types, including “OSS” and “commercial software”. Its old definitions caused problems for OSS use; the “commercial software” definition was even inconsistent with US law, the Federal Acquisition Regulation (FAR), and the DoD FAR Supplement (DFARS). In particular, it makes it clear that most OSS is commercial software as defined by law and regulation.
Makes it clear that special DAA approval is ONLY required if BOTH of the following are true: “(1) no source code to review, repair, and extend, and (2) limited or no warranty, but are required for mission accomplishment.” See checklist items (APP2090.1: CAT II) and (APP2090.2: CAT II). This is the big change.

Two related points:

Sadly, the AppDev STIG latest revision has a formatting glitch; all second-level headings aren’t numbered in the body, with the result that the table-of-contents numbers don’t match the body. Still, it has the updated technical content, and future versions will presumably fix the formatting.
The wording of DoDI 8500.2’s DCPD-1 has been confusing people for years (I hear that at least parts of NASA have also used this text, inheriting the same confusion). In the short term, the DoD CIO’s formal clarification should help. In the longer term, there is an effort to switch the DoD to a single set of federal information assurance controls defined in NIST Special Publication 800-53. Its equivalent control, SA-6(1), has much clearer text.

But the editorial gaff in the AppDev STIG, and the work on improving the wording of controls long term, shouldn’t detract from the main point.

The main point is:

Open Source Software (OSS) is now much easier to use in the DoD.

Open Source Award!

Sun, 06 Nov 2011 12:45 GMT

I’ve learned that Open Source for America (OSFA) has awarded me a 2011 Open Source Award - Individual Award for my work to advocate consideration of “open source software in the US Department of Defense (DoD)”. They specifically point to my papers Why Open Source Software / Free Software? Look at the Numbers! and Nearly all FLOSS is Commercial.

The winners of all the 2011 awards were:

Open Source Deployment in Government: (DHS) Science and Technology (S&T) Directorate‘s Homeland Open Security Technology (HOST) program. (I’m involved in this, too.)
Open Source Project: Open Source Geospatial Foundation (OSGeo) for its OpenLayers web mapping project.
Individual Awards: David A. Wheeler (me) and Melanie Chernoff

Thanks so much, OSFA! I’m honored.

Open Document Format 1.2 approved!

Tue, 04 Oct 2011 12:59 GMT

Hooray! Open Document Format for Office Applications (ODF or OpenDocument) Version 1.2 has been approved as an OASIS Standard. Finally, the world has a standard vendor-independent format for storing and exchanging ordinary office documents (particularly word processing documents, spreadsheets, and presentations) that you can edit.

Historically, people have only been able to exchange these documents if they use the same program, locking users into specific vendor products. In short, users often don’t really own the documents they create; they are often beholden to the developers of their tools. This is especially nasty for government documents; all governments have to choose some product, and whatever product they use implicitly forces their citizens to use the same product (whether they want to or not). Over time these documents can no longer be read, as products are upgraded or people change products, so this is also a disaster for archiving. We can read the Magna Carta more easily than some documents saved 30 years ago. Heck, we can read Sumerian writings more easily than some documents saved 30 years ago, and that is a scary thing. ODF provides us the possibility of actually exchanging documents, and reading archives, regardless of what program was used to create them. In short, people now have real freedom when they create and exchange normal office documents — they are no longer locked into a particular supplier or version of program.

Rob Weir has some of the highlights of version 1.2, and he has also written an overview of ODF.

For me, the highlight is OpenFormula. Previous versions of the specification could exchange spreadsheets, but did not standardize the format of recalculated formulas. I led the subcommittee in the ODF Technical Committee to nail down exactly how to recalculate formulas. The result: We now have a real spec. My sincere thanks to the many who helped make this possible. Feel free to see my 2011-05-28 post about the success of OpenFormula.

I’m sure that there will continue to be refinements for years to come; that is normal for a standard. In some sense this is after the fact; like many good standards, it was developed through the cooperation of many of the implementors. It is already implemented, at least in part, in many places, and I expect even more completed implementations soon.

The key, though, is that users can finally own the documents they create. That is a major step forward, for all of us.

Petition the White House to cease issuing software patents

Fri, 30 Sep 2011 09:49 GMT

I encourage all US citizens to sign this petition to the US White House to “direct the patent office to cease issuing software patents”. I believe software patents impede innovation (instead of helping it), and they have become a threat to the US economy. Many organizations involved in software are now spending lots of time fending off patent trolls, fighting patent lawsuits, or cannot safely solve problems due to patent thickets. The recently-passed “America Invents Act” (AIA) completely failed to deal with this fundamental problem.

Signing a petition won’t immediately solve anything. That’s not how it works. But repeatedly making the government aware that there’s a real problem is a good first step to solving a problem. In the US, the right of the people to petition their government is guaranteed by the first amendment of the US Constitution (“Congress shall make no law …. abridging… the right of the people peaceably to assemble, and to petition the Government for a redress of grievances”). Everyone is affected today by software, and so far the government has not effectively dealt with the problem. Please use this opportunity to make the government aware of a real problem.

Off-the-Shelf (OTS) Software Maintenance Strategies

Fri, 16 Sep 2011 18:04 GMT

Off-the-shelf (OTS) software is simply software that is ready-made and available for use. Even when you need a custom system, building it from many OTS components has many advantages, which is why everyone does it. OTS works because you can save money and time, increase quality, and increase innovation through resource pooling.

However, people can get easily confused by the many different ways that off-the-shelf (OTS) software can be maintained. Terminology varies, and there hasn’t been an obvious way to describe how these different approaches are related. In 2010 I chatted with several others about how to make this clearer, and then created a picture that I think clarifies things. My thanks to helpful critiques from Heather Burke and John Scott. So here’s the picture, followed by a discussion on what it means.

Off-the-Shelf (OTS) Maintenance Strategies

If OTS software is commercial, it’s commercial OTS (COTS) software. By U.S. law, any software is commercial if it is (1) sold, licensed, or leased to the public, and (2) has a non-governmental use. There are two kinds of COTS software: Open Source Software (OSS) and proprietary software. OSS, put briefly, is software whose licenses give users the freedom to run the program for any purpose, to study and modify the program, and to redistribute copies of either the original or modified program (without having to pay royalties to previous developers). Yes, practically all OSS is commercial.

OTS can also be retained and maintained internally by an organization. For example, the U.S. government develops and maintains some software internally. In the U.S. government world, such software often called government OTS (GOTS). This figure shows things from the point of view of the U.S. government, but if you work with some other organization, you can think of this figure with your organization in the place of the U.S. government. (Maybe this should be called “internal off-the-shelf” or “IOTS” instead!) The idea here is that any organization can have software that it controls internally, and view as internal OTS software, as well as the COTS software that is available to the public.

There are various reasons why the government should sometimes keep certain software in-house, e.g., because sole possession of the software gives the U.S. a distinct advantage over its adversaries. However, there is also considerable risk to the government if it tries to privately hold GOTS software within the government for too long. Technological advantage is usually fleeting. Often there is a commercially-developed item available to the public that begins to perform similar functions. As it matures, other organizations begin using this non-GOTS solution, potentially rendering the GOTS solution obsolete. Such cases often impose difficult decisions, as the government must determine if it will pay the heavy asymmetrical cost to switch, or if it will continue “as usual” with its now-obsolete GOTS systems (with high annual costs and limitations that may risk lives or missions).

Either COTS or GOTS may be maintained by a single maintainer or by a community. In community maintenance there is often a single organization who determines if proposals should be accepted, but the key here is that the work tends to be distributed among those affected. An Open GOTS (OGOTS) project is a GOTS project which uses multiple-organization collaborative development approaches to develop and maintain software, in a manner similar to OSS. Some people use the term “Government Open Source Software” (GOSS) instead of OGOTS (in particular, GOSS for Govies uses the term GOSS instead).

GOTS (including OGOTS) is basically a special case of “gated software” with development inside a government. However, governments are bigger than most companies, and (in democracies) they are supposed to serve all of their citizenry, and those factors make them rather different than most other gated communities. Community development of proprietary software (“gated software”) outside governments is less common, but it can happen (historically some parts of Unix were developed this way). The term Open Technology Development (OTD) involves community development among government users (in the case of government developers), and thus it includes both OSS and OGOTS (aka GOSS).

I should note that I have a broad view of maintenance. I’ve often said that there is only one program — “Hello, World” — and that the rest is maintenance. That’s overstated for effect, but I believe there is a lot of truth in that statement.

This figure, and some of the text above, is in section 1.3 of the paper Open Technology Development (OTD): Lessons Learned & Best Practices for Military Software (also available via MIL-OSS), which is released under the Creative Commons BY-SA license. If you’re interested in more, please see the paper! The figure and some of the text are also part of “Software is a Renewable Military Resource” by John Scott, Dr. David A. Wheeler, Mark Lucas, and J.C. Herz, Journal of Software Technology, February 2011, Vol. 14, Number 1.

I hope this figure makes it easier to understand the different approaches for maintaining off-the-shelf (OTS) software.

Ask not who holds the copyrights

Sat, 10 Sep 2011 11:13 GMT

Asking “who has the copyright?” for intellectual works (like software, documents, and data) is almost always the wrong question to ask. Instead, ask “what rights do I have (or can I get)?” and “do those rights let me do what I want to do?”. In a vast number of situations, those are the right questions to ask instead. Even people who should know better can fall into this subtle trap!

This became obvious to me when it was revealed that even the smart people at the Apache Software Foundation fell into this. In the recent Accumulo proposal, there were unnecessary copyright hurdles because Apache was unnecessarily asking for a copyright transfer, instead of the necessary rights (in this case, there was no copyright to transfer!).

So I’ve justed posted Ask Not Who Holds the Copyright, which I hope will clear this up.

MIL-OSS 2011

Sun, 04 Sep 2011 22:35 GMT

I recently went to the MIL-OSS (“military open source software”) 2011 Working Group (WG) / Conference in Atlanta, Georgia. Topics included the open prosthetics project, releasing government-funded software as OSS, replacing MATLAB with Python, the “Open Technology Dossier Protocol” (OTDP), confining users using SELinux, an explanation of DoD policies on OSS, Charlie Schweik’s study on what makes a success OSS project, and more. Some people started developing a walkie-talkie Android app at the conference. Here’s a summary of the conference, if you’re curious.

First, a few general comments. If this conference is any guide, it is slowly getting easier to get OSS into government (including military) systems. OSS is already used in many places, but it’s often “don’t ask, don’t tell”, and there are still lots of silly bureaucratic barriers that prevent the use of OSS where it should be used or at least considered. But there were many success stories, with slide titles like “how we succeeded”.

Although the conference had serious purposes, it was all done in good humor. All participants got the MIL-OSS poster of Uncle Sam (saying “I want YOU to Open Source!”). The theme of the conference was the WarGames movie; the first finder for each of the WarGames Easter eggs would get a silly 80s-style prize (such as an Atari T-shirt).

As the MIL-OSS 2011 presentations list shows, I gave three talks:

Publicly Releasing Open Source Software (OSS) Developed for the U.S. Government. This presentation explained when the government or contractors can publicly release software, as open source software, if it was developed using U.S. government funds. This presentation summarized my paper Publicly Releasing Open Source Software Developed for the U.S. Government (also see Kane McLean’s one-page summary of this paper, the “OSS Releasability Quick Reference”, which was given to every conference participant). I think this is an important topic. Billions of dollars go into developing software, yet most of the time, the taxpayers (who paid for it) don’t get the benefits. It turns out that this software often can be released; this is the decoder ring for these Byzantine rules. This can have incredible benefits. For example, the DoD funded the work that created the Internet, and then released as OSS an implementation of its key TCP/IP protocols. The Internet has mightily benefitted the DoD, in fact, it’s benefitted the whole world. (And yes, it had the required WarGames Easter egg. Slide 15 says “Talk to others who have experience with OSS” — the egg is in the supporting bullet, “Q: What is it doing? A: It’s learning!”)
Why the GPL Might not Destroy the Universe. This tongue-in-cheek talk tries to counter some of the silly, over-the-top fears about the GNU General Public License (GPL). I figure any presentation can’t be bad if it includes photos of Godzilla, flying saucers, zombies, and a poster saying “If you program open source, you’re programming COMMUNISM!”.
HOST Lessons Learned (with Tom Dunn). This summarized interviews of various people on the roadblocks to using or developing open technology (including open source software) in the government.

The conference was complicated by the recent passing of Hurricane Irene. The area itself was fine, but some people had trouble flying in. The first day’s whole schedule was delayed so speakers could arrive (using rescheduled flights). That was probably the best thing to do in the circumstance — it was basically like a temporary time zone change — but it meant that one of my talks that day (Why the GPL Might not Destroy the Universe) was at 9:10pm. And I wasn’t even the last speaker. Eeeek. Around 15 speakers had still not arrived when the conference arrived, but all but one managed to get there before they had to speak.

Here are few notes on the talks:

Andy Henshaw (GTRI) spoke on “Replacing MATLAB: Python Tools for Scientists and Engineers”. His basic point is that “Python is a good replacement for MATLAB in a lot of cases”. Although Python isn’t fast by itself, it’s often useful as a glue, with the intensive data-handling being done by hand-crafted libraries. He focused on (and discussed) the libraries NumPy, SciPy, matplotlib, and ipython. He also discussed differences between MATLAB and Python for MATLAB users. In Matlab, the basic type is a matrix, it uses 1-based indexing, ‘*’ means matrix multiplication, and function calls use pass-by-value with lazy copy-on-write. In contrast, in Python with libraries like these, the basic type is a multidimensional array, it uses 0-based indexing, ‘*’ means element-wise multiplication (use dot() for matrix multiplication or use the matrix class), and function calls use pass-by-reference.
I learned interesting things about AdaCore (who make GNAT pro, SPARK Pro, and Code Peer). They don’t have a separate support organization — their engineers provide support directly, since support is really what they sell.
Maj Wilson/Kane McLean discussed changing culture. They argued that the mind has two independent decision-making functions that work simultaneously: the emotional mind and the rational mind. The emotional mind is like an elephant; it’s illogical and determined, emphasizes getting stuff done, and has mental “muscle memory”. The rational mind is like a jockey; it’s logical and reasoned, emphasizes organization but often can’t “get off the saddle”, and does long-term / strategic planning. You need to convince both, so you should try to shrink the change, shape a clear path forward, and repeat what works. They believe that culture change in a big bureaucracy happens from both the top (the “clouds”) and the bottom (the “grass roots”); resistance often comes from the middle. The solution for change, then, is to “seed clouds” and “grow the grass”.
The “Open Technology Dossier Protocol” (OTDP) was pitched by Winston Messer and Nick Bollweg. Basically, they’d like every OSS project to put, on their web site, a small XML file that would let various search systems learn more about their project. That way, each project can update their own information.
David Egts (Red Hat) explained “SELinux user confinement” - a new capability in RHEL 6 to easily confine users using SELinux. Just install the “policycoreutils-python” package, which includes the semanage tool that lets you control much more precisely what specific users may do.
Alex S. Voultepsis explained how the intelligence community (IC) has built up an internal infrastructure with the tools that people want to use; in a vast number of cases, they use OSS to do this. For example, Intellipedia is implemented using MediaWiki, the same software that runs Wikipedia.
Dan Risacher discussed the DoD Oct 16, 2009 memo on open source software. He noted that we have a “Government IP knot”: “Government rules are designed to enable a program manager to control their program, not to enable sharing it”. A way to cut this knot is to make it clear that the software will be released as OSS; then everyone knows what the rules are. He wants to be a “developer advocate” - the DoD needs to be able to innovate faster than its opponents.
John Kuniholm presented on the “Open Prosthetics Project”. He is missing part of an arm, and explained some of the complications of making prosthetics. A key need is really good open source CAD tools. That is a general problem, not unique to the military or government — currently the tools are hideously expensive, and until that changes, the promise of cheap 3D printers will be harder to realize.
Charlie Schweik has been doing a lot of quantitative studies of OSS projects, to determine what separates successful projects from abandoned projects. He expects to have a book on soon on this topic! In the initiation stage, the key factors were: Leadership by doing, clear vision, and well-articulated goals. Other factors were Project marketing; project financing; knowledge continuity; being a multideveloper project. A really key factor, once a project is initiated, is gaining a developer (and then gaining more later). There are many conflicting claims, e.g., some say that smaller groups are better (Brooks), that larger groups are better (Linus’ law), or that size doesn’t matter; his data shows that Linus’ law is the correct one. Face-to-face communication doesn’t seem to be as important as it used to be, due to better communication technology. He’s gathered lots more info; I’m looking forward to seeing the whole thing.
One great thing was that everyone was motivated to actually solve problems, immediately. There is already an official DoD Open Source Software (OSS) Frequently-Asked Questions (FAQ), but there’s a need for a less-official FAQ, so during the conference a new MIL-OSS OSS FAQ was created. On the last day there was a discussion between various software developers and military folks, particularly about military needs. A real problem in military situations — and disasters like hurricanes — is that centralized communications systems fail. Within a short time, people were suddenly developing an OSS walkie-talkie application for Android and hosting it on github.

Many discussions revolved around the problems of getting authentication/authorization working without passwords, in particular using the ID cards now widely used by nearly all western governments (such as DoD CAC cards). Although things can work sometimes, it’s incredibly painful to get them to work on any system (OSS or not), and they are fragile. Dmitri Pal (Red Hat)’s talk “CAC and Kerberos From Vision to Reality” discussed some of the problems and ways to possibly make it better. The OpenSSH developers are actively hostile to the X.509 standard that everyone uses for identity certificates; I agree with the OpenSSH folks that X.509 is clunky, but that is what everyone uses, and not supporting X.509 means that openssh is useless for them. Every card reader is incompatible with the others, so every time a new model comes out, drivers have to be written and it often doesn’t work anyway (compare that to USB keyboards, which “just work” every time even through KVM switches). I think some group needs to be formed, maybe a “Simple Authorization without passwords” group, with the goal of setting standards and building OSS components so that systems by default (maybe by installing one package) can trivially use PKI and other systems and have it “just work” every time. No matter that client, server (relying party), or third-party authenticator/authorization server is in use.

If you’re interested in more of my personal thoughts about OSS and the U.S. Department of Defense (DoD), also see FLOSS Weekly #160, the interview of David A. Wheeler by Randal Schwartz and Simon Phipps. Good general sites for more info are the MIL-OSS website and the DoD CIO Free Open Source Software (FOSS) site.

There’s more to be done, but a lot is already happening.

GNOME 3 Shell is terrible, I am switching to XFCE

Wed, 03 Aug 2011 18:31 GMT

I’m a long-time user of Fedora and GNOME. GNOME 2 has served me well over the years, so I was interested in what the GNOME people were cooking for GNOME 3. Fedora 15 comes with the new GNOME 3 shell; since change can sometimes be good, I’ve tried to give the new GNOME 3 shell a fair trial.

But after giving GNOME 3 (especially GNOME shell) some time, I’ve decided that I hate the GNOME 3 shell as it’s currently implemented. It’s not just me; the list of people who have complaints about the GNOME 3 shell include Linus Torvalds, Dedoimedo (see also here), k3rnel.net (Juan “Nushio” Rodriguez), Martin Sourada, junger95, and others. LWN noted the problems of GNOME 3 shell way back. So many people are leaving GNOME 3, often moving to XFCE, that one person posted a poem titled “GNOME 3, Unity, and XFCE: The Mass Exodus”.

The GNOME 3 shell is beautiful. No doubt. But as far as I can tell, the developers concentrated on making it beautiful, cool and different, but as a consequence made it far less useful and efficient for people. Dedoimedo summarizes GNOME 3.0 shell as, “While it’s a very pretty and comely interface, it struck me as counterproductive, designed with a change for the sake of change.” In a different post Dedoimedo says, “Gnome 3 is a toy. A beautiful, aesthetic toy. But it is not a productivity item… I am not a child. My computer is not a toy. It’s a serious tool… Don’t mistake conservative for inefficient. I just want my efficiency.”.

Some developers have tried to fix its worst problems of GNOME 3 shell with extensions, and if GNOME developers work at it, I think they could change it into something useful. But most of these problems aren’t just maturity issues; GNOME 3 shell is broken by design. So I’m going to switch to XFCE so I can get work done, and perhaps try it again later if they’ve started to fix it. Thankfully, Fedora 15 makes it easy to switch to another desktop like XFCE, so I can keep on happily using Fedora.

So what’s wrong?

I’ve been trying to figure out why I hate GNOME 3 so much, and it comes down to two issues: (1) GNOME 3’s shell makes it much harder to do simple, common tasks, and (2) GNOME 3 shell often hides how to do tasks (it’s not “discoverable”). These are not just my opinions, lots of people say these kinds of things. k3rnel.net says, “Gnome’s ‘Simplicity’ is down right insulting to a computer enthusiast. It makes it impossible to do simple tasks that used to flow naturally, and it’s made dozens of bizarre ‘design decisions’, like hiding Power Off behind the ‘Alt’ key.” Let me give you examples of each of these issues.

First of all, GNOME 3 (particularly its default GNOME shell) creates a lot of extra steps and work to do simple tasks that used to be simpler. To run a program whose name you don’t know, you have go to the far top left to the hot spot (or press “LOGO”), move your mouse to the hideously hard-to-place (and not obvious) “Applications” word slightly down the right, then mouse to the far right to choose a category, then mouse back to choose it. That’s absurd; the corners of the screen are especially easy to get to, and they fail to use that fact when choosing non-favorite applications. Remarkably, there doesn’t seem to be a quick way to simply show the list of (organized) applications you can start; there’s not even a keyboard shortcut for “LOGO Applications”. Eek. This is a basic item; even Windows 95 was easier. Would it really have killed anyone to make moving to some other area (say, the bottom left corner) show the applications? And why are the categories on the far right, where they are least easy to get to and not where any other system puts them? (Yes, the favorites area lets you start some programs, but you have to find it the first time, and some of us use many programs.) Also, you can’t quickly alt-tab between arbitrary windows (Alt-tab only moves between apps, and the undiscoverable alt-` only moves between windows of the same app). GNOME shell makes it easy to do instant messaging, but it makes it harder to do everything else. Fail.

GNOME 3’s capabilities are not discoverable, either. To log off or change system settings you click on your name — and that’s already non-obvious. But it gets worse. To power off, you click on your name, then press the ALT key to display the power off option, then select it. How the heck would a normal user find out about this? The only obvious way to power down the system is to log out, then power off from the front. If you know an application name, pressing LOGO (aka WINDOWS) and typing its name is actually a nice feature, but that is not discoverable either. If you want a new process or window (like a new terminal window or file manager window), you have to know press control when you select its icon to start a new process (for Terminal, you can also start it and press shift+control+N, but that is not true for all programs). The need to press control isn’t discoverable (it’s also a terrible default; if I press a program icon, I want a new one; if I wanted an existing one I’d select its window instead). Fail.

There are some nice things about GNOME 3 shell. As I mentioned earlier, I like the ability to press LOGO start typing a program name (which you can then select) - that is nice. But even then, this is not discoverable; how would a user new to the interface know that they should press the LOGO button? This functionality is trivial to get in other desktop environments; I configured XFCE to provide the same functionality in less than a minute (in a way that is less pretty, but much easier for a human to use).

The implementors seem to think that new is automatically better. Rediculous. I don’t use computers to have the newest fad interface, I use them to get things done (and for the pleasure of using them). I will accept changes, but they should be obvious improvements. Every change just for its own sake imposes relearning costs, especially for people like me who use many different computers and interfaces, and especially if they make common operations harder. Non-discoverability is especially nasty; people don’t want to read manuals for basic GUI interfaces, they want to get things done.

I don’t think GNOME 3 is mature, either. For example, as of 2011-07-28, GNOME 3 does not support screensavers — it just shows a blank screen after a timeout. But the previous GNOME 2 had screensavers. Heck, Windows 3.0 (of 1993) did better than that; it had screensavers, and I’m sure there were screensavers before then.

I’ve tried to get used to it, because I wanted to give new ideas a chance. Different can be better! But so far, I’m not impressed. The code may be cleaner, and it may be pretty, but the user experience is currently lousy.

If you’re stuck using the GNOME 3 Shell, you basically must read the GNOME shell cheat sheet, because so much of what it does is un-intuitive, incompatible with everything else, and non-discoverable. Needing to read a manual to use a modern user interface is not a good sign.

You could try switching to the GNOME 3 fallback mode, as discussed by Dedoimedo and others. This turns on a more tradtional interface. Several people have declared that GNOME 3 fallback is better than GNOME shell itself. But I was not pleased; it’s not really well-supported, and it’s really not clear that this will be supported long term.

You can also try various tweaks, configurations, and additional packages to make GNOME 3 shell more tolerable. If you’re stuck with GNOME 3 shell, install and use gnome-tweak-tool; that helps. You should also install the Fedora gnome-shell-extensions-alternative-status-menu package, which lets you see “Power off” as an option.

But after trying all that, I decided that it’d be better to switch to another more productive and mature desktop environment. Obvious options are XFCE and KDE.

XFCE is a lightweight desktop environment, and is what I’ve chosen to use instead of the default GNOME 3 shell. I found out later that other people have switched to XFCE after trying and hating GNOME 3’s shell. XFCE doesn’t look as nice as GNOME 3, indeed, the GNOME 3 shell is really quite flashy by comparison. But the GNOME shell makes it hard for me to get anything done, and that’s more important.

I expect that it wouldn’t be hard for the developers to make it better; hopefully the GNOME folks will work to improve it. If many of GNOME 3’s problems are fixed, then I’ll be happy to try it again. But I’m in no hurry; XFCE works just fine.

I’m creating a new page on my website called Notes on Fedora so that I can record “how to” stuff, in case that others find it useful. For example, I’ve recorded how to turn on some stuff in XFCE to make it prettier. Enjoy!

Upcoming FLOSS in Government Conferences for 2011

Wed, 03 Aug 2011 16:01 GMT

If you’re interested in free/libre/open source software in government (particularly the U.S. federal government), there are two upcoming conferences you should consider.

One is Government Open Source Conference (GOSCON) 2011 on August 23, 2011. It will be held at the Washington Convention Center, Washington, DC.

The other is the Military Open Source Software (MIL-OSS) WG3 conference on August 30 - September 1, 2011. It will be held in Atlanta, Georgia.

I’ll be speaking at both. But don’t let that dissuade you :-).

Microsoft, co-author of the Linux kernel

Thu, 14 Jul 2011 19:56 GMT

Truth is often stranger than fiction. Microsoft was the fifth-largest corporate contributor to the Linux kernel version 3.0.0, as measured by the number of changes to its previous release. Only Red Hat, Intel, Novell, and IBM had more contributions. Microsoft was #15 as measured by number of lines changed, which is smaller but is still an impressively large number.

This work by Microsoft was to clean up the “Microsoft Hyper-V (HV) driver” so that the Microsoft driver would be included in the mainline Linux kernel. Microsoft originally submitted this set of code changes back in July 2009, but there were a lot of problems with it, and the Linux kernel developers insisted that it be fixed. The Linux community had a long list of issues with Microsoft’s code, but the good news is that Microsoft worked to improve the quality of its code so that it could be accepted into the Linux kernel. Other developers helped Microsoft get their code up to par, too. ( Steve Friedl has some comments about its early technical issues.) There’s something rather amusing about watching Microsoft (a company that focuses on software development) being forced by the Linux community to improve the quality of Microsoft’s code. Anyone who thinks that FLOSS projects (which typically use widespread public peer review) always produce lower quality software than proprietary vendors just isn’t watching the real world (see my survey paper of quantitative FLOSS studies if you want more on that point). Peer review often exposes problems, so that they can be fixed, and that is what happened here.

Microsoft did not do this for the sheer thrill of it. Getting code into the mainline Linux kernel release, instead of just existing as a separate patch, is vitally important for an organization if they want people to use their software (if it needs to be part of the Linux kernel, as this did). A counter-example is that the Xen developers let KVM zoom ahead of them, because the Xen developers failed to set a high priority on getting full support for Xen into the mainline Linux kernel. As Thorsten Leemhuis at The H says, “There are many indications that the Xen developers should have put more effort into merging Xen support into the official kernel earlier. After all, while Xen was giving developers and distribution users a hard time with the old kernel, a new virtualisation star was rising on the open source horizon: KVM (Kernel-based Virtual Machine)… In the beginning, KVM could not touch the functional scope and speed of Xen. But soon, open source developers, Linux distributors, and companies such as AMD, Intel and IBM became interested in KVM and contributed a number of improvements, so that KVM quickly caught up and even moved past Xen in some respects.” Xen may do well in the future, but this is still a cautionary tale.

This doesn’t mean that Microsoft is suddenly releasing all its programs as free/libre/open source software (FLOSS). Far from it. It is obvious to me that Microsoft is contributing this code for the same reason many companies contribute to the Linux kernel and other FLOSS software projects: Money.

I think it is clear that Microsoft hopes that these changes to Linux will help Microsoft sell more Windows licenses. These changes enable Linux to run much better (e.g., more efficiently) on top of Microsoft Windows’ hypervisor (Hyper-V). Without them, people who want to run Linux on top of a hypervisor are much more likely to use products other than Microsoft’s. Microsoft doesn’t want to be at a competitive disadvantage in this market, so to sell its product, it chose to contribute changes to the Linux kernel. With this change, Microsoft Windows becomes a more viable option as a host operating system, running Linux as a guest.

Is this a big change? In some ways it is not. Microsoft has developed a number of FLOSS packages, such as WiX (for installing software on Windows), and it does all it can to encourage the development of FLOSS that run on Windows.

Still, it’s something of a change for Microsoft. Microsoft CEO Steve Ballmer stated in 2001 that Linux and the GNU GPL license were “a cancer”. This was in many ways an attack on FLOSS in general; the GNU GPL is the most popular FLOSS license by far, and a MITRE report found that the “GPL sufficiently dominates in DoD applications for a ban on GPL to closely approximate a full ban of all [FLOSS]”. This would have been disastrous for their customer, because MITRE found that FLOSS software “plays a far more critical role in the [Department of Defense] than has been generally recognized”. I think many other organizations would say the same. This is not even the first time Microsoft has gotten involved with the GPL. Microsoft sold Windows Services for Unix (SFU), which had GPL software, showing that even Microsoft understood that it was possible to make money while using the GPL license. But this more case is far more extreme; in this case Microsoft is actively helping a product (the Linux kernel) that it also competes with. I don’t expect Microsoft to keep contributing significantly to the Linux kernel, at least for a while, but that doesn’t matter; here we see that cash trumps ideology. More generally, this beautifully illustrates collaborative development: Anyone can choose to work on specific areas of a FLOSS program, for their own specific or selfish reasons, to co-produce works that help us all.

U.S. government must balance its budget

Mon, 04 Jul 2011 18:54 GMT

(This is a blog entry for U.S. citizens — everyone else can ignore it.)

We Americans must demand that the U.S. government work to balance its budget over time. The U.S. government has a massive annual deficit, resulting in a massive national debt that is growing beyond all reasonable bounds. For example, in just Fiscal Year (FY) 2010, about $3.4 trillion was spent, but only $2.1 trillion was received; that means that the U.S. government spent more than a trillion dollars more than it received. Every year that the government spends more than it receives it adds to the gross federal debt, which is now more than $13.6 trillion.

This is unsustainable. The fact that this is unsustainable is certainly not news. The U.S. Financial Condition and Fiscal Future Briefing (GAO, 2008) says, bluntly, that the “Current Fiscal Policy Is Unsustainable”. “The Moment of Truth: Report of the National Commission on Fiscal Responsibility and Reform” similarly says “Our nation is on an unsustainable fiscal path”. Many others have said the same. But even though it’s not news, it needs to be yelled from the rooftops.

The fundamental problem is that too many Americans — aka “we the people” — have not (so far) been willing to face this unpleasant fact. Fareed Zakaria nicely put this in February 21, 2010: “ … in one sense, Washington is delivering to the American people exactly what they seem to want. In poll after poll, we find that the public is generally opposed to any new taxes, but we also discover that the public will immediately punish anyone who proposes spending cuts in any middle class program which are the ones where the money is in the federal budget. Now, there is only one way to square this circle short of magic, and that is to borrow money, and that is what we have done for decades now at the local, state and federal level … The lesson of the polls in the recent elections is that politicians will succeed if they pander to this public schizophrenia. So, the next time you accuse Washington of being irresponsible, save some of that blame for yourself and your friends”.

But Americans must face the fact that we must balance the budget. And we must face it now. We must balance the budget the same way families balance their budgets — the government must raise income (taxes), lower expenditures (government spending), or both. Growth over time will not fix the problem.

How we rellocate income and outgo so that they match needs to be a political process. Working out compromises is what the political process is supposed to be all about; nobody gets everything they want, but eventually some sort of rough set of priorities must be worked out for the resources available. Compromise is not a dirty word to describe the job of politics; it is the job. In reality, I think we will need to both raise revenue and decrease spending. I think we must raise taxes to some small degree, but we can’t raise taxes on the lower or middle class much; they don’t have the money. Also, we will not be able to solve this by taxing the rich out of the country. Which means that we must cut spending somehow. Just cutting defense spending won’t work; defense is only 20% of the entire budget. In contrast, the so-called entitlements — mainly medicare, medicaid, and social security — are 43% of the government costs and rapidly growing in cost. I think we are going to have to lower entitlement spending; that is undesirable, but we can’t keep providing services we can’t pay for. The alternative is to dramatically increase taxes to pay for them, and I do not think that will work. Raising the age before Social Security benefits can normally be received is to me an obvious baby step, but again, that alone will not solve the problem. It’s clearly possible to hammer out approaches to make this work, as long as the various camps are willing to work out a compromise.

To get there, we need to specify and plan out the maximum debt that the U.S. will incur in each year, decreasing that each year (say, over a 10-year period). Then Congress (and the President) will need to work out, each year, how to meet that requirement. It doesn’t need to be any of the plans that have been put forward so far; there are lots of ways to do this. But unless we agree that we must live within our means, we will not be able to make the decisions necessary to do so. The U.S. is not a Greece, at least not now, but we must make decisions soon to prevent bad results. I am posting this on Independence Day; Americans have been willing to undergo lots of suffering to gain control over their destinies, and I think they are still able to do so today.

In the short term (say a year), I suspect we will need to focus on short-term recovery rather than balancing the budget. And we must not default. But we must set the plans in motion to stop the runaway deficit, and get that budget balanced. The only way to get there is for the citizenry to demand it stop, before far worse things happen.